Method and system for generating a graphic overlay in a surround view of a vehicle

Abstract

The disclosure relates to system (100) and method (200) for generating a graphic overlay (308) in a vehicle's surrounding (304). The system (100) comprises processing circuitry (130) configured to receive visual data for vehicle's surrounding (304) and process the visual data using a semantic segmentation model to obtain information on the vehicle's surrounding (304). Further, the processing circuitry (130) is configured to classify the visual data into one or more environment objects (306) in the vehicle's surroundings (304) and generate in real time, the graphic overlay (308) based on the classified visual data into the one or more environment objects (306). Elected for publication:

Description

[0001] 202305531

[0002] 1

[0003] METHOD AND SYSTEM FOR GENERATING A GRAPHIC OVERLAY IN A SURROUND VIEW OF A VEHICLE

[0004] TECHNICAL FIELD

[0005] The present disclosure in general relates to graphic overlay. More particularly, the present disclosure relates to a method and a system for generating a graphic overlay in a surround view of a vehicle.

[0006] BACKGROUND

[0007] In current surround view systems used in vehicles, environment information of vehicle is often not taken into account during surround view rendering, such as parking slots and overlays. This omission may lead to overlays being displayed incorrectly, such as on obstacles or other vehicles, providing inaccurate and misleading information to a driver of the vehicle. These inaccuracies not only pose a safety risk by giving false cues for manoeuvring but also disrupt the aesthetics of the surround view visualization. For example, parking slots or overlays may appear incorrectly positioned on obstacles or off the ground due to limitations in projection behaviour, eroding the driver's confidence in automated parking systems.

[0008] Conventional approaches to overlay rendering are typically hardcoded and lack adaptability to real world conditions with obstacles. The parameters used for overlay generation are often fixed and do not dynamically adjust based on the vehicle’s environment. While conventional methods may perform acceptably in ideal conditions, they fail in realistic scenarios where obstacles like other vehicles are present. Overlays may be drawn directly on these obstacles, creating visual confusion and hindering safe parking or driving manoeuvres. Moreover, rendering distant parking slots as if they float above the ground further diminishes the usability and trust in the system.

[0009] Additionally, conventional methods involve computationally intensive 2D to 3D conversions or frequent modifications to overlay parameters, which vary for each type of overlay. Hence, conventional approaches require significant software changes and computational resources, making them inefficient for practical deployment.202305531

[0010] 2

[0011] Thus, there is a need for an improved solution that may solve the aforementioned problems of conventional graphic overlay techniques.

[0012] SUMMARY

[0013] Though graphic overlay techniques are widely known, however, the existing solutions fail to intelligently adapt overlay rendering based on environment inputs, ensuring accurate, reliable, and visually coherent visualizations to enhance driver confidence and safety.

[0014] Therefore, there is a need for an improved method and system for generating a graphic overlay.

[0015] It is therefore an object of the present disclosure to provide a method and a system for generating a graphic overlay in a surround view of a vehicle.

[0016] This and other objects are achieved by means of a system and a method defined in the appended claims. The term exemplary is in the present context to be understood as serving as instance, example, or illustration.

[0017] According to an aspect of the present disclosure, a method for generating a graphic overlay in a surround view of a vehicle is provided. The method comprises of receiving visual data for vehicle’s surrounding. Further, the method comprises of processing the visual data using a semantic segmentation model to obtain information on the vehicle’s surrounding, wherein the processing comprises classifying the visual data into one or more environment objects in the vehicle’s surroundings. In addition, the method comprises of generating in real time, the graphic overlay based on the classified visual data into the one or more environment objects.

[0018] Optionally, the method comprises of segregating the visual data based on a pixel value associated with each pixel of the visual data. The visual data is labelled into one or more predefined classes to classify the visual data into one or more environment objects. Further, the method comprises of producing an outline of the classified visual data into the one or more environment objects. In addition, the method comprises of detecting an intersection point between the generated graphic overlay and the outline of the classified visual data. Furthermore, the method comprises of displaying the graphic202305531

[0019] 3

[0020] overlay based on the intersection point, wherein the intersection point defines boundaries for the generated graphic overlay.

[0021] Optionally, the method comprises of obtaining an outline of a free space region from the produced outline of the classified visual data. In addition, the method comprises of masking the graphic overlay using one or more computer graphic techniques based on the outline of the free space region.

[0022] Optionally, the graphic overlays comprise one or more of: parking guidelines, trajectory lines, reverse guidelines, dynamic parking lines, and / or distance markers.

[0023] Optionally, the semantic segmentation model comprises a deep learning model trained on a dataset on images of vehicle’s surround view in one or more classes.

[0024] Optionally, the information on the vehicle’s surrounding comprises data on location and shape of obstacles, free space regions, and drivable path.

[0025] Optionally, the one or more computer graphic techniques comprises one or more of: a stencil based technique, a scissor testing technique, a shader based masking technique, and a clipping planes technique.

[0026] According to another aspect of the present disclosure, a system for generating a graphic overlay in a surround view of a vehicle is provided. The system comprises processing circuitry configured to receive visual data for vehicle’s surrounding and process the visual data using a semantic segmentation model to obtain information on the vehicle’s surrounding. The processing of visual data comprises classifying the visual data into one or more environment objects in the vehicle’s surroundings. Furthermore, the processing circuitry is configured to generate in real time, the graphic overlay based on the classified visual data into the one or more environment objects.

[0027] Optionally, the processing circuitry is configured to segregate the visual data based on a pixel value associated with each pixel of the visual data, wherein the visual data is labelled into one or more predefined classes to classify the visual data into one or more environment objects. Further, the processing circuitry is configured to produce an outline of the classified visual data into the one or more environment objects. In addition, the processing circuitry is configured to detect an intersection point between202305531

[0028] 4

[0029] the generated graphic overlay and the outline of the classified visual data. The processing circuitry is further configured to display the graphic overlay based on the intersection point, wherein the intersection point defines boundaries for the generated graphic overlay.

[0030] Optionally, the processing circuitry is configured to obtain an outline of a free space region from the produced outline of the classified visual data. In addition, the processing circuitry is configured to mask the graphic overlay using one or more computer graphic techniques based on the outline of the free space region.

[0031] Optionally, the graphic overlays comprise one or more of: parking guidelines, trajectory lines, reverse guidelines, dynamic parking lines, and / or distance markers.

[0032] Optionally, the semantic segmentation model comprises a deep learning model trained on a dataset on images of vehicle’s surround view in one or more classes.

[0033] Optionally, the information on the vehicle’s surrounding comprises data on location and shape of obstacles, free space regions, drivable path.

[0034] According to another aspect of the present disclosure, there is provided a computer program when loaded and run on a system, causes processing circuitry of the system to perform corresponding steps of method for generating a graphic overlay in a surround view of a vehicle.

[0035] According to another aspect of the present disclosure, there is provided a computer readable medium having stored thereon a computer program.

[0036] Some embodiments disclosed herein have one or more of the following advantages:

[0037] - The proposed system and method provide accurate and reliable surround view information to the driver of the vehicle, ensuring safer vehicle manoeuvres.

[0038] - The proposed system and method efficiently deploy and adapts overlay rendering dynamically based on one or more environment objects using semantic segmentation techniques, thereby improving user experience.202305531

[0039] 5

[0040] - The proposed solution may be easily implemented in current generation Surround View Systems (SVS) without requiring computationally intensive 2D to 3D conversions.

[0041] - The proposed system and method support diverse overlay geometries and parameter configurations, accommodating a wide range of overlay types without the need for extensive software modifications.

[0042] - In addition, the proposed system and method may be used in plurality of use cases in the automotive industries. For example, the proposed system and method can be applied to Parking Assistant System, Heads Up Displays (AR Displays), Collision Warnings, and for navigation purposes etc.

[0043] - The proposed system and method display distant parking slots accurately aligned with the ground thereby enhancing the usability and increasing trust in the proposed system and method.

[0044] Other advantages may be readily apparent to one having skill in the art. Certain embodiments may have none, some, or all of the recited advantages.

[0045] BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The foregoing will be apparent from the following more particular description of the example embodiments, as illustrated in the accompanying drawings in which like reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the example embodiments.

[0047] FIG. 1 illustrates a block diagram of a system for generating a graphic overlay in a surround view of a vehicle, according to some embodiments herein;

[0048] FIG. 2 illustrates a flowchart of a method for generating the graphic overlay in the surround view of the vehicle, according to some embodiments herein;

[0049] FIG. 3A and 3C illustrates scenarios of generating the conventional graphic overlay in the surround view of the vehicle by conventional techniques;

[0050] FIG. 3B and 3D illustrates exemplary scenarios of generating the graphic overlay in the surround view of the vehicle, according to some embodiments herein;202305531

[0051] 6

[0052] FIG. 4 illustrates additional details of the method for generating the graphic overlay in the surround view of the vehicle, according to some embodiments herein;

[0053] FIG. 5A and 5B illustrates exemplary scenarios of generating the graphic overlay in the surround view of the vehicle using one or more computer graphic techniques, according to some embodiments herein;

[0054] FIG. 6A and 6B illustrates series of images showcasing a behaviour and positioning of the graphic overlay lines on a road scene, according to some embodiments herein; and FIG. 7 illustrates an example computing environment implementing the system, as shown in FIG. 1 , generating the graphic overlay in a surround view of a vehicle, according to some embodiments herein.

[0055] DETAILED DESCRIPTION

[0056] Aspects of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings. The systems and methods disclosed herein can, however, be realized in many different forms and should not be construed as being limited to the aspects set forth herein. Like numbers in the drawings refer to like elements throughout.

[0057] The terminology used herein is for the purpose of describing particular aspects of the disclosure only and is not intended to limit the invention. It should be emphasized that the term “comprises / comprising” when used in this specification is taken to specify the presence of stated features, integers, steps, or components, but does not preclude the presence or addition of one or more other features, integers, steps, components, or groups thereof. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0058] Embodiments of the present disclosure will be described and exemplified more fully hereinafter with reference to the accompanying drawings. The solutions disclosed herein can, however, be realized in many different forms and should not be construed as being limited to the embodiments set forth herein.

[0059] It will be appreciated that when the present disclosure is described in terms of a system and a method, it may also be embodied in one or more processors and one or more memories coupled to the one or more processors, wherein the one or more memories202305531

[0060] 7

[0061] store one or more programs that perform the steps, services and functions disclosed herein when executed by the one or more processors.

[0062] As mentioned there remains a need to provide an improved solution for generating graphic overlay in a surround view of a vehicle. Accordingly, the present disclosure provides a method and a system for generating the graphic overlay in a surround view of a vehicle. The system of the present disclosure provides accurate and reliable surround view information to the driver of the vehicle, ensuring safer vehicle manoeuvres.

[0063] FIG. 1 illustrates a block diagram of a system 100 for generating a graphic overlay (discussed later in FIG. 3B and 3D) in a surround view of a vehicle (not shown), according to some embodiments herein. The system 100 may be mounted in the vehicle (not shown) and may be configured to facilitate generation of the graphic overlay (discussed later in FIG. 3B and 3D) in a surround view of the vehicle when the vehicle is moving. The system 100 comprises one or more sensors 110, a global positioning system (GPS) 120, and processing circuitry 130. The processing circuitry 130 is configured to receive visual data for vehicle’s surrounding and process the visual data using a semantic segmentation model to obtain information on the vehicle’s surrounding. The processing of visual data comprises classifying the visual data into one or more environment objects (discussed later in FIG. 3B and 3D) in the vehicle’s surroundings. Furthermore, the processing circuitry 130 is configured to generate in real time, the graphic overlay (discussed later in FIG. 3B and 3D) based on the classified visual data into the one or more environment objects (discussed later in FIG. 3B and 3D). The one or more sensors 110 are configured to provide visual data for vehicle’s surrounding. In an example, the one or more sensors 110 may comprise a fisheye camera arranged to capture visual data for vehicle’s surrounding. In another example, the graphic overlay may be generated in real time on a live feed taken from the fisheye camera and displayed on a display 150 inside the vehicle. In an example, the fisheye camera may be positioned to a rear end of the vehicle. In another example, the fisheye camera may be positioned to a front end of the vehicle. In an example, the graphic overlay may be a visual element superimposed on a display or screen to provide additional information, guidance, or interaction.202305531

[0064] 8

[0065] In an example, the processing circuitry 130 may be coupled to a memory 140 for storing visual data of vehicle’s surrounding. In addition, the processing circuitry 130 may be communicatively coupled to the one or more sensors 110, the GPS 120, and the display 150.

[0066] In some embodiments, the processing circuitry 130 is configured to segregate the visual data based on a pixel value associated with each pixel of the visual data, wherein the visual data is labelled into one or more predefined classes to classify the visual data into one or more environment objects (discussed later in FIG. 3B and 3D). In an example, the one or more predefined classes may be stored in the memory 140. Further, the processing circuitry 130 is configured to produce an outline (discussed later in FIG. 3B and 3D) of the classified visual data into the one or more environment objects (discussed later in FIG. 3B and 3D). In addition, the processing circuitry 130 is configured to detect an intersection point (discussed later in FIG. 3B and 3D) between the generated graphic overlay and the outline (discussed later in FIG. 3B and 3D) of the classified visual data. The processing circuitry 130 is further configured to display the graphic overlay based on the intersection point (discussed later FIG. 3B and 3D), wherein the intersection point (discussed later in FIG. 3B and 3D) defines boundaries for the generated graphic overlay. Thereby, the system 100 provides accurate and reliable surround view information to the driver of the vehicle, ensuring safer vehicle manoeuvres. In addition, the system 100 efficiently deploy and adapts overlay rendering dynamically based on one or more environment objects (discussed later in FIG. 3B and 3D) using semantic segmentation techniques, thereby improving user experience.

[0067] In some embodiments, the processing circuitry 130 is configured to obtain an outline (discussed later in FIG. 5A and 5B) of a free space region (discussed later in FIG. 5A and 5B) from the produced outline (discussed later in FIG. 3B and 3D) of the classified visual data. In addition, the processing circuitry 130 is configured to mask the graphic overlay using one or more computer graphic techniques based on the outline of the free space region.

[0068] In some embodiments, the graphic overlays comprise one or more of: parking guidelines, trajectory lines, reverse guidelines, dynamic parking lines, and / or distance markers.202305531

[0069] 9

[0070] In some embodiments, the semantic segmentation model comprises a deep learning model trained on a dataset on images of vehicle’s surround view in one or more classes.

[0071] In some embodiments, the information on the vehicle’s surrounding comprises data on location and shape of obstacles, free space regions, drivable path.

[0072] FIG. 2 illustrates a flowchart of a method 200 for generating the graphic overlay in the surround view of the vehicle, according to some embodiments herein. The method 200 may facilitate generation of the graphic overlay in the surround view of the vehicle by way of the system 100 of FIG. 1.

[0073] At step 202, the method 200 provides a step of receiving visual data for vehicle’s surrounding.

[0074] At step 204, the method 200 provides a step of processing the visual data using a semantic segmentation model to obtain information on the vehicle’s surrounding.

[0075] At step 206, the method 200 provides a step of classifying the visual data into one or more environment objects (discussed later in FIG. 3B and 3D) in the vehicle’s surroundings.

[0076] At step 208, the method 200 provides a step of generating, in real time, the graphic overlay based on the classified visual data into the one or more environment objects (discussed later in FIG. 3B and 3D).

[0077] In some embodiments, the method 200 provides a step of segregating the visual data based on a pixel value associated with each pixel of the visual data, wherein the visual data is labelled into one or more predefined classes to classify the visual data into one or more environment objects (discussed later in FIG. 3B and 3D). The method 200 further provides a step of producing an outline (discussed later in FIG. 3B and 3D) of the classified visual data into the one or more environment objects (discussed later in FIG. 3B and 3D). In addition, the method 200 provides a step of detecting the intersection point (discussed later in FIG. 3B and 3D) between the generated graphic overlay and the outline (discussed later in FIG. 3B and 3D) of the classified visual data. Furthermore, the method 200 provides a step of displaying the graphic overlay based202305531

[0078] 10

[0079] on the intersection point (discussed later in FIG. 3B and 3D), wherein the intersection point (discussed later in FIG. 3B and 3D) defines boundaries for the generated graphic overlay. In an example, the intersection point may be a location where the generated graphic overlay and the outline meet or cross each other.

[0080] In some embodiments, the method 200 provides a step of obtaining the outline of the free space region from the produced outline of the classified visual data. Further, the method 200 provides a step of masking the graphic overlay using one or more computer graphic techniques based on the outline of the free space region.

[0081] In some embodiments of method 200, the graphic overlays comprise one or more of: parking guidelines, trajectory lines, reverse guidelines, dynamic parking lines, and / or distance markers.

[0082] In some embodiments of method 200, the semantic segmentation model comprises a deep learning model trained on a dataset on images of vehicle’s surround view in one or more classes. In an example, the deep learning model may be one or more of: Encoder-Decoder Architectures, Fully Convolutional Networks (FCNs), Dilated (Atrous) Convolution-Based Models, Attention-Based Models, Transformer-Based Models, Real-Time Models, and GAN-Based Models.

[0083] In some embodiments of method 200, the information on the vehicle’s surrounding comprises data on location and shape of obstacles, free space regions, and drivable path.

[0084] In some embodiments of method 200, the one or more computer graphic techniques comprises one or more of: a stencil-based technique, a scissor testing technique, a shader based masking technique, and a clipping planes technique.

[0085] The system 100 and the method 200 may be easily implemented in current generation Surround View Systems (SVS) without requiring computationally intensive 2D to 3D conversions. Furthermore, the system 100 and the method 200 support diverse overlay geometries and parameter configurations, accommodating a wide range of overlay types without the need for extensive software modifications. FIG. 3A, 3B, 3C, and 3D illustrates exemplary scenarios of generating the graphic overlay in the surround view of the vehicle, according to some embodiments herein.202305531

[0086] 11

[0087] FIG. 3A illustrates a first case 300A of displaying a conventional graphic overlay 302, wherein the conventional graphic overlay 302 is generated by conventional techniques. The first case 300A provides an information of a vehicle surrounding 304 comprising one or more environment objects 306. In an example, the one or more environment objects 306 further comprises other vehicles, animals, trees, lane marker, traffic signs, traffic lights, sidewalk, and / or pedestrians. Embodiments of the present disclosure are intended to include and / or otherwise cover any type of one or more environment objects 306 that may be present in vicinity of the vehicle, without deviating from the scope of the present disclosure. While using conventional techniques, the conventional graphic overlay 302 is generated on the one or more environment objects 306, thereby increasing the risk of vehicle manoeuvres and compromising the safety of vehicle.

[0088] FIG. 3B illustrates a second case 300B of displaying a graphic overlay 308 in accordance with the system 100 according to some embodiments herein. In an example, the second case 300B provides an information of the vehicle surrounding 304 comprising one or more environment objects 306 and the free space region 310. In an example, the one or more environment objects 306 comprises other vehicles, animals, sidewalk, and / or trees etc. The system 100 segregates the visual data of the vehicle surrounding 304 based on the pixel value associated with each pixel of the visual data, wherein the visual data is labelled into one or more predefined classes to classify the visual data into one or more environment objects 306. Further, the system 100 produces the outline 312 of the classified visual data into the one or more environment objects 306. In addition, the system 100 detects the intersection point 314 between the generated graphic overlay and the outline 312 of the classified visual data and displays the graphic overlay 308 based on the intersection point 314, wherein the intersection point 314 defines boundaries for the generated graphic overlay. Hence, providing accurate and reliable surround view information to the driver of the vehicle, ensuring safer vehicle manoeuvres.

[0089] Fig. 3C illustrates a third case 300C of displaying the conventional graphic overlay 302, wherein the conventional graphic overlay 302 is generated by conventional techniques. The third case 300C provides the information of the vehicle surrounding 304 comprising one or more environment objects 306. In an example, the one or more environment objects 306 further comprises pedestrians. While using conventional techniques, the202305531

[0090] 12

[0091] conventional graphic overlay 302 is generated on the one or more environment objects 306, thereby increasing the risk of vehicle manoeuvres and compromising the safety of vehicle.

[0092] FIG. 3D illustrates a fourth case 300D of displaying the graphic overlay 308 in accordance with the present system 100. In an example, the fourth case 300D provides the information of the vehicle surrounding 304 comprising one or more environment objects 306 and the free space region 310. In an example, the one or more environment objects 306 further comprises pedestrians. The system 100 segregates the visual data of the vehicle surrounding 304 based on the pixel value associated with each pixel of the visual data, wherein the visual data is labelled into one or more predefined classes to classify the visual data into one or more environment objects 306. Further, the system 100 produces the outline 312 of the classified visual data into the one or more environment objects 306. In addition, the system 100 detects the intersection point 314 between the generated graphic overlay and the outline 312 of the classified visual data and displays the graphic overlay 308 based on the intersection point 314, wherein the intersection point 314 defines boundaries for the generated graphic overlay. Hence, providing accurate and reliable surround view information to the driver of the vehicle, ensuring safer vehicle manoeuvres.

[0093] FIG. 4 illustrates additional details 400 of the method 200 for generating the graphic overlay 308 (shown in FIG. 3B and 3D) in the surround view of the vehicle, according to some embodiments herein. The generation of the graphic overlay 308 is performed in the following steps:

[0094] - At 402, the system 100 acquires data and measures data from the vehicle’s surrounding.

[0095] - At 404, the system 100 perceives the data provided to the system 100. The step of perception 404 involves performing semantic segmentation 406 and road surface detection 408.

[0096] - At 410, the system 100 visualizes results provided from the semantic segmentation 406 and the road surface detection 408. Visualization 410 comprises performing graphic overlay cut 412 and rendering graphics 414.

[0097] - At 416, the system 100 provides an improved visualization.

[0098] - At 418, the system 100 generates an accurate and reliable graphic overlay.202305531

[0099] 13

[0100] FIG. 5A and 5B illustrates exemplary scenarios of generating the graphic overlay in the surround view of the vehicle using one or more computer graphic techniques, according to some embodiments herein.

[0101] FIG. 5A illustrates a first scenario 500A of generating the graphic overlay 308 (shown in FIG. 3B and 3D) in the surround view of the vehicle (not shown) using one or more computer graphic techniques. The first scenario 500A depicts a zoomed in version of the vehicle’s surrounding 502. In the first scenario 500A, the system 100, by the processing circuitry 130 obtains the outline 506 of the free space region 508 from the produced outline (shown in FIG. 3B and 3D) of the classified visual data of one or more environment objects 504. Further, the processing circuitry 130 masks the graphic overlay 308 using one or more computer graphic techniques based on the outline 506 of the free space region 508.

[0102] FIG 5B illustrates a second scenario 500B of generating the graphic overlay (shown in FIG. 3B and 3D) in the surround view of the vehicle (not shown) using one or more computer graphic techniques. The second scenario 500B depicts a zoomed out or wider version of the vehicle’s surrounding 502.

[0103] Details of the second scenario 500B are similar to details of the first scenario 500A as discussed above and hence are not repeated for the sake of brevity.

[0104] The system 100 and the method 200 display distant parking slots accurately aligned with the ground thereby enhancing the usability and increases trust in the proposed system 100 and method 200.

[0105] Furthermore, the system 100 and the method 200 may be used in plurality of use cases in the automotive industries. For example, the system 100 and the method 200 may be applied to Parking Assistant System, Heads Up Displays (AR Displays), Collision Warnings, and for navigation purposes etc.

[0106] FIG. 6A and 6B illustrates series of images 602A to 602K showcasing a behaviour and positioning of the graphic overlay lines captured by the fisheye camera in the vehicle’s surrounding, according to some embodiments herein. The images 602A to 602K display the conventional graphic overlay 302, wherein the conventional graphic overlay 302 is generated by conventional techniques. Further, images 604A-604K display the graphic202305531

[0107] 14

[0108] overlay 308 in accordance with the system 100 and method 200. In addition, images 602A to 602K show that the conventional graphic overlay 302 is generated on the one or more environment objects 306, thereby increasing the risk of vehicle manoeuvres and compromising the safety of vehicle. On the other hand, images 604A to 604K show graphic overlay 308 is generated one or more computer graphic techniques based on the outline 506 (shown in FIG. 5A and 5B) of the free space region 508 (shown in FIG.

[0109] 5A and 5B). Hence, providing accurate and reliable surround view information to the driver of the vehicle, ensuring safer vehicle manoeuvres.

[0110] FIG. 7 illustrates an example computing environment 700 implementing the system 100, as shown in FIG. 1 for generating the graphic overlay 308 in the surround view of the vehicle (not shown), according to some embodiments herein. As depicted in FIG. 7, the computing environment 700 comprises at least one data processing unit 706 that is equipped with a control unit 702 and an Arithmetic Logic Unit (ALU) 704, a plurality of networking devices 708 and a plurality Input output, I / O devices 710, a memory 712, a storage 714. The data processing unit 706 may be responsible for implementing the system 100 and method 200 described in FIGs 1 to 6. For example, the data processing unit 706 in some embodiments be equivalent to the processing circuitry 130 of the system 100 as described above in reference with FIG. 1. The data processing unit 706 is capable of executing software instructions stored in memory 712. The data processing unit 706 receives commands from the control unit 702 in order to perform its processing. Further, any logical and arithmetic operations involved in the execution of the instructions are computed with the help of the ALU 704.

[0111] The computer program is loadable into the data processing unit 706, which may, for example, be comprised in an electronic apparatus (such as the platform). When loaded into the data processing unit 706, the computer program may be stored in the memory 712 associated with or comprised in the data processing unit 706. According to some embodiments, the computer program may, when loaded into and run by the data processing unit 706, cause execution of method steps according to, for example, any of the methods illustrated in FIG. 2 as described herein.

[0112] The overall computing environment 700 may be composed of multiple homogeneous and / or heterogeneous cores, multiple CPUs of different kinds, special media and other202305531

[0113] 15

[0114] accelerators. Further, the plurality of data processing unit 706 may be located on a single chip or over multiple chips.

[0115] The algorithm comprising of instructions and codes required for the implementation are stored in either the memory 712 or the storage 714 or both. At the time of execution, the instructions may be fetched from the corresponding memory 712 and / or storage 714 and executed by the data processing unit 706.

[0116] In case of any hardware implementations various networking devices 708 or external I / O devices 710 may be connected to the computing environment to support the implementation through the networking devices 708 and the I / O devices 710.

[0117] The embodiments disclosed herein can be implemented through at least one software program running on at least one hardware device and performing network management functions to control the elements. The elements shown in FIG. 7 include blocks which can be at least one of a hardware device, or a combination of hardware device and software module.202305531

[0118] 16 Reference signs

[0119] • System 100

[0120] • Sensors 110

[0121] • GPS 120

[0122] • Processing circuitry 130

[0123] • Memory 140

[0124] • Display 150

[0125] • Method 200

[0126] • First case 300A

[0127] • Conventional graphic overlay 302

[0128] • Vehicle’s surrounding 304, 502

[0129] • One or more environment objects 306, 504 • Second case 300B

[0130] • Graphic overlay 308

[0131] • Free space region 310

[0132] • Outline of the classified visual data 312

[0133] • Intersection point 314

[0134] • Third case 300C

[0135] • Fourth case 300D

[0136] • Additional details 400 of method 200

[0137] • First scenario 500A

[0138] • Second scenario 500B

[0139] • Outline of a free space region 506202305531

[0140] 17

[0141] • Free space region 508

[0142] • Images display the conventional graphic overlay 602A 602K • images display the graphic overlay 604A 604K

[0143] • Computing environment 700

[0144] • Control unit 702

[0145] • Arithmetic Logic Unit (ALU) 704

[0146] • Processing unit 706

[0147] • Network devices 708

[0148] • I / O devices 710

[0149] • Memory 712

[0150] • Storage 714

Claims

20230553118Claims:

1. A method (200) for generating a graphic overlay (308) in a surround view of a vehicle, c h a r a c t e r i z e d i n t h a t, the method comprising:receiving (202) visual data for vehicle’s surrounding (304, 502);processing (204) the visual data using a semantic segmentation model to obtain information on the vehicle’s surrounding (304, 502), wherein the processing comprises:classifying (206) the visual data into one or more environment objects (306, 504) in the vehicle’s surroundings (304, 502); andgenerating (208) in real time, the graphic overlay (308) based on the classified visual data into the one or more environment objects (306, 504).

2. The method (200) according to claim 1 , wherein generating the graphic overlay (308) comprises:segregating the visual data based on a pixel value associated with each pixel of the visual data, wherein the visual data is labelled into one or more predefined classes to classify the visual data into one or more environment objects (306, 504);producing an outline (312) of the classified visual data into the one or more environment objects (306, 504);detecting an intersection point (314) between the generated graphic overlay and the outline (312) of the classified visual data; anddisplaying the graphic overlay (308) based on the intersection point (314), wherein the intersection point (314) defines boundaries for the generated graphic overlay.

3. The method (200) according to any of the preceding claims, wherein generating the graphic overlay (308) comprises:obtaining an outline (506) of a free space region (508) from the produced outline20230553119masking the graphic overlay (308) using one or more computer graphic techniques based on the outline of the free space region (508).

4. The method (200) according to any of the preceding claims, wherein the graphic overlays (308) comprise one or more of: parking guidelines, trajectory lines, reverse guidelines, dynamic parking lines, and / or distance markers.

5. The method (200) according to any of the preceding claims, wherein the semantic segmentation model comprises a deep learning model trained on a dataset on images of vehicle’s surround view in one or more classes.

6. The method (200) according to any of the preceding claims, wherein the information on the vehicle’s surrounding (304, 502) comprises data on location and shape of obstacles, free space regions, and drivable path.

7. The method (200) according to any of the preceding claims, wherein the one or more computer graphic techniques comprises one or more of: a stencil based technique, a scissor testing technique, a shader based masking technique, and a clipping planes technique.

8. A system (100) for generating a graphic overlay (308) in a surround view of a vehicle, c h a r a c t e r i z e d i n t h a t, the system (100) comprising:processing circuitry (130) configured to:receive visual data for vehicle’s surrounding (304, 502);process the visual data using a semantic segmentation model to obtain information on the vehicle’s surrounding (304, 502), wherein the processing comprises:classify the visual data into one or more environment objects (306, 504) in the vehicle’s surroundings (304, 502); andgenerate in real time, the graphic overlay (308) based on the classified visual data into the one or more environment objects (306, 504).202305531209. The system (100) according to claim 8, wherein the processing circuitry (130) is configured to:segregate the visual data based on a pixel value associated with each pixel of the visual data, wherein the visual data is labelled into one or more predefined classes to classify the visual data into one or more environment objects (306, 504);produce an outline (312) of the classified visual data into the one or more environment objects (306, 504);detect an intersection point (314) between the generated graphic overlay and the outline (312) of the classified visual data; anddisplay the graphic overlay (308) based on the intersection point (314), wherein the intersection point (314) defines boundaries for the generated graphic overlay.

10. The system (100) according to any of the claims 8 or 9, wherein the processing circuitry (130) is configured to:obtain an outline (506) of a free space region (508) from the produced outline of the classified visual data; andmask the graphic overlay (308) using one or more computer graphic techniques based on the outline (506) of the free space region (508).

11. The system (100) according to any of the claims 8-10, wherein the graphic overlays (308) comprise one or more of: parking guidelines, trajectory lines, reverse guidelines, dynamic parking lines, and / or distance markers.

12. The system (100) according to any of the claims 8-11 , wherein the semantic segmentation model comprises a deep learning model trained on a dataset on images of vehicle’s surround view in one or more classes.

13. The system (100) according to any of the claims 8-12, wherein the information on the vehicle’s surrounding (304, 502) comprises data on location and shape of obstacles, free space regions, drivable path.2023055312114. A computer program comprising instructions, which, when the program is executed by a computer, cause the computer to carry out a method (200) of any one of the claims 1 to 7.

15. A computer readable medium having stored thereon the computer program of claim

Images