METHOD FOR DISPLAYING IMAGES FROM A CAMERA SYSTEM OF A VEHICLE

DE502018015305D1Active Publication Date: 2024-11-28ROBERT BOSCH GMBH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
DE502018015305
Authority / Receiving Office
DE · DE
Patent Type
Patents
Current Assignee / Owner
Priority Date
2017-10-11
Filing Date
2018-09-10
Publication Date
2024-11-28
Estimated Expiration
2038-09-10

AI Technical Summary

Technical Problem

All-round vision systems for vehicles overwhelm drivers with excessive information, requiring significant computing power to display complex data, while often presenting irrelevant information that does not add value to the driver's understanding of the surroundings.

Method used

A method to display detected obstacles in a virtual three-dimensional space using virtual objects with a selection criterion to reduce information content by applying predefined textures or filters, such as gray shading, to enhance contrast and reduce computational effort, allowing the driver to focus on relevant information.

Benefits of technology

This approach reduces the driver's cognitive load by presenting only essential information, improving obstacle recognition and reducing computing power consumption, enabling safer navigation and vehicle control.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader
Need to check novelty before this filing date? Find Prior Art

Description

Technical area:

[0001] The present invention relates to a method for displaying images from a camera system of a vehicle. In particular, the camera system is an all-around vision system that can capture images around a vehicle. State of the art:

[0002] Such all-round vision systems are known in the art as surround-view systems (SVS). For example, the following publications describe such systems: "360 Wrap-Around Video Imaging Technology Ready for Integration with Fujitsu Graphics SoCs," January 2014; Esparza Garcia: "3D Reconstruction for Optimal Representation of Surroundings in Automotive HMIs, Based on Fisheye Multicamera Systems," dissertation 2015. Such systems use various data types to display the maximum amount of information. This way, the driver is presented with the maximum amount of information the system can provide. Such data types can be obtained, in particular, from sensor data fusion, from information from a cloud, and from many other sources.Typical data from sensor data fusion includes information about occupied and unoccupied areas around the vehicle, which is determined using video sensors, radar sensors, lidar sensors, or other vehicle environmental sensors. Information from a cloud can include, in particular, weather conditions and available parking spaces.

[0003] Due to the sheer volume of information that the SVS is supposed to display, the vehicle driver can become overwhelmed if it isn't displayed appropriately. This leads to data being collected unnecessarily that provides no added value to the driver, as the driver fails to recognize the information. At the same time, the current technology requires considerable computing power to create a very complex display that integrates all the information available to the surround-view system into a single image. However, this considerable computing power is often unnecessary to present the driver with the information that is currently relevant. Disclosure of the invention

[0004] The invention is defined by the independent claims. The method according to the invention enables information to be displayed in different representations. The representations are determined based on a selection criterion. This allows, on the one hand, to display only relevant information to the driver, and, on the other hand, the computing power of the system generating the counter-representation is significantly reduced.

[0005] The inventive method for displaying images from a camera system of a vehicle is intended for displaying obstacles from the surroundings of the vehicle detected by said camera system on a display device. The display takes place in a virtual three-dimensional space. In this virtual three-dimensional space, the detected obstacles are represented as virtual three-dimensional objects. The viewer thus receives the impression that this is a spatial representation in which spatial objects are depicted. This corresponds to a virtual view of the actual scenery from a corresponding virtual camera position. The driver of the vehicle can thus be provided with an overview of the current scenery.In order to optimally display the relevant information on the one hand, and to reduce the computing power of the system generating the display on the display device on the other, a selection criterion is used to determine whether the virtual three-dimensional objects and / or the virtual three-dimensional space are to be covered with textures generated by the camera system or with at least one predefined texture. The predefined texture can also be a single color in different shades, so that no texture is displayed on the display device at all. By omitting the actual textures, i.e. the textures generated by the camera system, the information content of the display on the display device can be reduced on the one hand, and the computing effort required to generate the display is reduced on the other.This allows the driver of the vehicle to focus on relevant information, while at the same time significantly reducing the required computing power.

[0006] The subclaims contain preferred developments of the invention.

[0007] Advantageously, the selection criterion determines whether the textures intended to occupy the virtual three-dimensional objects and / or the virtual three-dimensional space are processed using a predefined filter. In particular, the predefined filter can increase the contrast of the textures. This still allows for the display of textures while simultaneously improving the recognition of the virtual three-dimensional objects on the display device.

[0008] In particular, the predefined filter is intended to include a gray shading. The gray shading reduces the color depth of the textures, thereby enhancing contrast. At the same time, the gray shading reduces the information content of the textures, which in turn makes it easier for the driver to recognize the virtual three-dimensional objects on the display device.

[0009] The predefined texture is advantageously a predefined brightness gradient. The predefined brightness gradient only displays a contour and / or outer surfaces of the virtual three-dimensional objects. This means that only the outlines and / or the shape of the virtual three-dimensional objects are recognizable on the display device. Thus, no texture is displayed, which significantly reduces the computational effort required to generate the representation on the display device. At the same time, the driver of the vehicle can steer the vehicle safely and reliably through the obstacles in the environment, which is particularly advantageous when parking the vehicle. In particular, the driver is not distracted from the actual obstacles by the detailed textures.

[0010] In a further preferred embodiment, at least some of the virtual three-dimensional objects, in particular all virtual three-dimensional objects, are highlighted by an at least partial overlay on the display device. The overlay advantageously comprises a frame around the corresponding virtual three-dimensional object. In this way, the obstacles in the environment can be reliably visualized. The driver of the vehicle can thus clearly see where obstacles are located in the environment and whether these obstacles pose a danger to their vehicle. Further overlays that give the driver additional information about features in the environment are also advantageous. For example, it is possible to highlight a freely available parking space by means of corresponding markings on the display device.

[0011] The selection criterion is advantageously user input. This allows the user to freely choose whether they require a simplified display with at least one predefined texture or a more complex display using the camera system's textures. The user can also specify whether at least some of the camera system's textures should be processed with a predefined filter. This allows the driver to freely choose which view offers the greatest possible information content in a current situation without being confused by a multitude of currently unnecessary information.

[0012] The selection criterion includes activating a parking assistance system. In a further embodiment, it is preferably provided that the selection criterion includes the successful detection of a parking space. Thus, a change in the display occurs automatically. In this way, the driver of the vehicle can be offered the currently objectively best display. Thus, while the parking assistance system is active, a display with only predefined textures can be used to easily illustrate to the driver which obstacles the parking assistance system has detected and how the parking strategy is implemented based on these obstacles.

[0013] The invention also relates to a computer program configured to execute the method as described above. The computer program is, in particular, a computer program comprising machine-readable instructions that, when executed on a computing device, cause the computing device to execute the steps of the method described above. The computing device is, in particular, a control unit of a vehicle.

[0014] The invention further comprises a machine-readable storage medium. The above-described computer program is stored on the machine-readable storage medium. The machine-readable storage medium is, in particular, an optical data carrier and / or a magnetic data carrier and / or a flash memory.

[0015] Finally, the invention relates to a control unit for executing the method as described above. Likewise, the control unit is advantageously designed to execute the computer program as described above. Short description of the drawing

[0016] Embodiments of the invention are described in detail below with reference to the accompanying drawings. In the drawing: Figure 1 shows a schematic representation of a vehicle with a control unit for carrying out a method according to an embodiment of the invention, Figure 2 shows a first example of a representation which is generated by means of the method according to the embodiment of the invention, Figure 3 shows a second example of a representation which is generated by means of the method according to the embodiment of the invention, Figure 4 shows a third example of a representation which is generated by means of the method according to the embodiment of the invention, and Figure 5 shows a fourth example of a representation which is generated by means of the method according to the embodiment of the invention.

[0017] Implementation of the claimed invention: Figure 1 shows schematically a vehicle 1 comprising a control unit 2 according to an embodiment of the invention.

[0018] The vehicle 1 comprises a camera system 3, which is designed to capture the surroundings of the vehicle 1. Thus, the entire surroundings of the vehicle 1 can be scanned by the camera system 3. The control unit 2 is used to create a representation on a display device 4 in order to present the surroundings to the driver. It is provided that the control unit 2 displays the obstacles from the surroundings of the vehicle, captured by the camera system 3, on the display device 4 in a virtual three-dimensional space 6 as virtual three-dimensional objects 5. This is described in the Figure 2 - 5 The choice of the representation to be shown on the display device 4 is made based on a selection criterion.

[0019] The selection criterion is advantageously a user input. This allows the user to choose which different displays they wish to view. Alternatively, the selection criterion can be determined automatically by selecting the display based on the successful detection of a parking space and / or based on the activation of a parking assistance system. The goal is to always display the vehicle driver an optimal image of the surroundings that includes relevant information without overwhelming the driver with a large amount of information. At the same time, the selective selection of the display significantly reduces the computational effort within control unit 2, since displays with reduced information content are also shown on the display device, and displays with maximum information content do not have to be continuously generated.

[0020] Figure 2shows a schematic representation of a first example of a display on the display device 4. Detected obstacles from the surroundings of the vehicle are displayed as virtual three-dimensional objects 5 in a virtual three-dimensional space 6. For better distinguishability, it is also provided that frames 7 are displayed around the obstacles 5. Thus, the driver of the vehicle 1 can easily recognize the obstacles in the surroundings based on the virtual three-dimensional objects 5. For better clarity, it is also preferably provided that a representation 10 of the vehicle 1 is displayed in the virtual three-dimensional space 6.

[0021] In the Figure 2In the example shown, all virtual three-dimensional objects 5 and the virtual three-dimensional space are covered with textures that were captured by the camera system 3. This means that a photorealistic representation is created. The driver of the vehicle 1 is thus provided with a multitude of different information in the representation. If individual obstacles are not recognized and thus not displayed as virtual three-dimensional objects 5, the driver still has the option of recognizing these obstacles based on the textures. However, there is a risk that the driver of the vehicle will be overwhelmed by the multitude of information.Thus, the driver of the vehicle 1 may overlook important information, which means that there is a risk that the vehicle 1 will collide with an obstacle because the driver has overlooked the approach of the representation 10 of the vehicle 1 to a virtual three-dimensional object 5.

[0022] In Figure 3 A second example is shown. In this example, the display is without textures. The scenery is the same as in Figure 2 . In contrast to Figure 2 The virtual three-dimensional objects 5 and the virtual three-dimensional space 6 are not covered with textures. This means that the virtual three-dimensional objects 5 and the virtual three-dimensional space 6 only have a brightness gradient that includes a representation of the outlines of the virtual three-dimensional objects 5. Preferably, the virtual three-dimensional objects 5 are highlighted by highlights 7.

[0023] In the Figure 3 In the representation shown, the information content is limited solely to the presence and absence of obstacles. The driver of vehicle 1 is thus not overwhelmed by the multitude of different textures; rather, the driver of vehicle 1 can concentrate on avoiding collisions with obstacles in the environment, which means always maintaining a distance between the representation 10 of vehicle 1 and the virtual three-dimensional objects 5.

[0024] Figure 4 Finally, shows a third example of the representation. It uses textures as in Figure 2 In contrast to Figure 2In this example, a filter is applied over the textures. Such a filter is, in particular, a shade of gray. This increases the contrast of the display, while simultaneously reducing the color depth. This way, a display is realized that, in terms of its information content, lies between the Figure 2 and the one in Figure 3 shown representation. Textures are also displayed to the driver of the vehicle so that potentially undetected obstacles can be recognized by the driver. At the same time, the detected obstacles can be reliably recognized by the driver through the virtual three-dimensional objects 5.

[0025] Finally, Figure 5 a fourth example. In this example, the display is divided into a first area 8 and a second area 9. The first area 8 corresponds to the first example, as in Figure 2shown, while the second area 9 corresponds to the third example, as shown in Figure 3 shown. This results in a transition from textures generated by camera system 3 to textures processed with a predefined filter. This allows the advantages of a high level of detail for close-up range to be combined with the advantages of increased contrast for long-distance range.

Claims

1. Method for displaying images from a camera system (3) of a vehicle (1), • wherein obstacles from the surrounding area of the vehicle (1), which are detected using the camera system (3), are displayed as virtual three-dimensional objects (5) on a display apparatus (4) in a virtual three-dimensional space (6), and • wherein it is determined on the basis of a selection criterion whether the virtual three-dimensional objects (5) and / or the virtual three-dimensional space (6) ∘ have textures which are generated by the camera system (3) placed on them, or ∘ have at least one predefined texture placed on them, wherein the selection criterion comprises activation of a parking assistance system, wherein a change of the display takes place automatically, wherein a display with only predefined textures is used during an active parking assistance system.

2. Method according to Claim 1, characterized in that it is determined on the basis of the selection criterion whether the textures to be placed on the virtual three-dimensional objects (5) and / or the virtual three-dimensional space (6) are processed by means of a predefined filter.

3. Method according to Claim 2, characterized in that the predefined filter comprises a greyscale.

4. Method according to any of the preceding claims, characterized in that the predefined texture is a predefined brightness gradient to display merely a contour and / or outer surfaces of the virtual three-dimensional objects (5).

5. Method according to any of the preceding claims, characterized in that at least some of the virtual three-dimensional objects (5), in particular all the virtual three-dimensional objects (5), are highlighted on the display apparatus (4) by an at least partial overlay (7), in particular by a border.

6. Method according to any of the preceding claims, characterized in that the selection criterion is a user input.

7. Computer program comprising instructions which, when the program is executed by a computer, cause the latter to carry out the steps of the method according to any of the preceding claims.

8. Machine-readable storage medium on which the computer program according to Claim 7 is stored.

9. Controller (2) configured to carry out the method according to any of Claims 1 to 6.