METHOD FOR REPRESENTING A VEHICLE'S ENVIRONMENT

Description

State of the art

[0001] The present invention relates to a method for displaying the environment of a vehicle, in particular for displaying obstacles in the vehicle's environment. The invention further relates to a computer program for carrying out such a method. Finally, the invention relates to a control unit for a vehicle configured to carry out such a method, in particular to execute a computer program.

[0002] Document DE 10 2008 046 214 A1 discloses a method for monitoring the environment of a vehicle, in which the environment is captured by means of at least one image capture unit, wherein an overall image is generated from individual images captured by means of the at least one image capture unit using an image processing unit and output by means of a display unit.

[0003] Document DE 10 2011 082 475 A1 discloses a driver assistance system for supporting a driver in collision-relevant situations of a vehicle, wherein the driver assistance system comprises a sensor for measuring distance, an evaluation unit and means for keeping data on the surface geometry of the vehicle available.

[0004] The document EP 2 555 519 A1 discloses a vehicle peripheral observation device comprising a plurality of cameras for imaging an area that includes a road surface of the vehicle periphery, an image processor for displaying the images taken using the plurality of cameras on a monitor, and an obstacle detection device for detecting an obstacle in the vehicle periphery.

[0005] Systems capable of displaying a visual representation of the vehicle's surroundings to the driver are known from the prior art. These systems are also called surround-view systems (SVS). Such surround-view systems combine a multitude of different sensor signals, all of which are displayed to the driver, thus providing them with a maximum amount of information. This information is obtained, for example, from signals from video sensors, radar sensors, lidar sensors, ultrasonic sensors, or similar environmental sensors. The information can also be combined with previously stored data, particularly data stored in a cloud. As a result, a surround-view system displays a vast amount of information.There is a risk that the driver of a vehicle equipped with a surround-view system may not recognize hazards, such as the vehicle approaching an obstacle, in time due to the large amount of information. Disclosure of the invention

[0006] The method according to the invention allows for the rapid and reliable detection of hazardous situations caused by the vehicle approaching an obstacle. This is achieved by displaying, in addition to a first display that provides the driver with an overview of the surroundings, a second display that specifically alerts the driver to the danger of the approaching obstacle.

[0007] The inventive method for displaying a vehicle's environment comprises the following steps, which are carried out in the sequence listed below: First, the vehicle's environment is detected, and obstacles in the vehicle's environment are identified using the vehicle's environmental sensors. Environmental sensors are, in particular, optical or acoustic sensors for detecting obstacles in the vehicle's environment. These can be, in particular, camera systems, ultrasonic systems, and / or radar systems, as well as similar systems. It is particularly advantageous to detect the environment using at least one camera in order to display visual representations of the environment to the vehicle's driver. By performing the detection step, it is thus known, on the one hand, in which environment the vehicle is located, and on the other hand, obstacles in the vehicle's environment are identified.The next step involves displaying an initial representation. This initial representation comprises a representation of the environment with its obstacles and a representation of the vehicle on a display device. The vehicle representation is, in particular, an animation of the vehicle itself, superimposed with the representation of the environment and the obstacles. Advantageously, the representation of the environment and the obstacles is a visual representation generated from the data obtained in the previously described acquisition and determination step. The display device serves as an interface to the vehicle's driver. Advantageously, the display device is a display unit or monitor and is located, for example, inside the vehicle, such as in the vehicle's center console.Furthermore, the system detects when the vehicle is approaching a identified obstacle, specifically when a predefined distance to the obstacle has been breached. This indicates that the distance between the vehicle and the obstacle is decreasing and the vehicle is already close to the obstacle. Consequently, there is a risk of the vehicle colliding with the obstacle. This can be particularly relevant in areas with limited visibility, such as along the vehicle's side fenders. Such approach to obstacles must be displayed to the driver reliably and unambiguously. Therefore, a second display is shown, showing a larger section of the surroundings and obstacles compared to the first display. This second display also includes a representation of the vehicle, which is identical to that shown in the first display.Should the second representation be shown from a different perspective than the first, the vehicle representation is advantageously adjusted accordingly. The second representation is again displayed on the same display device that showed the first. The section of the representation of the environment with the obstacles shown in the second representation corresponds specifically to the area of ​​the environment and the vehicle where the vehicle's approach to the obstacle was detected. This highlights the point where the vehicle is approaching the obstacle. The driver's attention is thus drawn to this approach, enabling the driver to take appropriate measures to avoid a collision between the vehicle and the obstacle.The display device thus presents a large amount of information, with the previously described method explicitly highlighting particularly relevant information. This eliminates the risk that the information indicating the vehicle is approaching another obstacle will be overlooked amidst the multitude of displayed details, preventing the driver from noticing it. This enhances vehicle safety by reducing the risk of damage to obstacles in the vehicle's vicinity and to the vehicle itself.

[0008] According to the invention, the first and second illustrations show images from different virtual perspectives of a three-dimensional model of the environment. The three-dimensional model of the environment was generated based on the captured environment and the identified obstacles. The capture of the environment and the identification of obstacles have been described previously. The different virtual perspectives allow the driver's focus to be directed directly to the area where the vehicle is approaching an obstacle. This ensures that the driver clearly recognizes that the vehicle is approaching an obstacle.

[0009] According to the invention, the second representation is generated by changing the virtual perspective and zooming in from the first representation. This is achieved, according to the invention, by a virtual camera pan, whereby the driver of the vehicle is shown an animation on the display device that corresponds to the change in perspective. The image displayed on the display device is thus modified in such a way that the driver gets the impression that the perspective is moving towards the hazard, i.e., that the perspective is moving towards the point on the vehicle where the obstacle is approaching. The driver's attention is thus immediately drawn to said hazard. This is particularly supported by the fact that the second representation emerges directly from the first representation.

[0010] The dependent claims contain preferred further developments of the invention.

[0011] In another alternative, the first and second representations are separate. This has the advantage that the first representation remains unchanged, while the second representation serves only to highlight the vehicle's approach to an obstacle. This separate representation also makes it possible to visualize multiple points where the vehicle approaches an obstacle. This is particularly advantageous during parking maneuvers when the vehicle comes close to obstacles, such as other parked vehicles and / or boundary walls, at numerous points.

[0012] The second display is advantageously presented on the display device in such a way that it overlays at least a portion of the first display. This gives the driver the impression that the display device is sending a message. This message appears in a separate window that overlays the original display, i.e., the first display. Thus, the driver is given the impression that the second display has a higher priority than the first. The driver can immediately and unambiguously recognize that the vehicle is approaching an obstacle.

[0013] Alternatively, the first and second displays are shown side-by-side on the display device. For this, the first display is scaled to make room for the second. If the first display is not displayed in such a way that it occupies the entire screen, then such scaling is unnecessary. This alternative ensures that there is no overlap. Therefore, no information is lost when the second display is shown, as would be the case if the second display were superimposed. The driver can still see the first display in its entirety, just as before the second display appeared.

[0014] In the first and / or second representation, the representations of obstacles approaching the vehicle are advantageously highlighted. This can be achieved, in particular, by placing a frame around said obstacles in the representation, which can advantageously be flashing. Similarly, the obstacles in the representation can be overlaid with a signal color, which can also be flashing. Other measures are also possible. In this way, the driver is explicitly shown where a collision between the vehicle and the obstacle could occur. The driver is thus explicitly given the opportunity to initiate appropriate driving maneuvers to avoid a potential collision by recognizing it early.

[0015] The invention also relates to a computer program product. This computer program product comprises instructions which, when executed on an evaluation unit, cause the evaluation unit to perform the steps of the method described above. An evaluation unit can, in particular, be a vehicle control unit. This control unit simply needs to be connected to environmental sensors and a display device.

[0016] Finally, the invention relates to a control unit for a vehicle. The control unit includes at least one interface for connecting environmental sensors. A display device can also be connected to the interface. Thus, the control unit is particularly capable of controlling the environmental sensors to detect the vehicle's surroundings and to identify obstacles in the vehicle's vicinity. The control unit is therefore advantageously configured to execute a computer program as described above or to perform a method as described above. The control unit can thus assist the driver of the vehicle in navigating the vehicle through environments with obstacles, thereby avoiding the risk of collisions between the vehicle and obstacles. Brief description of the drawings

[0017] Exemplary embodiments of the invention are described in detail below with reference to the accompanying drawings. The drawings show: Figure 1 is a schematic view of a flowchart of a method according to an embodiment of the invention. Figure 2 is a schematic view of a vehicle with a control unit according to an embodiment of the invention. Figure 3 is a first schematic representation of a first view when carrying out the method according to an embodiment of the invention. Figure 4 is a second schematic representation of the first view when carrying out the method according to the embodiment of the invention. New description page 8 of the main application (clean copy). Figure 5 is a schematic representation of partial steps of a change in a virtual perspective when switching from the first view to a second view when carrying out the method according to an embodiment of the invention.Figure 6 is a first schematic representation of a combination of the first representation and the second representation when carrying out the method according to the embodiment of the invention; Figure 7 is a second schematic representation of the combination of the first representation and the second representation when carrying out the method according to the embodiment of the invention; and Figure 8 is a third schematic representation of the combination of the first representation and the second representation when carrying out the method according to an embodiment that is not part of the claimed invention. Embodiments of the invention

[0018] Figure 1Figure 1 schematically shows a flowchart of a method according to an embodiment of the invention. The flowchart comprises the steps of sensing an environment and detecting obstacles 100, displaying a first representation 200, detecting that the vehicle is approaching an obstacle 300, and displaying a second representation 400. These steps are explained below with reference to the [reference to be added]. Figure 2 The vehicle shown, 1, is equipped with a control unit 11. The control unit 11 is used to execute the function described in Figure 1The method shown is characterized by the control unit 11 being connected to a display device 10. Thus, the control unit 11 can cause the display device 10 to display information. Furthermore, the control unit 11 is connected to environmental sensors 8 and 9, which include ultrasonic sensors 8 and a camera system 9. The camera system 9 enables, in particular, three-dimensional detection of the vehicle 1's surroundings. The ultrasonic sensors 8 allow for the very precise detection of short to medium distances.

[0019] To carry out the method according to the exemplary embodiment, the environment of the vehicle 1 is first detected, along with the detection of obstacles in the vehicle's environment. This is done using the environmental sensors 8, 9, so that, in particular, a pictorial representation, i.e., a representation including textures, of the vehicle 1's environment is available. This means that a highly accurate image of the environment can be displayed to a user, in particular the driver of the vehicle.

[0020] Subsequently, a first representation 5 is displayed, with an exemplary first representation 5 in Figure 3 as in Figure 4As shown, the camera system 9 enables the generation of a three-dimensional model of the environment, additionally utilizing the distance information from the ultrasonic sensors 8. This means that the textures captured by the camera system 9 can be processed into a three-dimensional model based on the distance information, allowing obstacles to be displayed to the driver of vehicle 1 by providing the driver with a comprehensive virtual view of the vehicle 1 and its surroundings from a virtual perspective within the three-dimensional model.

[0021] In the Figures 3 and 4An example parking situation is shown. The driver of vehicle 1 wants to park in a parking space. The parking space is bordered by a first obstacle 2 and a second obstacle 3, where the first obstacle 2 and the second obstacle 3 are adjacent parked vehicles. In the first illustration, the driver recognizes these vehicles and can therefore avoid the first obstacle 2 and the second obstacle 3. As in Figure 4 Although the area is marked, potential hazards exist. These hazards are primarily located at the outer corners of the first obstacle 2 and the second obstacle 3. As the parking maneuver progresses, these outer corners will approach the sides of vehicle 1, creating a risk of collision between vehicle 1 and the first obstacle 2 and the second obstacle 3.

[0022] Therefore, step 300 of detecting the approach of vehicle 1 to one of the identified obstacles 2, 3 in the environment occurs, whereby a predefined distance to the obstacle 2, 3 is breached. This can be detected in particular by the ultrasonic sensors 8, as the distance between vehicle 1 and obstacles 2, 3 decreases. As soon as such an approach of vehicle 1 is detected and the distance to the obstacle 2, 3 falls below the predefined distance, a potential collision is assumed, which is why the aforementioned step 400 of displaying this information in a second representation 6, 7 takes place.

[0023] The display of message 400 in the second representation 6, 7 explicitly warns the driver of a danger of collision between the vehicle and obstacles 2, 3. This display of message 400 in the second representation 6, 7 can occur in different ways.

[0024] Figure 5schematically shows a first possibility for displaying 400 of the second representation 6, 7. Starting from the one in Figure 3 or in Figure 4 The first representation 5 shown represents a change in virtual perspective as well as a change in zoom from the first representation 5 to the second representation 6. This means that the second representation 6 emerges from the first representation 5. In Figure 5 Individual intermediate frames of the transition from the first display 5 to the second display 6 are shown, with said transition preferably being a continuous animation. In particular, the first display 5 is shown on the display device 10 by completely filling the display device 10. Due to the change in perspective and zoom, the second display 6 also completely fills the display device 10. The first display 5 is no longer shown.

[0025] It is evident that the second illustration 6, 7 thus shows a section of the first illustration 5, with the perspective and zoom also changed. This allows the driver of vehicle 1 to see the area of ​​the surroundings where the approach of vehicle 1 to the obstacle 2, 3, 4 is most noticeable. Figure 5 The first obstacle 2 is approaching. Thus, the driver is explicitly alerted that vehicle 1 is approaching an obstacle 2, 3, 4.

[0026] The in Figure 5However, the demonstrated option has the disadvantage that the first display 5 is no longer visible. This means that the driver of the vehicle no longer has an overview of the parking situation, since only the second display 6 is shown on the display device 10. This is also disadvantageous when the vehicle 1 approaches several obstacles 2, 3, 4. In the parking situation according to the exemplary embodiment, the vehicle 1 is not only approaching the first obstacle 2, but is also approaching the second obstacle 3.

[0027] In order to continue to comprehensively warn the driver of vehicle 1 of approaching obstacles 2, 3, 4, two different second representations 6, 7 are displayed. One of the second representations 6 shows the approach of vehicle 1 to the first obstacle 2, while another second representation 7 shows the approach of vehicle 1 to the second obstacle 3. In the Figures 6 and 7 Two different ways are shown in which the second representations 6, 7 together with the first representation 5 can be displayed on the display device 10. Figure 6 The second representations 6, 7 of the first representation 5 are superimposed. Figure 7 The first representation 5 and the second representations 6 and 7 are displayed side by side. In Figure 6It is thus evident that there is a risk of information loss in the first display 5, since the overlay with the second displays 6, 7 obscures all areas of the first display 5. In contrast, the second displays 6, 7, due to their foreground placement in front of the first display, appear as higher-priority messages, thus directing the driver's attention specifically to the approach to obstacles 2, 3. For displaying the first display 5 and the second displays 6, 7 side by side, as in Figure 7 As outlined, scaling is necessary because the first representation 5, which previously occupied the entire display device 10, now needs to be reduced in size. However, this prevents any loss of information when displaying the first representation 5.

[0028] Figure 8Figure 5 shows another possibility for the combined display of first representation 5 and second representations 6 and 7, which is not part of the claimed invention. Figure 8 Vehicle 1 has almost reached its final parking position. Thus, in addition to the first obstacle 2 and the second obstacle 3, a third obstacle 4 is present, with the third obstacle 4 being a boundary wall. Therefore, there are multiple areas where vehicle 1 approaches obstacles 2, 3, and 4. A second representation 6 is therefore displayed using only schematic elements. This means that, in addition to a representation of vehicle 1, the other obstacles 2, 3, and 4 are shown only schematically. In the Figure 8In the example shown, edges are displayed that represent the edge of a detected obstacle. This gives the driver of vehicle 1 a precise overview of the distance between their vehicle 1 and the obstacles 2, 3, and 4 in the vicinity of vehicle 1. This allows vehicle 1 to drive safely and without collisions in the area despite the obstacles 2, 3, and 4. The second illustration, 7, shows a texture analogous to those described previously. Figures 6 and 7 .

[0029] The previously described method optimizes the display based on a detected approach to an obstacle 2, 3, 4, in order to warn the driver in good time before said approach. This leads to increased safety during the operation of vehicle 1, as the risk of collisions with obstacles 2, 3, 4 in the vicinity is reduced.

Claims

1. Method for reproducing an environment of a vehicle (1), comprising the steps of: • detecting (100) an environment of the vehicle (1) and ascertaining obstacles (2, 3, 4) in the environment by means of environmental sensors (8, 9) of the vehicle (1), • displaying (200) a first illustration (5) comprising a representation of the environment with the obstacles (2, 3, 4) and a representation of the vehicle (1) on a display device (10), • determining (300) an approach of the vehicle (1) to an ascertained obstacle (2, 3, 4) by way of which a predefined spacing from the obstacle (2, 3, 4) is undershot, and • displaying (400) a second illustration (6, 7) with a detail of the representation of the environment with the obstacles (2, 3, 4) and of the representation of the vehicle (1) on the display device (10) that is enlarged in comparison to the first illustration (5), wherein the detail represents that region of the environment and the vehicle (1) where the approach of the vehicle (1) to the obstacle (2, 3, 4) was determined, wherein the first illustration (5) and the second illustration (6, 7) represent images from different virtual perspectives on a three-dimensional model of the environment, wherein the three-dimensional model has been generated based on the detected environment and the ascertained obstacles (2, 3, 4), and wherein the second illustration (6, 7) is generated by a change of the virtual perspective and a change of a zoom from the first illustration (5), wherein virtual camera panning is carried out by showing the driver of the vehicle on the display device an animation corresponding to the change of perspective.

2. Method according to one of the preceding claims, characterized in that the first illustration (5) and the second illustration (6, 7) are separate illustrations.

3. Method according to Claim 2, characterized in that the second illustration (6, 7) is illustrated on the display device (10) in such a manner that it superimposes at least a sub-region of the first illustration (5).

4. Method according to Claim 2, characterized in that the first illustration (5) and the second illustration (6, 7) are shown side by side on the display device (10).

5. Method according to one of the preceding claims, characterized in that in the first illustration (5) and / or the second illustration (6, 7) the representation of the obstacle (2, 3, 4) which the vehicle (1) approaches is highlighted.

6. Computer program product comprising instructions which, when executed on an evaluation unit, prompt the evaluation unit to carry out the steps of the method according to one of the preceding claims.

7. Control unit (11) for a vehicle (1), comprising at least one interface for connecting environmental sensors (8, 9) and a display device (10), wherein the control unit (11) is specified to carry out a method according to one of Claims 1 to 5 or a computer program product according to Claim 6.