Methods for increasing the driving safety of a vehicle that is at least partially automated, as well as vehicles

Abstract

Method for increasing the driving safety of a vehicle (10) that is at least partially automated, in which environmental data of a real environment (12) of the vehicle (10) are generated by means of a camera (11) and transmitted to a control unit (13), a hazard analysis relating to an instantaneous driving situation of the vehicle (10) in the environment (12) represented by the environmental data is carried out by means of the control unit (13), a characteristic value is determined based on the hazard analysis which is suitable to assess a potential hazard situation, and depending on the determined characteristic value, an augmented image content (15) is determined and integrated into the real environment (12) from the viewer's perspective by means of a display device (16) to indicate the potential hazard situation;furthermore, a vehicle (10) with at least a partially automated driving function and a control unit (13) which is configured to carry out such a procedure.

Description

[0001] The invention relates to a method for increasing the driving safety of a vehicle with at least partially automated driving capabilities. The invention further relates to a vehicle with at least partially automated driving functions.

[0002] Vehicles are known to implement levels 1 (assisted driving), 2 (partially automated driving), and 3 (highly automated driving) out of a total of five levels on the path to fully autonomous driving. Vehicles implementing level 4 (fully automated driving) are in development. Level 5 refers to autonomous vehicles in which the occupants are merely passengers without any driving responsibilities.

[0003] At Level 2, the driver is in constant control of the vehicle and must continuously monitor traffic. Only certain tasks are performed automatically by the vehicle on a temporary basis – without human intervention. For example, a Level 2 vehicle is capable of simultaneously maintaining its lane, braking, and accelerating on the highway.

[0004] Highly automated vehicles (Level 3) can perform certain driving tasks independently and without human intervention, but only for a limited time and under suitable conditions specified by the manufacturer. These driving tasks can include, for example, overtaking, braking, and accelerating – depending on the traffic situation.

[0005] At Level 4, the driver can completely relinquish control of the vehicle and becomes a passenger. The vehicle handles journeys on certain routes (e.g., highways, parking garages) entirely autonomously. It is then permitted to drive without occupants. Passengers may, for example, sleep, use their smartphones, or read a newspaper. The system recognizes its limitations in time to reach a safe state in accordance with regulations. At the end of a fully automated journey, the occupants can take over control of the vehicle again. However, if they are unable or unwilling to do so, the vehicle must reach a safe state and, for example, head for a parking space.

[0006] Furthermore, so-called "Augmented Reality Head-up Displays" (AR-HUDs) are known. These – like conventional head-up displays – provide display functions close to the vehicle's windshield (e.g., Blackprint Display).

[0007] Furthermore, probability models implemented in software are known which can provide an analysis of a dangerous situation during a vehicle journey.

[0008] For example, DE 10 2017 220 268 A1 describes a method for detecting and visualizing system uncertainty when displaying augmented image content in a motor vehicle's head-up display (AR-HUD). The head-up display serves to show contact-analogous navigation and warning or hazard information to a driver of the motor vehicle, displaying information according to their position and orientation in the real world.

[0009] DE 10 2009 003 220 A1 discloses a method and a device for collision warning. The method comprises contact-analogous marking of an object representing a concrete collision hazard by means of a visual display for the driver of a motor vehicle.

[0010] Against this background, the invention aims to increase the driving safety of a vehicle that drives at least partially automatically. Furthermore, the invention aims to provide a vehicle with at least partially automated driving functionality whose driving safety is enhanced.

[0011] This problem is solved by a method having the features of claim 1 and by a vehicle having the features of claim 4. Further particularly advantageous embodiments of the invention are disclosed in the respective dependent claims.

[0012] It should be noted that the features listed individually in the claims can be combined with one another in any technically meaningful way (even across category boundaries, for example between method and apparatus) and demonstrate further embodiments of the invention. The description further characterizes and specifies the invention, particularly in conjunction with the figures.

[0013] It should also be noted that the conjunction “and / or” used herein, which stands between two features and links them together, is always to be interpreted in such a way that in a first embodiment of the object according to the invention only the first feature may be present, in a second embodiment only the second feature may be present, and in a third embodiment both the first and the second feature may be present.

[0014] The invention relates to a method for increasing the driving safety of a vehicle that is at least partially automated. Environmental data from the vehicle's real-world environment is generated by a camera and transmitted to a control unit. Using the control unit, which is preferably an electronic control unit with a microprocessor, a microcontroller, a digital signal processor (DSP), or similar device, a hazard analysis is performed based on the vehicle's current driving situation within the environment represented by the environmental data. Furthermore, a characteristic value is determined based on the hazard analysis, which is suitable for evaluating a potential hazard situation. In this context, the potential hazard situation does not represent an actual collision situation that would require action by the vehicle's driver.Rather, the term "potential hazard" refers to a situation (also referred to here as a "hotspot") that could develop into an acute collision situation only upon the occurrence of one or possibly several further events. A hotspot is therefore an area in the real environment that the vehicle is likely to enter and where a potential hazard to road traffic could arise.

[0015] The procedure further stipulates that, depending on the determined parameter, an augmented image is generated and integrated into the real environment by means of a display device to indicate the potential hazard situation from the perspective of an observer (e.g., a driver). Such a display device can also be referred to as a field-of-view display.

[0016] Augmented image content (e.g., symbols, indicators) refers to content where information is displayed according to its position and orientation in the real environment, in order to give the viewer the impression of a natural image. Accordingly, the augmented image content is preferably displayed near the window, e.g., near the windshield of the vehicle, through which the viewer perceives the vehicle's real surroundings. However, the invention is not necessarily limited exclusively to the windshield. For example, displaying augmented image content near a side or rear window of the vehicle is also conceivable.

[0017] Partially automated driving corresponds to level 2 of the five levels mentioned above, leading up to autonomous driving (level 5).

[0018] It is important to understand that hazard analysis is not limited solely to the environmental data generated by the camera. Other vehicle sensors, such as radar, lidar, ultrasonic sensors, and the like, can provide additional environmental data that can be considered in the hazard analysis. Furthermore, vehicle data, such as current speed, acceleration, direction of travel, etc., can be incorporated into the hazard analysis.

[0019] The invention makes it possible to direct the attention or gaze of the viewer, e.g., the driver, to specific situations in road traffic. This direction is achieved through the displayed augmented image content resulting from the hazard analysis. In this way, the driver of the vehicle is alerted to potential hazards before intervention is necessary. Accordingly, the method according to the invention is to be understood merely as an indicator, not as an action-guiding method.

[0020] As previously described, the driver of a vehicle with semi-automated driving functions (i.e., Level 2) must always keep their attention and eyes on the road while driving. However, they do not know what might require the vehicle to take over, i.e., actively intervene. Furthermore, the driver does not know whether the vehicle is correctly assessing or registering potential hazards. The invention closes these information gaps. The driver can now pay closer attention to areas of the road that contain a potential hazard and thus reliably fulfill their Level 2 monitoring task, thereby increasing overall driving safety.

[0021] In an advantageous embodiment, the indicator assesses the potential hazard situation based on different urgency levels, with each urgency level being assigned a different color in which the augmented image content is displayed by the display device. For example, the augmented image content can be displayed in white in a case where there is no particular urgency, i.e., where only normal attention is required from the viewer, and in a case with a higher level of urgency or increased attention required, in a different color, e.g., red. Accordingly, a targeted prioritization of attention among several potential hazard situations can be implemented, further improving driving safety. For example, the prioritization can be based on distance ora travel time until reaching the area of ​​the potential hazard situation is taken into account, whereby, for example, the potential hazard situation that will be reached next by the vehicle is given higher priority.

[0022] Further preferred embodiments provide that the indicator is suitable for evaluating several potential hazard situations, wherein an augmented image content is determined for each potential hazard situation and displayed by the display device to identify the respective hazard situation. In this way, several potential hazard situations can be simultaneously indicated by different augmented image content in the viewer's field of vision, so that the driver is alerted to different potential hazard situations at an early stage.

[0023] Furthermore, the invention relates to a vehicle with at least a partially automated driving function, i.e., a driving function that corresponds at least to level 2 of the five levels described herein up to fully autonomous driving. The vehicle has a camera for generating environmental data of the vehicle's real environment, and a control unit configured to receive the environmental data and perform a hazard analysis related to the vehicle's current driving situation in the environment represented by the environmental data, to determine a characteristic value based on the hazard analysis that is suitable for evaluating a potential hazard situation, and, depending on the determined characteristic value, to generate an augmented image and display it integrated into the real environment by means of a display device from the perspective of an observer (e.g., the driver) to indicate the hazard situation.

[0024] It is understood that, with regard to vehicle-related definitions of terms as well as the effects and advantages of vehicle-specific features, full reference can be made to the disclosure of analogous definitions, effects, and advantages of the method according to the invention, and vice versa. A repetition of explanations of analogous features, their effects, and advantages can therefore be omitted in favor of a more concise description, without such omissions being to be interpreted as a limitation of any of the disclosed subject matter of the invention.

[0025] In a preferred embodiment, the display device is designed as an augmented reality head-up display (AR-HUD). This displays the augmented image content on a vehicle window through which the viewer sees the real surroundings (e.g., windshield, side window, rear window).

[0026] In a further preferred embodiment, the display device is configured to display the augmented image content on a windshield of the vehicle.

[0027] A further advantageous embodiment provides that the vehicle has a highly or fully automated driving function, accordingly a driving function of levels 3 or 4 described herein. At levels 3 or 4, the driver may have limited confidence in the driving functions implemented in the vehicle, which is why monitoring the driving situation by the driver may be desirable, even though the road traffic does not need to be monitored by the driver. The invention further increases confidence, since the driver can always reliably verify whether the vehicle automatically and reliably detects potential hazards. In this way, unnecessary, sudden interventions by the driver and, consequently, abrupt and potentially unsafe driving maneuvers by the vehicle can be avoided, thus increasing the vehicle's driving safety.

[0028] Further features and advantages of the invention will become apparent from the following description of exemplary embodiments of the invention, which are not to be understood as limiting and are explained in more detail below with reference to the drawing. This drawing shows the only Fig. 1 schematically a part of a vehicle according to an embodiment of the invention in which a method according to an embodiment of the invention is implemented.

[0029] The in Fig. 1 The vehicle 10 shown has at least a partially automated driving function, i.e. a driving function according to levels 2, 3 or 4 described herein.

[0030] Fig. It can be seen from Figure 1 that the vehicle 10 has at least one camera 11. This camera serves to generate environmental data of a real environment 12 of the vehicle 10. Furthermore, the vehicle 10 has an electronic control unit 13 which is configured to receive the environmental data and to perform a hazard analysis related to a current driving situation of the vehicle 10, as exemplified in Figure 1. Fig. 1 is shown viewed through a windshield 14 of the vehicle 10, to be carried out in the environment 12 represented by the environmental data.

[0031] The control unit 13 is further configured to determine a characteristic value based on the hazard analysis, which is suitable for evaluating at least one potential hazard situation. Depending on the determined characteristic value, the control unit 13 determines an augmented image content 15, which is displayed in Fig. 1 is shown as an elliptical marking, for example. Other geometric figures, symbols, indicators, and the like are conceivable. To indicate the potential hazard situation, the augmented image content is displayed by means of a display device 16, which can, for example, be designed as an augmented reality head-up display, from the perspective of a viewer, e.g., a person in Fig. 1. The driver of vehicle 10, not shown, is integrated into the real environment 12, as shown in Fig. 1 can be seen.

[0032] The in Fig. The potential danger situation depicted in Figure 1 could, for example, consist of an object or a person, e.g., a child, falling between the [objects / objects] shown. Fig. The augmented image content 15, shown in Figure 1, can be displayed on the side of the road, drawing the driver's attention to this area early and specifically. This example illustrates that the display of the augmented image content 15 does not require any specific action or intervention on the part of the driver. It merely draws their attention to the potential hazard. Furthermore, the driver of vehicle 10 recognizes that the vehicle's at least partially automated driving function is capable of detecting such a potential hazard, thereby strengthening the driver's confidence in the reliability of the at least partially automated driving function.

[0033] An embodiment of the method according to the invention, which is carried out in vehicle 10 from Fig. 1 is implemented, i.e., its control device 13 is configured to execute the method according to the invention, and comprises the following steps: - Generating environmental data of the real environment 12 of the vehicle 10 using the camera 11, - Transferring the generated environmental data to the control unit 13, - Performing a hazard analysis related to an instantaneous driving situation of the vehicle 10 in the environment 12 represented by the environmental data by the control unit 13, - Determining a characteristic value based on the hazard analysis, whereby the characteristic value is suitable for assessing a potential hazardous situation, - Determining an augmented image content 15 depending on the determined characteristic value and - Displaying the augmented image content 15 to indicate the potential danger situation by means of the display device 16 (e.g. AR-HUD) from the perspective of a viewer (e.g. the one in Fig. 1 (driver not shown) integrated into the real environment 12. REFERENCE MARK LIST: 10 vehicles 11 Camera 12 Real-world environment 13 Control unit 14 Windscreen 15 Augmented Image Content 16 Display device QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2017 220 268 A1

[0008] DE 10 2009 003 220 A1

[0009]

Claims

Method for increasing the driving safety of a vehicle (10) that is at least partially automated, in which environmental data of a real environment (12) of the vehicle (10) are generated by means of a camera (11) and transmitted to a control unit (13), a hazard analysis relating to an instantaneous driving situation of the vehicle (10) in the environment (12) represented by the environmental data is carried out by means of the control unit (13), a characteristic value is determined based on the hazard analysis which is suitable to assess a potential hazard situation, and depending on the determined characteristic value, an augmented image content (15) is determined and integrated into the real environment (12) from the viewer's perspective by means of a display device (16) to indicate the potential hazard situation. Method according to claim 1, wherein the characteristic value assesses the potential danger situation on the basis of different urgency levels, wherein each urgency level is assigned a different color in which the augmented image content is displayed by the display device (16). Method according to claim 1 or 2, wherein the characteristic value is suitable for evaluating several potential hazard situations, wherein for each potential hazard situation an augmented image content (15) is determined and displayed by means of the display device (16) to identify the respective hazard situation. Vehicle (10) with at least a partially automated driving function, comprising a camera (11) for generating environmental data of a real environment (12) of the vehicle (10), a control unit (13) which is configured to receive the environmental data and to perform a hazard analysis relating to an instantaneous driving situation of the vehicle (10) in the environment (12) represented by the environmental data, to determine a characteristic value based on the hazard analysis which is suitable for evaluating a potential hazard situation, and depending on the determined characteristic value, to determine an augmented image content (15) and to display it integrated into the real environment (12) from the viewer's perspective by means of a display device (16) to indicate the potential hazard situation. Vehicle according to claim 4, wherein the display device (16) is designed as an augmented reality head-up display. Vehicle according to claim 4 or 5, wherein the display device (16) is configured to display the augmented image content (15) on a windscreen (14) of the vehicle (10). Vehicle according to one of claims 4 to 6, which has a highly or fully automated driving function.

Images