Method for improving driving safety of an at-partially automated

Abstract

The invention relates to a method for increasing the driving safety of an at least partially automated driving vehicle (10), in which environmental data of a real environment (12) of the vehicle (10) is generated by means of a camera (11) and transmitted to a control device (13), by means of which the vehicle (10) is driven by means of the camera (11). A hazard analysis is performed on a current driving situation of the vehicle (10) in the environment presented by the environment data (12), a characteristic value suitable for evaluating a potential hazard situation is determined on the basis of the hazard analysis, and an enhanced image content (15) is determined from the determined characteristic value. And displaying, by means of a display device (16), the enhanced image content from the perspective of the observer in a fused manner into the real environment (12) in order to mark the potentially hazardous situation; the invention further relates to a vehicle (10) having an at least partially automated driving function and a control device (13) which is designed to carry out a method of this type.

Description

Technical Field

[0001] This invention relates to a method for improving driving safety in vehicles with at least partially automated driving capabilities. Furthermore, this invention also relates to a vehicle having at least partially automated driving functions. Background Technology

[0002] On the road to fully autonomous vehicles, vehicles currently known to have achieved Level 1 (assisted driving), Level 2 (partially automated driving), and Level 3 (highly automated driving) out of a total of five levels. Vehicles achieving Level 4 (fully automated driving) are in the planning stages. Level 5 is known as autonomous vehicles, in which vehicle occupants will become passengers who do not need to perform driving tasks.

[0003] In Level 2, the driver is always in control of the vehicle and must continuously monitor traffic conditions. Only some tasks are temporarily performed by the vehicle itself – without human intervention. For example, a Level 2 vehicle can simultaneously maintain its lane, brake, and accelerate on a highway.

[0004] Highly automated vehicles (Level 3) can perform specific driving tasks without human intervention, but only for a limited time and under appropriate conditions preset by the manufacturer. For example, these driving tasks may include overtaking, braking, and acceleration as needed depending on traffic conditions.

[0005] At Level 4, the driver can completely relinquish control of the vehicle and become a passenger. The vehicle navigates a specific route (e.g., a highway, a parking lot) entirely independently. The vehicle can even operate without occupants. For example, passengers can sleep, use their smartphones, or read a newspaper. The system promptly identifies its limits, enabling it to reach a safe state in compliance with regulations. At the end of fully automated driving, the occupant can regain control of the vehicle. However, if the occupant is unable or unwilling to do so, the vehicle must reach a safe state, such as driving towards a parking space.

[0006] In addition, there is the so-called "Augmented Reality Head-Up Display" (AR-HUD). It - similar to a traditional head-up display - provides display functionality near the windshield of a vehicle (e.g., a blackprint display).

[0007] Furthermore, probabilistic models implemented in software are known to provide analysis of hazardous situations during vehicle operation.

[0008] Therefore, for example, DE 10 2017 220 268 A1 describes a method for identifying system insecurity and visualizing system insecurity in a field-of-view display (AR-HUD) of a motor vehicle while displaying augmented image content. The field-of-view display is used to show navigation prompts and warnings or hazard warnings of simulated contact to the driver of a motor vehicle, wherein the information is displayed based on its location and orientation in the real world.

[0009] DE 10 2009 003 220 A1 discloses a method and apparatus for collision warning. The method includes displaying a target representing a specific collision risk as a simulated ground contact marker for the driver of a motor vehicle in the field of view. Summary of the Invention

[0010] In this context, the object of the present invention is to improve the driving safety of at least partially automated driving vehicles. Furthermore, the object of the present invention is to provide a vehicle with at least partially automated driving functions, thereby improving its driving safety.

[0011] This objective is achieved by a method having the features of claim 1 and a vehicle having the features of claim 4. Other particularly advantageous embodiments of the invention are disclosed in the corresponding dependent claims.

[0012] It should be noted that the features listed individually in the claims can be combined with each other in any technically reasonable manner (even across class boundaries, such as between methods and apparatus), and show other design options of the invention. The specification, in particular, additionally sets forth the features of the invention and provides a detailed description of the invention in conjunction with the accompanying drawings.

[0013] Furthermore, it should be noted that the conjunction “and / or” used herein to connect two features should always be interpreted as meaning that in a first design scheme according to the subject matter of the invention, only the first feature may exist, in a second design scheme, only the second feature may exist, and in a third design scheme, both the first and second features may exist.

[0014] The subject of this invention is a method for improving driving safety in vehicles with at least partially automated driving. Here, environmental data of the vehicle's real-world environment is generated by means of a camera and transmitted to a control device. The control device (preferably an electronic control device having a microprocessor, microcontroller, digital signal processor (DSP), etc.) performs a hazard analysis on the vehicle's current driving situation in the environment presented by the environmental data. Furthermore, based on this hazard analysis, characteristic values ​​suitable for assessing potential hazardous situations are determined. Here, a potential hazardous situation, in the sense of this invention, is not an emergency collision situation requiring action from the vehicle's driver. Rather, a potential hazardous situation refers to a situation (also referred to herein as a hotspot) that could only develop into an emergency collision situation upon the occurrence of one or more possible subsequent events. A hotspot is therefore an area in the real-world environment into which the vehicle is expected to enter and where road traffic hazards may occur.

[0015] The method also specifies that enhanced image content is determined based on identified feature values, and that this enhanced image content is displayed from the observer's (e.g., driver's) perspective using a display device in a manner blended into the real environment to mark potentially hazardous situations. Such a display device may also be referred to as a field-of-view display.

[0016] Enhanced image content (e.g., symbols, cues) should be understood as content that displays information based on its location and orientation in the real environment to provide a natural visual experience for the observer of the real environment. Therefore, enhanced image content is preferably displayed near the windows through which the observer perceives the vehicle's real environment, such as near the windshield. However, the invention is not necessarily limited to the windshield. For example, it is equally conceivable to display enhanced image content near the side windows or rear windows of the vehicle.

[0017] Partially automated driving corresponds to Level 2, which has already been described at the beginning of the total of 5 levels mentioned up to autonomous driving (Level 5).

[0018] It goes without saying that hazard analysis is not limited to environmental data generated by cameras. Other vehicle sensors (such as radar sensors, lidar sensors, ultrasonic sensors, etc.) can provide additional environmental data that can be considered in hazard analysis. Furthermore, vehicle data (such as current speed, acceleration, and direction of travel) can be incorporated into hazard analysis.

[0019] This invention enables the guidance of an observer's (e.g., a driver's) attention or gaze to specific situations in road traffic. This guidance is achieved by displaying enhanced image content as a result of hazard analysis. In this way, the driver of a vehicle receives a warning of potential hazards before intervention is necessary. Accordingly, the method according to the invention should be understood only as a suggestive method, not a method of directing action.

[0020] As already described, drivers of vehicles with partial autonomous driving capabilities (i.e., Level 2) must constantly keep their attention or line of sight focused on road conditions while driving. However, drivers are unaware of what events might necessitate takeover, i.e., active intervention. Furthermore, drivers cannot ascertain whether the vehicle has correctly assessed or recorded any potential hazards. This invention fills these information gaps. Now, drivers can pay more attention to road traffic areas with potential hazards and thus reliably perform their Level 2 monitoring tasks, thereby improving overall driving safety.

[0021] In an advantageous implementation, the feature values ​​assess potential hazards according to different urgency levels, wherein a different color is assigned to each urgency level, and the display device displays enhanced image content in this color. For example, in cases where the urgency is not particularly high, that is, where only normal observer attention is required, white can be used to display the enhanced image content; in cases where the urgency is relatively high or where increased attention is required, another color, such as red, can be used. Accordingly, targeted prioritization of attention guidance can be implemented among multiple potential hazards, which further improves driving safety. For example, prioritization can be based on distance to the potential hazard area or travel time, wherein, for example, a higher priority is assigned to potential hazards that the vehicle is about to arrive at.

[0022] Another preferred embodiment specifies that the feature values ​​are suitable for assessing multiple potential hazards, wherein enhanced image content is determined for each potential hazard, and the enhanced image content is displayed by means of a display device to mark the corresponding hazard. In this way, multiple potential hazards can be marked simultaneously in the observer's field of vision through different enhanced image contents, thereby allowing the driver to receive early warnings about different potential hazards.

[0023] Furthermore, the subject of this invention is a vehicle with at least partially automated driving capabilities, i.e., a vehicle with driving capabilities equivalent to at least Level 2 of the five levels of driving capabilities currently described up to fully autonomous driving. The vehicle has a camera for generating environmental data of the vehicle's real-world environment. Additionally, the vehicle has a control device designed to: receive the environmental data; perform a hazard analysis on the vehicle's current driving situation in the environment presented by the environmental data; determine characteristic values ​​suitable for assessing potential hazards based on the hazard analysis; determine enhanced image content based on the determined characteristic values; and display the enhanced image content, from the perspective of an observer (e.g., a driver), in a manner integrated into the real-world environment to mark potential hazards using a display device.

[0024] It goes without saying that for vehicle-related terminology definitions and the functions and advantages of vehicle-related features, reference can be made to the disclosures of similar definitions, functions, and advantages of the method according to the invention, and vice versa. Therefore, to make the description more concise, repeated descriptions of features, their functions, and advantages with the same meaning may be omitted, and such omissions should not be construed as limiting any aspect of the subject matter disclosed in this invention.

[0025] In a preferred embodiment, the display device is designed as an augmented reality head-up display (AR-HUD). The AR-HUD displays augmented image content on a vehicle window (e.g., windshield, side window, rear window) through which an observer views the real environment.

[0026] In another preferred embodiment, the display device is designed to display or present enhanced image content on the windshield of the vehicle.

[0027] Another advantageous embodiment specifies that the vehicle has highly automated or fully automated driving functions, correspondingly having Level 3 or Level 4 driving functions described herein. At Level 3 or Level 4, the driver's confidence in the driving functions implemented in the vehicle may be limited, thus the driver may want to monitor driving conditions even though they do not need to monitor road traffic. This invention further enhances this confidence because the driver can reliably know at any time whether the vehicle automatically and reliably identifies potential hazards. In this way, unnecessary sudden intervention by the driver can be avoided, and therefore sudden and potentially unsafe driving operations are avoided, thereby improving vehicle driving safety. Attached Figure Description

[0028] Other features and advantages of the invention will become apparent from the following description of embodiments of the invention, which should not be construed as limiting, and these embodiments will be described in more detail below with reference to the accompanying drawings. In the drawings:

[0029] The onlyFigure 1 A portion of a vehicle according to an embodiment of the present invention is illustrated schematically, showing the method of implementing an embodiment of the present invention in the vehicle. Detailed Implementation

[0030] Figure 1 The vehicle 10 shown has at least partially automated driving functions, that is, driving functions of level 2, level 3 or level 4 as described herein.

[0031] from Figure 1 As can be seen, vehicle 10 has at least one camera 11. This camera is used to generate environmental data of the real-world environment 12 of vehicle 10. Furthermore, vehicle 10 has an electronic control unit 13, which is designed to receive environmental data and monitor the current driving status of vehicle 10 in the environment 12 presented by the environmental data, such as... Figure 1 Hazard analysis is performed on the driving conditions observed through the windshield 14 of the vehicle 10, as exemplarily shown in the example.

[0032] Furthermore, the control device 13 is designed to determine characteristic values ​​suitable for assessing at least one potential hazard based on a hazard analysis. Based on the determined characteristic values, the control device 13 determines the enhanced image content 15, in... Figure 1 In this example, the image content is indicated by an elliptical mark. Other geometric shapes, symbols, indicators, etc., are also conceivable. The image is displayed from the viewer's perspective (e.g., in a virtual environment) via a display device 16 (which may be designed, for example, as an augmented reality head-up display). Figure 1 The enhanced image content is displayed from the perspective of the driver of vehicle 10 (not shown) and integrated into the real environment 12 to mark potential hazards, such as... Figure 1 As can be seen in the text.

[0033] For example, Figure 1 The potential danger shown may be that objects or people (such as children) may... Figure 1 The image suddenly appears between parked vehicles shown. The enhanced image content 15 displayed preemptively and specifically directs the driver's attention to this area. This example demonstrates that no specific action or intervention from the driver is required while the enhanced image content 15 is displayed. It simply directs the driver's attention specifically to the potentially hazardous situation. Furthermore, the driver of vehicle 10 is aware that the vehicle's at least partially automated driving functions are capable of recognizing this type of potential hazard, thereby enhancing the driver's confidence in the reliability of the at least partially automated driving functions.

[0034] exist Figure 1In one embodiment of the method according to the invention, the vehicle 10 implements the method according to the invention, and the vehicle control device 13 is designed to perform the method according to the invention, which includes the following steps:

[0035] - Environmental data of the real environment 12 of the vehicle 10 is generated by using camera 11.

[0036] - The generated environmental data is transmitted to the control device 13.

[0037] - The control device 13 performs a hazard analysis on the current driving situation of the vehicle 10 in the environment 12 presented by the environmental data.

[0038] - Based on hazard analysis, characteristic values ​​are determined, where these characteristic values ​​are suitable for assessing potential hazards.

[0039] - Determine the enhanced image content 15 based on the determined feature values, and

[0040] - Using a display device 16 (e.g., AR-HUD) from the observer (e.g., in Figure 1 The enhanced image content 15 is displayed from the perspective of the driver (not shown) in a way that blends into the real environment 12 to mark potential hazards.

[0041] List of reference numerals

[0042] 10 vehicles

[0043] 11 cameras

[0044] 12 Realistic Environments

[0045] 13 Control devices

[0046] 14 windshields

[0047] 15 Enhanced Image Content

[0048] 16 display devices

Claims

1. A method for improving driving safety in a vehicle (10) with at least partially automated driving, wherein, Using a camera (11), environmental data of the real environment (12) of the vehicle (10) is generated and transmitted to a control device (13). The control device (13) performs a hazard analysis on the current driving situation of the vehicle (10) in the environment presented by the environmental data (12), determines characteristic values ​​suitable for assessing potential hazards based on the hazard analysis, determines enhanced image content (15) based on the determined characteristic values, and displays the enhanced image content from the observer's perspective in a manner that is integrated into the real environment (12) to mark potential hazards using a display device (16).

2. The method according to claim 1, wherein, The feature values ​​assess potential hazards based on different urgency levels, wherein a different color is assigned to each urgency level, and the display device (16) displays the enhanced image content in this color.

3. The method according to claim 1 or 2, wherein, The feature values ​​are suitable for evaluating multiple potential hazards, wherein enhanced image content (15) is determined for each potential hazard and the enhanced image content is displayed by means of a display device (16) to mark the corresponding hazard.

4. A vehicle (10) having at least partially automated driving capabilities, the vehicle having a camera (11) for generating environmental data of a real environment (12) of the vehicle (10) and a control device (13), the control device being designed to: receive the environmental data; perform a hazard analysis on the current driving situation of the vehicle (10) in the environment (12) presented by the environmental data; determine characteristic values ​​suitable for assessing potential hazards based on the hazard analysis; determine enhanced image content (15) based on the determined characteristic values; and display the enhanced image content from the observer's perspective in a manner blended into the real environment (12) to mark potential hazards by means of a display device (16).

5. The vehicle according to claim 4, wherein, The display device (16) is designed as an augmented reality head-up display.

6. The vehicle according to claim 4 or 5, wherein, The display device (16) is designed to display enhanced image content (15) on the windshield (14) of the vehicle (10).

7. The vehicle according to any one of claims 4 to 6, wherein the vehicle has a highly automated driving function or a fully automated driving function.

Images