Methods for identifying objects related to vehicle safety
By combining vehicle environment sensors with line-of-sight acquisition and machine learning models, information about objects related to safety is identified and stored, solving the problem that vehicle sensors cannot assess the correlation of objects and improving vehicle safety performance and navigation experience.
Patent Information
- Application Number
- CN202280039771.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2021-06-07
- Filing Date
- 2022-04-20
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2042-04-20
AI Technical Summary
In existing technologies, vehicle environment sensors cannot effectively assess the correlation between identified objects and the vehicle, especially in navigation and driving tasks, resulting in the inability to fully utilize the information from these objects.
By collecting signals from vehicle environmental sensors and combining them with a gaze acquisition device to identify the duration of occupants' gaze, the correlation of objects is determined. Machine learning models are then used to identify road signs and landmarks, and safety-related trajectory and object information is stored in a digital map for use by driver assistance systems.
It improves vehicle safety performance, especially the functions of driver assistance systems, by recognizing and storing the trajectories and information of objects involved in safety, thereby enhancing navigation and the driving experience.
Smart Images

Figure CN117413302B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a method for identifying objects related to vehicle safety in the environment surrounding a vehicle, wherein the surrounding environment and objects located therein can be detected based on signals collected by the vehicle's environmental sensors. Background Technology
[0002] DE 10 2019 004 075 A1 discloses a method for determining the correlation between an object in the surrounding environment of a motor vehicle and the vehicle using a driver assistance system, as well as such a driver assistance system. The method involves using the electronic computing device of the driver assistance system to compare objects detected by the system's detection equipment in the surrounding environment with objects stored in the system's storage device, and marking the detected objects based on the comparison results. Simultaneously, the driver assistance system uses a gaze direction acquisition device to acquire the driver's gaze direction, and determines the correlation of the marked objects based on the acquired gaze direction. Summary of the Invention
[0003] The purpose of this invention is to provide a method for identifying objects that are related to vehicle safety.
[0004] According to the present invention, this objective is achieved by a method having the features described in claim 1.
[0005] Advantageous designs of the present invention are the subject of the dependent claims.
[0006] A method for identifying objects related to vehicle safety in the environment surrounding a vehicle, particularly for identifying traffic routes, road signs, and landmarks, is designed to detect the surrounding environment and objects located therein based on signals collected by vehicle environmental sensors. According to the invention, if a vehicle-mounted gaze acquisition device detects that the gaze of at least one occupant, and in particular all occupants, lingers on an object for a period exceeding a preset minimum observation duration, then the object identified by the vehicle's object recognition device and temporarily classified as unrelated to the vehicle will be classified as potentially relevant. Furthermore, based on the detected gaze directions of at least one occupant, and in particular all occupants, it can be determined that at least one occupant, and in particular all occupants, tracked a potentially relevant moving object for at least a preset duration. If a vehicle-based response to the object is required, such as through steering and / or braking or acceleration by the driver or a driver assistance system, the trajectory formed by the detected and potentially relevant moving object after the preset duration will be identified as related to vehicle safety.
[0007] By applying this method, safety performance can be improved by paying attention to objects on a safety-related trajectory, particularly in relation to driver assistance systems. In this process, the correlation between the various objects is related to the vehicle's driving task.
[0008] This method allows for the easy identification of objects as being involved in vehicle safety based on trajectories previously identified as being involved in safety. Specifically, if an object is on a previously identified safety-related trajectory, it will subsequently be identified as being involved in safety.
[0009] In one embodiment of this method, the trajectory of a moving object identified as involving safety—as a traffic route involving safety—is stored in the vehicle's digital map and / or in a computer unit coupled to the vehicle using data technology. Furthermore, such information is accumulated in the digital map through a relatively large number of trips; specifically, typically, if an object is classified as involving safety, then its trajectory with location information is marked, for example, plotted, in the digital map. That is, the relevance of such traffic routes in the digital map is repeatedly verified before storage, after which the traffic route is stored in the digital map as a route involving safety.
[0010] In one improvement to this method, digital maps stored in the vehicle and / or computer unit are readily available for use by other vehicles. Simultaneously, these tracks are provided to driver assistance systems so that, in the future, objects located on these tracks can be identified as relevant to a driving task of the vehicle and other vehicles.
[0011] Therefore, in another embodiment, an object detected at a later time can be classified as a relevant object based on whether it lies on a stored safe traffic route. If an object detected later lies on a previously determined safe trajectory, then that object is a safe moving object for the vehicle.
[0012] Furthermore, one possible embodiment of this method is designed such that, in order to identify road signs and / or attractions, a model is used to classify specific objects as potentially relevant static objects. These potentially relevant static objects are then displayed to at least one occupant, who in turn categorizes them as road signs and / or attractions. In other words, road signs and attractions are distinguished; road signs can be used for navigation, while attractions can enhance the driving experience.
[0013] In addition, in one improvement to this method, static objects classified as potentially relevant are stored as road signs and / or attractions in a digital map. This allows the corresponding road signs and / or attractions to be displayed in the vehicle and can be used for navigation (e.g., during a rally) and to enhance the driving experience (e.g., when driving to attractions). Attached Figure Description
[0014] Embodiments of the present invention will now be explained in more detail with reference to the accompanying drawings.
[0015] in:
[0016] Figure 1 The illustration shows a section of road with two lanes facing each other, and a vehicle traveling in the right lane. Detailed Implementation
[0017] This single attached diagram shows a road segment F with two oncoming lanes F1 and F2. Vehicle 1 is traveling in the right lane F1, ahead of vehicle 2, while vehicle 3 is traveling in the opposite direction to vehicle 2 in the left lane F2.
[0018] On the road segment F, directly in front of vehicle 1, an on-ramp F3 merges into the right lane F1, where a third vehicle 4, acting as the first object O1, intends to enter the right lane F1. A rural lane F4 extends from the left lane F2, on which a tractor 5, acting as the second object O2, is traveling towards the left lane F2.
[0019] Next to F1 on the right, there are two other objects, O3 and O4. The third object, O3, is a palm tree 6, and the fourth object, O4, is a sightseeing tower 7.
[0020] Vehicle 1 has an environmental sensor 1.1, which includes multiple detection units, not further described, installed within and / or on vehicle 1. For example, the detection units may be designed as lidar-based, radar-based, ultrasonic-based sensors and / or as multi-functional cameras.
[0021] The environmental sensor 1.1 has a field of view S1, while the other field of view S2 of the occupants of vehicle 1 almost coincides with the field of view S1 of the environmental sensor 1.1, or rather, a portion of the field of view S1 of the environmental sensor 1.1. In particular, there are two occupants on vehicle 1.
[0022] Based on the signals collected by the environmental sensor 1.1, the surrounding environment of the vehicle 1 and objects O1 to O4 located in the surrounding environment can be detected. The objects O1 to O4 can be identified by means of the object recognition device 1.2 of the vehicle 1.
[0023] In this process, the following problem may arise: without background information for each of the identified objects O1 to O4, it is impossible, or rather, impossible, to fully evaluate the relevance of these objects to vehicle 1. Here, relevance refers to both a driving task and objects O3 and O4 used for navigation and / or as attractions.
[0024] To determine the correlation between each object O1 to O4 and vehicle 1, the following method is provided.
[0025] According to this method, all identified objects O1 to O4 are selected, and these objects are classified as irrelevant to the driving task by the driver assistance system of vehicle 1 (which is not further described). These objects O1 to O4 will be temporarily classified as potentially relevant.
[0026] The driver assistance system includes a satellite-based positioning unit, navigation data, environmental sensors 1.1, an object recognition device 1.2, and an in-vehicle camera 1.3 with a gaze acquisition device 1.4. Typically, the driver assistance system selects objects O1 through O4 and discards others to reduce computation time. In one possible embodiment, this could be a driver assistance system for automated (e.g., highly automated) driving operations of vehicle 1.
[0027] Specifically, if the vehicle-side gaze acquisition device 1.4 detects that, at a certain moment t1 within a preset time period, the occupant's gaze lingers on objects O1 to O4 for a duration exceeding a preset minimum observation time, then these objects O1 to O4 will be classified as potentially relevant. The prerequisite here is that the field of view S1 of the environmental sensor 1.1 is substantially consistent with the occupant's other field of view S2.
[0028] The traffic route design for detecting the safety of vehicle 1 involves observing whether, upon identifying moving objects O1 and O2, objects O1 and O2 become relevant to the driving task of vehicle 1 due to their own motion and the motion of vehicle 1 (i.e., continued driving). For example, tractor 5, as the second object O2, may enter the road segment F from the rural lane F4, and become relevant to the safety of vehicle 1 at another moment t2 after a preset time period has ended.
[0029] Similarly, in ramp F3, the third vehicle 4, which is the first moving object O1, will also become related to vehicle 1 at another time t2. In particular, if the trajectory T of the third vehicle 4 and / or tractor 5 requires vehicle 1 to react to the corresponding objects O1, O2, for example by steering and / or braking or acceleration with the help of the driver or driver assistance system, then after the preset time period, the trajectories T of the moving objects O1, O2 that are identified and classified as potentially related to vehicle 1 are identified as involving the safety of vehicle 1.
[0030] The moving objects O1 and O2 are identified as trajectories T involving safety, and after repeated confirmation of their relevance to the driving task (e.g., by confirmation from other vehicles in the convoy to which vehicle 1 belongs but not shown in detail), they will be stored in the digital map as traffic routes involving safety.
[0031] This digital map is stored in vehicle 1 and / or a computer unit (not shown in detail) coupled to the vehicle using data technology. For example, the digital map containing information about safe traffic routes can be provided to other vehicles via vehicle 1 and / or via the computer unit. This digital map is particularly useful when another vehicle is located on the corresponding road segment F. This information about safe traffic routes can especially be provided to driver assistance systems.
[0032] For the identification of road signs and / or attractions, the method is designed to classify the object O3 identified by the object recognition device 1.2 as a static object O3 of the palm tree 6 style, and the identified object O4 as a static object O4 of the observation tower 7 style, by means of a stored model (especially a machine learning model).
[0033] Next, the objects O3 and O4 will be displayed to the occupants of vehicle 1, particularly on a display device (e.g., in the dashboard area). This allows the occupants to categorize objects O3 and O4 as road signs and / or landmarks. For example, a query may be inserted on the display device. In this context, such road signs are particularly relevant to the navigation of vehicle 1, especially in situations like rallies where navigation using road signs is required.
[0034] This is also designed to store road signs and attractions in a digital map, which is then made available to vehicle 1 and, if necessary, to other vehicles in the same convoy.
Claims
1. A method for identifying objects (O1 to O4) related to the safety of a vehicle (1) in the surrounding environment of the vehicle (1), wherein, The vehicle (1) detects the surrounding environment and objects (O1 to O4) in the surrounding environment based on signals collected by the environmental sensors (1.1). Its features are, -If the vehicle-side gaze acquisition device (1.4) detects that at least one occupant of the vehicle (1) has their gaze fixed on the corresponding object (O1 to O4) for a period exceeding the preset minimum observation duration, then the objects (O1 to O4) identified by the vehicle's object recognition device (1.2) and temporarily classified as unrelated to the vehicle (1) will be classified as potentially related. - Based on the detected gaze direction of the at least one occupant, it is determined that the at least one occupant tracks moving objects (O1, O2) classified as potentially relevant for at least a preset time period. -If it is necessary to use the vehicle (1) to react to the corresponding objects (O1, O2), then the trajectory (T) of the moving objects (O1, O2) that are collected and classified as potentially related to the vehicle (1) after the preset time period ends is identified as involving the safety of the vehicle (1). The trajectory (T) of moving objects (O1, O2) is identified as a safety-related traffic route and stored in a digital map of the vehicle (1) and / or in a computer unit coupled to the vehicle (1) using data technology.
2. The method according to claim 1, characterized in that, The digital map stored in the vehicle (1) and / or computer unit is made available to other vehicles.
3. The method according to claim 1 or 2, characterized in that, Other objects detected at later times will be classified as relevant objects based on whether they are located on stored safe traffic routes.
4. The method according to claim 1 or 2, characterized in that, In order to identify road signs and / or attractions, the corresponding objects are classified into potentially related static objects (O3, O4) using a model. The static objects (O3, O4) classified into potentially related static objects are displayed to at least one occupant, who classifies the static objects (O3, O4) into road signs and / or attractions respectively.
5. The method according to claim 4, characterized in that, Static objects (O3, O4) that will be classified as potentially relevant will be stored in the digital map as landmarks and / or attractions.
Citation Information
Patent Citations
Method for determining the relevance of an object in the environment of a motor vehicle using a driver assistance system and driver assistance system
DE102019004075A1
Passenger support method, passenger support system, and vehicle
DE102017206585A1
Method for determining an object perception probability with which a driver of a motor vehicle perceives objects in an environment of the motor vehicle, as well as control device for a motor vehicle and motor vehicle
DE102018200980A1
Autonomous driving using standard navigation map and lane configuration determined based on past trajectory of vehicle
JP2020064611A
