Method for recognizing safety-related objects in a vehicle - Patents.com

Abstract

The invention relates to a method for recognizing objects (O1-O4) in the vicinity of a vehicle (1) that are relevant to the safety of the vehicle (1). The surroundings and objects (O1-O4) in this surroundings are detected from the detection signals of the surrounding sensor system (1.1) of the vehicle (1). When the gaze detection device (1.4) on the vehicle side recognizes that the gaze of the vehicle occupant of the vehicle (1) remains on each object (O1-O4) for longer than a predetermined minimum gaze duration, each object (O1-O4) recognized by the object recognition device (1.2) of the vehicle (1) and initially classified as not relevant to the safety of the vehicle (1) is classified as potentially relevant, and from the detected gaze direction of the vehicle occupant it is determined that each classified moving object (O1, O2) is tracked by the vehicle occupant for a predetermined period of time, and if a reaction is required by the vehicle (1) to each object (O1, O2), the trajectory (T) of each classified moving object (O1, O2) is identified as relevant to the safety of the vehicle (1) after the lapse of a predetermined period of time.

Description

[Technical Field]

[0001] The present invention relates to a method for recognizing objects relevant to the safety of a vehicle in the vicinity of the vehicle, in which the surroundings and the objects in the vicinity are detected on the basis of detection signals of the surrounding sensor system of the vehicle. [Background technology]

[0002] German Patent Application No. DE 10 2019 004 075 A1 discloses a method for determining the relevance of an object in the vicinity of a motor vehicle to the motor vehicle using a driver assistance system of the motor vehicle, and such a driver assistance system. The method provides that objects detected by a detection device of the driver assistance system in the vicinity of the motor vehicle are compared by an electronic computer of the driver assistance system with objects stored in a memory device of the driver assistance system, and the objects are characterized by the electronic computer depending on the comparison. In this case, the gaze direction of the motor vehicle driver is detected by a gaze direction detector of the driver assistance system, and the relevance of the characterized object is determined depending on the detected gaze direction. Summary of the Invention [Problem to be solved by the invention]

[0003] The invention is based on the problem of providing a method for recognizing safety-relevant objects in a vehicle. [Means for solving the problem]

[0004] This problem is solved according to the invention by a method having the features set forth in claim 1.

[0005] Advantageous embodiments of the invention are the subject matter of the dependent claims.

[0006] A method for recognizing objects relevant to vehicle safety in the vicinity of a vehicle, particularly traffic signs, landmarks, and places of interest, is proposed in which the vicinity and objects therein are detected based on detection signals from a vehicle's vicinity sensor system. According to the present invention, if a vehicle-side gaze detection device recognizes that the gaze of at least one vehicle occupant of the vehicle, particularly all vehicle occupants, remains on the respective object for longer than a predetermined minimum gaze duration, each object recognized by the vehicle's object recognition device and initially classified as not relevant to vehicle safety is classified as potentially relevant. In addition, based on the detected gaze directions of at least one vehicle occupant, particularly all vehicle occupants, it is determined that each moving object classified as potentially relevant is tracked by at least one vehicle occupant, particularly all vehicle occupants, for at least a predetermined period of time. If the vehicle needs to react to each object, for example, by steering movement and / or braking or acceleration by the driver or a driver assistance system, the trajectory of each detected moving object classified as potentially relevant to the vehicle is identified as relevant to vehicle safety after the predetermined period has elapsed.

[0007] By applying this method, safety can be increased, especially with regard to the functioning of driver assistance systems, by paying attention to safety-relevant objects on the trajectory, where the relevance of each object is related to the driving task of the vehicle.

[0008] In this way, objects on the identified safety-relevant trajectory can be easily identified as being relevant to the safety of the vehicle, especially if there are objects on the trajectory that were previously identified as being relevant to safety, these objects are recognized as being relevant to safety.

[0009] In one embodiment of the method, the trajectory of each moving object identified as safety-relevant is stored as a safety-relevant traffic signal in a digital map of the vehicle and / or in a computing unit data-technically coupled to the vehicle. For this purpose, whenever an object is classified as safety-relevant, the trajectory is recorded together with position information in the digital map, e.g., plotted, so that such information for a relatively large number of trips is further collected in the digital map. That is, the relevance of the traffic signal is checked several times before the relevance itself is stored in the digital map, and the traffic signal is then stored in the digital map as safety-relevant.

[0010] In one development of the method, the digital map stored in the vehicle and / or in the computing unit is provided to other vehicles, in which case the driver assistance systems are provided with trajectories so that in the future they can consider objects on these trajectories as relevant for the driving task of the vehicle and other vehicles.

[0011] To this end, in another embodiment, a later detected object is classified as relevant depending on its presence in one of the stored safety-relevant traffic directions: if the later detected object is on a trajectory that was previously determined to be safety-relevant, then this object represents a moving object relevant to the safety of the vehicle.

[0012] Further, possible embodiments of the method contemplate using the model to classify each stationary object as potentially relevant to recognize landmarks and / or places of interest, and displaying each stationary object classified as potentially relevant to at least one vehicle occupant, who may rank each stationary object classified as potentially relevant as a landmark and / or place of interest. That is, it is contemplated that landmarks and places of interest may be distinguished, and that landmarks may be used for navigation, and places of interest may enhance the driving experience.

[0013] Furthermore, in one development of the method, the stationary objects classified as potentially relevant are saved in a digital map as landmarks and / or places of interest, whereby each landmark and / or each place of interest is displayed in the vehicle and can be used for navigation, for example during rally driving, and to improve the driving experience, for example by driving to the place of interest.

[0014] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a schematic diagram of a road segment with two oncoming lanes and a vehicle traveling in the right lane. DETAILED DESCRIPTION OF THE INVENTION

[0016] Figure 1 shows a road section F with two oncoming lanes F1 and F2. A vehicle 1 is traveling in the right lane F1, a first vehicle 2 is traveling in front of it, and a second vehicle 3 is traveling opposite in the left lane F2.

[0017] In a section of road portion F located immediately ahead of vehicle 1, an access road F3 merges into right lane F1, and a third vehicle 4, acting as a first object O1, is about to enter right lane F1. A farm road F4 extends from left lane F2, and a tractor 5, acting as a second object O2, is traveling on this farm road F4 in the direction of left lane F2.

[0018] Next to the right lane F1 are two other objects O3 and O4, the third object O3 is a palm tree 6 and the fourth object O4 is an observation tower 7.

[0019] The vehicle 1 has an ambient sensor system 1.1 including a number of detection units, not shown in detail, arranged on the vehicle 1. For example, the detection units are designed as lidar-based, radar-based, ultrasound-based sensors and / or multifunction cameras.

[0020] The surroundings sensor system 1.1 has a field of view S1, and another field of view S2 of the vehicle 1 occupants corresponds approximately to the field of view S1 of the surroundings sensor system 1.1 or to a part thereof. In particular, there are two vehicle occupants in the vehicle 1.

[0021] The periphery of the vehicle 1 and objects O1 to O4 present therein are detected based on the detection signals of the periphery sensor system 1.1, and the object recognition device 1.2 of the vehicle 1 recognizes the plurality of objects O1 to O4.

[0022] In that case, the problem may arise that without contextual information about the individual recognized objects O1-O4, the relevance of those objects for the vehicle 1 cannot be assessed or may be assessed insufficiently, where this relevance is relevant to both the driving task and the objects O3, O4 for navigation purposes and / or as places of interest.

[0023] To determine the relevance of each object O1 to O4 for the vehicle 1, the method described below is contemplated.

[0024] According to this method, all recognized objects O1-O4 that are classified as not relevant to the driving task by a driver assistance system of the vehicle 1, not shown in detail, are selected. These objects O1-O4 are first classified as potentially relevant.

[0025] The driver assistance system comprises a satellite-based positioning unit, navigation data, a surrounding sensor system 1.1, an object recognition device 1.2, and an interior camera 1.3 including an eye gaze detection device 1.4. The driver assistance system typically selects objects O1-O4 and discards other objects, e.g., to reduce computation time. In a possible embodiment, this may be a driver assistance system for automated, e.g., highly automated, driving of a vehicle 1.

[0026] In particular, if the vehicle-side gaze detection device 1.4 recognizes that at a time t1 in a predetermined period the gaze of a vehicle occupant of the vehicle 1 remains on each of the objects O1-O4 for longer than a predetermined minimum gaze duration, these objects O1-O4 are classified as potentially relevant. A prerequisite for this is that the field of view S1 of the peripheral sensor system 1.1 substantially corresponds to the field of view S2 of the vehicle occupant (a field of view separate from the field of view S1 of the peripheral sensor system 1.1).

[0027] With regard to the recognition of traffic guidance related to the safety of the vehicle 1, when the moving objects O1, O2 are recognized, it is intended to observe whether the movement of the respective objects O1, O2 and the movement of the vehicle 1, i.e., in the subsequent course, will make this object relevant for the driving task of the vehicle 1. For example, a tractor 5 as the second object O2 may come from the farm road F4 and travel towards the road section F, and may become relevant to the safety of the vehicle 1 at another time t2 after a predetermined period of time has elapsed.

[0028] A third vehicle 4 as a first moving object O1 on the approach path F3 may also be relevant for the vehicle 1 at another time t2. In particular, the trajectory T of each detected moving object O1, O2 classified as potentially relevant for the vehicle 1 is identified as relevant to the safety of the vehicle 1 after a predetermined period of time has elapsed if, based on the trajectory T of the third vehicle 4 and / or the tractor 5, the vehicle 1 needs to react to the respective object O1, O2, for example by means of a steering movement and / or braking or acceleration by the driver or a driver assistance system.

[0029] The trajectories T of the moving objects O1, O2 identified as being safety relevant are stored in the digital map as safety relevant traffic guidance after being checked several times for their relevance for the driving tasks of other vehicles not shown in detail in the vehicle fleet to which vehicle 1 belongs.

[0030] This digital map is stored in the vehicle 1 and / or in a computing unit (not shown in detail) that is data-technically coupled to the vehicle 1. For example, the digital map together with the stored safety-relevant traffic guidance can be provided to other vehicles via the vehicle 1 and / or the computing unit, for example, in particular when the respective other vehicle is on the corresponding road portion F. In particular, the safety-relevant traffic guidance is provided to the driver assistance systems.

[0031] With regard to the recognition of landmarks and / or places of interest, the method contemplates that, using stored models, in particular machine learning models, each object O3, O4 recognized by the object recognition device 1.2 is classified as a stationary object O3, O4 in the form of a palm tree 6 and an observation tower 7.

[0032] In that case, the respective object O3, O4 is displayed to a vehicle occupant of the vehicle 1, in particular on a display unit, for example in the area of ​​the instrument panel. The vehicle occupant or occupants can then rank the respective object O3, O4 as a landmark and / or place of interest. For example, a question is displayed on the display unit for this purpose. Such landmarks are then particularly relevant for navigation of the vehicle 1, for example for a rally drive navigated by landmarks.

[0033] Again, it is contemplated that the landmarks and places of interest will be stored on a digital map and therefore made available to vehicle 1 and possibly at least other vehicles in the same fleet. [Prior art documents] [Patent documents]

[0034] [Patent Document 1] German Patent Application Publication No. 102019004075

Claims

1. A method for recognizing safety-relevant objects (O1-O4) in the vicinity of a vehicle (1) using a driver assistance system including an object recognition device (1.2) and a gaze detection device (1.4), wherein the vicinity and the objects (O1-O4) in the vicinity are detected on the basis of detection signals of a surrounding sensor system (1.1) of the vehicle (1), comprising: - classifying each of the objects (O1-O4) recognized by the object recognition device (1.2) and initially classified as not relevant to the safety of the vehicle (1) as potentially relevant if the gaze detection device (1.4) recognizes at a time (t1) that the gaze of at least one vehicle occupant of the vehicle (1) remains on the respective object (O1-O4) for longer than a predetermined minimum gaze duration, - if it is determined based on the detected gaze direction of the at least one vehicle occupant that each moving object (O1, O2) among the objects (O1 to O4) classified as potentially relevant is tracked by the at least one vehicle occupant for at least a predetermined period from the time t1, and if it is necessary for the vehicle (1) to react to each moving object (O1, O2), then the trajectory (T) of each detected moving object (O1, O2) classified as potentially relevant for the vehicle (1) is identified as relevant to the safety of the vehicle (1) after the predetermined period has elapsed. A method characterized by:

2. The respective trajectories (T) of the moving objects (O1, O2) identified as being safety-relevant are stored as safety-relevant traffic guidance in a digital map of the vehicle (1) and / or in a computing unit data-technically coupled to the vehicle (1).

2. The method of claim 1.

3. The digital map stored in the vehicle (1) and / or the computing unit is provided to other vehicles.

3. The method according to claim 2.

4. To recognize landmarks and / or places of interest, the driver assistance system classifies each stationary object (O3, O4) as potentially relevant using a stored machine learning model, and the driver assistance system displays each stationary object (O3, O4) classified as potentially relevant on a display unit to at least one vehicle occupant, who can distinguish each stationary object (O3, O4) classified as potentially relevant as a landmark and / or place of interest. The method according to any one of claims 1 to 3, characterized in that

5. The stationary objects (O3, O4) classified as potentially relevant are saved in the digital map as landmarks and / or places of interest.

5. The method according to claim 4, which is dependent on claim 2.

Images