Improved lane assist

The lane assistance system uses environmental detection and eye tracking to predict lane deviations, ensuring timely and appropriate warnings by requiring multiple criteria, addressing the issue of inappropriate warnings in existing systems and enhancing driver acceptance and safety.

DE102015217783B4Active Publication Date: 2024-11-07AUMOVIO AUTONOMOUS MOBILITY GERMANY GMBH
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
DE102015217783
Authority / Receiving Office
DE · DE
Patent Type
Patents
Current Assignee / Owner
Filing Date
2015-09-17
Publication Date
2024-11-07
Estimated Expiration
2035-09-17

AI Technical Summary

Technical Problem

Existing driver assistance systems struggle with issuing inappropriate warning messages, leading to driver frustration or safety concerns due to uncertainty in determining when to issue or suppress warnings, particularly in lane departure scenarios.

Method used

A lane assistance system that combines environmental detection and eye tracking to predict lane deviations, issuing warnings only when multiple criteria are met, including driver gaze direction and intentional driving actions, to enhance decision-making reliability.

Benefits of technology

The system significantly reduces the likelihood of issuing incorrect warnings by requiring multiple criteria to be satisfied, ensuring timely and appropriate warnings, thereby improving driver acceptance and safety.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

The present invention relates to a method and a device for assisting a vehicle driver in the lateral guidance of the vehicle. An inventive method for a lane assist function for situation-appropriate warning of a vehicle driver when the vehicle leaves the lane and / or drifting out of the lane is predicted, comprising at least a) an environmental detection system directed in the direction of the ahead driving trajectory, and b) an optical detection system located inside the vehicle, designed for an eye-tracking function of the driver, is characterized by the fact that c) whenever the vehicle leaves the lane and / or drifting out of the lane is predicted, d) a signal is issued to assist and / or warn the driver, whereby e) however, contrary to the above, no signal for support and / or warning is issued to the driver whenever f) the vehicle leaves the lane and / or drifting out of the lane is predicted, and g) a possible reason for the departure and / or predicted drifting from the lane is identified in temporal relation to this event, as well as h) the eye-tracking function detects that the driver has directed his gaze in the direction in which the vehicle is leaving the lane and / or a drifting out of the lane is predicted.
Need to check novelty before this filing date? Find Prior Art

Description

Field of the invention:

[0001] The present invention relates to a driver assistance system for supporting a vehicle driver in the lateral guidance of the vehicle.

[0002] Modern vehicles are increasingly being equipped with driver assistance systems that detect the surroundings as standard. Camera-based systems such as optical parking aids, 360-degree surround view applications, lane departure warning (LDW), and the like are now state-of-the-art.

[0003] As the development trend continues to show, the number of driver assistance systems will continue to increase in order to provide the driver with the best possible support and to improve driving comfort.

[0004] With constantly new driver assistance systems being introduced, which are "very interesting" for the vehicle user during the market launch phase, there is also a great risk that the driver will feel increasingly patronized in the long run if these assistance systems "constantly know better" or issue warning messages.

[0005] This is especially true when the assistance system issues a warning message when it is not appropriate at all.

[0006] The sensitivities of the drivers must be taken into account very carefully here.

[0007] When implementing such systems, special attention must be paid to achieving customer acceptance, a) a warning message is not issued / generated too early, but b) a warning message is not sent / generated too late, whereby c) the transition area / grey area is often very narrow.

[0008] If the warning is given too early, the driver can easily feel patronized or bothered by the assistance system (the system is rejected due to a "malfunction"). If the warning is given too late, the functionality of the assistance system can easily be questioned, even leading to claims for compensation if an accident occurs and the assistance system failed to warn correctly.

[0009] From the document WO 2007 / 145566 A1, a generic assistance system or method for an assistance system is known, in which a supporting driver assistance system is disclosed by means of eye tracking (observation of the driver's eyes using an interior camera) and consideration of other criteria.

[0010] A disadvantage of this solution can be seen in the fact that this assistance system, or method for an assistance system according to the document WO 2007 / 145566 A1, is susceptible to failure, or the results are subject to a certain degree of uncertainty, so that it can easily lead (as described above) to the assistance system issuing or suppressing a warning message when this is not appropriate at all.

[0011] The document DE 10 2009 041 187 A1 describes a method for adapting one or more parameters of a driver assistance system that supports the driver of a motor vehicle. The parameters to be adjusted are a function of the driver's state detected by the driver assistance system. The driver's current gaze direction is detected by a gaze detection device. The period of time during which the driver is not looking at the road is determined from the current gaze direction. The parameter(s) of the driver assistance system are adapted if the determined period exceeds a critical time value.

[0012] DE 102005018697 A1 discloses a method and device for warning a driver in the event of a lane departure. A first evaluation unit detects a lane departure based on image information and / or vehicle data and / or driver activity.

[0013] The object of the invention is to present a solution that is robust against the issuance of erroneous warning messages or the suppression of required warning messages, and thus prevents drivers from becoming permanently fed up with such systems. Solution to the task:

[0014] This object is achieved by means of a method according to claim 1 or a lane assistance device according to claim 2. This solution is perceived by vehicle drivers as a useful assistance system / lane assistance function and should therefore achieve corresponding customer acceptance.

[0015] In order to be able to better classify this narrow transition area / grey area from a system perspective, or to make the right decision on the system side as to whether a warning message is already required or not, the following considerations are considered as a starting point, namely that a) in addition to the classical parameters (will be used later Fig. 1 explained in more detail) according to the state of the art, b) additionally the result of an “eye tracking” in the Fig. 2 described / presented manner (modified logic).

[0016] A method according to the invention for a lane assistance function for warning a driver in a situation-appropriate manner when the vehicle leaves the lane and / or drifting from the lane is predicted, comprising at least a) an environment detection system directed in the direction of the forward driving trajectory, and b) an optical detection system located inside the vehicle, designed for an eye-tracking function of the driver, is characterized in that c) whenever the vehicle leaves the lane and / or drifts from the lane are predicted, d) a signal is given to assist and / or warn the driver, where e) however, notwithstanding the foregoing, no signal for assistance and / or warning is given to the driver when f) the vehicle is leaving the lane and / or is predicted to drift out of the lane, and g) a possible reason for the departure and / or predicted drifting from the lane is determined in a temporal connection with this event, and h) the eye-tracking function detects that the driver has directed his gaze in the direction in which the vehicle is leaving the lane and / or that drifting from the lane is predicted.

[0017] The term eye tracking is understood by experts as a process in which one or more cameras are used to record the direction in which the driver is looking.

[0018] A possible reason for the departure and / or predicted drifting from the lane can be determined according to advantageous embodiments of the invention: - from a significant reduction in speed and / or a steering torque input by the driver; - from the existence of a route-relevant TCM accident report and / or a Car-to-X accident report; and / or - from the detection of a route-relevant obstacle by the environment detection system directed in the direction of travel or an additional environment detection system for obstacle detection, in particular a radar or lidar sensor system.

[0019] According to the invention, the method is characterized in that the large speed reduction and / or a steering torque input is only used and taken into account as a criterion if this action(s) has / have been initiated by the vehicle driver.

[0020] According to the invention, the method is characterized in that an existing route-relevant TCM accident report and / or c2x accident report is only used and taken into account as a criterion if it corresponds to the route section currently traveled by the vehicle.

[0021] According to the invention, the method is characterized in that a route-relevant obstacle detected by the environment detection system directed in the direction of travel is only used and taken into account as a criterion if the original travel trajectory would represent a collision course with respect to the obstacle.

[0022] A further subject of the invention is a lane assistance device for warning a driver in a situation-appropriate manner of a departure and / or predicted drifting of the vehicle from the lane.

[0023] The lane assistance device for the situation-appropriate warning of a driver of a departure and / or predicted drifting of the vehicle from the lane, comprising at least a) an environment detection system directed in the direction of the forward driving trajectory, and b) an optical detection system located inside the vehicle, designed for an eye-tracking function of the driver, and c) an evaluation device for evaluating data from the environment detection system, the internal optical detection system, vehicle sensor data, TCM data, Car-to-X data and / or additional environment detection systems, is characterized by the fact that the evaluation device is designed to d) whenever the vehicle leaves the lane and / or drifts from the lane are predicted, (e) to give a signal of assistance and / or warning to the driver, where f) however, not to issue a signal of assistance and / or warning to the driver when g) the vehicle is leaving the lane and / or is predicted to drift out of the lane, and h) a possible reason for the departure and / or predicted drifting from the lane is determined in a temporal connection with this event, as well as The eye-tracking function detects that the driver has directed his gaze in the direction in which the vehicle is leaving the lane and / or that drifting from the lane is predicted.

[0024] Vehicle sensor data includes, in particular, brake pedal positions from which a large reduction in speed and / or steering wheel movements from which a steering torque input by the driver is recorded and evaluated by the evaluation device.

[0025] Furthermore, the invention comprises a motor vehicle provided with such a lane assistance device.

[0026] Further advantages, features and possible applications of the present invention will become apparent from the following description in conjunction with the Fig. 1 and Fig. 2 illustrated embodiments. All described and / or illustrated features, individually or in any combination, constitute the subject matter or method of the present invention, regardless of their summary in the claims or their references.

[0027] They show: Fig. 1: A principle representation of a flowchart of a method for an LDW function according to the state of the art. Fig. 2: A principle representation of a flow chart for the implementation of the method according to the invention.

[0028] In the Fig. 1 and Fig. 2 assumes a vehicle with an LDW system, which has the necessary (not shown in detail) various communication interfaces / devices, as well as the components mentioned in the description.

[0029] The Fig. 1 shows a principle representation of a flowchart of a method for an LDW function according to the prior art.

[0030] As from the Fig. 1, the functionality is as follows: If an LDW system (10) detects that the vehicle is leaving the roadway (is in danger of leaving) or is crossing a boundary line (is about to be crossed) in step S1, the decision is made as to whether a signal is to be issued (step S6) or not (step S7) based on several parameters, whereby the parameters are interpreted as an indication that leaving the roadway (e.g. moving to the right to avoid an obstacle OR moving to the left to overtake - in each case without indicator) could be intended.

[0031] If it is detected or indicated that the departure from the roadway / crossing of a boundary line could be intentional, no warning signal is issued, as the driver obviously intends to follow this trajectory intentionally and would only be "disturbed" by a warning message.

[0032] The following parameters can be considered as an indication of an intentional lane departure, whereby these are known as state of the art in the corresponding logical connection shown. a) If a vehicle is braked quickly (large reduction in speed), this is to be considered as an indication that the driver is already reacting to a recognized situation (step S2, 1st alternative). b) The same applies if the driver applies a steering torque (step S2, 2nd alternative). c) If there are route-relevant (TCM) accident reports for the route travelled (step S3, 1st alternative). d) If there are route-relevant Car-to-X accident reports for the route travelled (step S3, 2nd alternative). e) If an obstacle has been detected by the lane monitoring system (i.e. the environment detection system that monitors lane keeping or by an additional environment detection system, e.g. radar / lidar) (step S4). f) When the driver looks in the direction in which the vehicle leaves the lane or crosses a boundary line (step S5).

[0033] As from the Fig. 1, the LDW system according to the prior art does not output a signal in step S7 if at least one of the aforementioned indicators is assessed positively (with Y=YES), whereas a hazard warning is output if all of the aforementioned indicators are assessed negatively (with N=NO).

[0034] A disadvantage of this solution can be seen in the fact that this assistance system, or method for an assistance system according to the state of the art, is susceptible to failure, or the results are subject to a certain degree of uncertainty, so that it can easily lead (as described above) to the assistance system issuing or suppressing a warning message when this is not appropriate at all.

[0035] The Fig. 2 shows a principle representation of a flow diagram for implementing a method according to the invention in a vehicle with an improved LDW system (20).

[0036] Deviating from Fig. 1, is in the inventive solution according to Fig. 2, the logic of the parameters to be considered is modified in such a way that at least two criteria must be met so that the LDW system does not issue a warning message (step S7).

[0037] As from the Fig. 2, the functionality is as follows: If the LDW system detects that the vehicle is leaving the lane (threatening to leave) or is crossing a boundary line (will be crossed shortly) in step S1, the decision is made as to whether a signal is to be issued (step S6) or not (step S7) based on several parameters, whereby the parameters are interpreted as an indication that leaving the lane (e.g. moving to the right to avoid an obstacle OR moving to the left to overtake - in each case without the indicator) could be intentional.

[0038] If it is detected or indicated that the departure from the roadway / crossing of a boundary line could be intentional, no signal is issued (step S7), since the driver obviously wants to drive on this trajectory intentionally and would only be "disturbed" by a warning message.

[0039] As already mentioned in Fig. 1 can be produced according to the method according to the invention Fig. 2 the following parameters can also be considered as an indication of an intentional lane departure (only in combination with each other), whereby these are new and inventive in the corresponding logical connection shown. a) If a vehicle is braked quickly (large reduction in speed), this is to be considered as an indication that the driver is already reacting to a recognized situation (step S2, 1st alternative). b) The same applies if the driver applies a steering torque (step S2, 2nd alternative). c) If there are route-relevant (TCM) accident reports for the route travelled (step S3, 1st alternative). d) If there are car-to-x accident reports relevant to the route travelled (step S3, 2nd alternative). e) If an obstacle has been detected by the road monitoring system (step S4).

[0040] Deviating from the state of the art, the following criterion must be met as an additional criterion (AND operation; no OR operation as in the state of the art) in addition to one of the previous / above-mentioned criteria in order for the LDW system to become temporarily "inactive" (does not issue a warning message). f) When the driver looks in the direction in which the vehicle is leaving the lane / is about to leave it or is crossing a boundary line / is predicted to cross it (step S50).

[0041] As from the Fig. 2, the LDW system does not output a signal (step S7) if at least two of the aforementioned indicators are assessed positively (with Y=YES), whereas a hazard warning is always output if only one of the aforementioned indicators is assessed positively (with Y=YES).

[0042] From this (according to the inventive concept of Fig. 2) resulting benefit(s): a) An “AND” (combination) of the evaluation criteria as shown in the attached Fig. 2, contributes significantly to increasing the robustness with regard to decision certainty / degree of decision certainty. b) A warning message will only be suppressed if at least two of the above criteria apply. (c) The proposed combination ensures that a required warning message cannot be inadvertently suppressed by the driver's distracted glance at the road. d) The quality of decision-making reliability is significantly increased compared to the state of the art. Note:

[0043] The term or statement used in the descriptions, “driver looks absentmindedly (apathetically) at the road”, is not a clear indication.

[0044] The following is a comparison of the present invention with the prior art.

[0045] The method / solution concept according to the invention achieves an improvement over the prior art, since faulty functioning is prevented and the degree of decision reliability of an LDW system is significantly increased.

[0046] First, a situation is assumed in which the vehicle drifts or leaves the lane, and that the driver looks absentmindedly (apathetically / randomly) at the road / current driving trajectory ahead.

[0047] Two cases are then distinguished: a) It is assumed that there is ONE obstacle or ONE warning message regarding an obstacle for the currently travelled section of the route, or b) It is assumed that there is NO obstacle or NO warning message regarding an obstacle for the section of road currently being traveled.

[0048] In state-of-the-art systems, malfunctions can easily occur in case b), as a warning message is always suppressed if the driver accidentally or absentmindedly focuses on the current road trajectory and drifts off the road, even if there is NO obstacle or NO warning message regarding an obstacle for the current section of road. In this case, a warning message should not be suppressed, as it represents a real hazard.

[0049] Suppression of a warning message is only permissible if the driver accidentally / absentmindedly focuses his gaze on the current road trajectory and drifts off the road, and there is ONE obstacle or ONE warning message regarding an obstacle for the section of road currently being traveled, since in this case it can be assumed that the driver is still driving the vehicle safely despite the absentminded look, since the driving behavior corresponds exactly to what is expected as a result of the detected / present obstacle.

[0050] The situation is different with the proposed solution according to the invention, in which a faulty function is counteracted or reliably prevented with a modified logic.

[0051] In a method proposed according to the invention with modified logic, a faulty function is prevented because a warning message is only suppressed if, in addition to the "characteristic" "that the driver has accidentally / absentmindedly directed his gaze at the current road trajectory and drifts off the road", another criterion must be met.

[0052] In the method according to the invention, a warning message is consequently NOT suppressed if there is NO obstacle at all or NO warning message regarding an obstacle for the currently traveled section of road. In this case, a warning message must not be suppressed, since there is no reason for the drift (is a warning message appropriate / necessary) and it therefore constitutes a genuinely dangerous situation.

[0053] Suppression of a warning message is only permissible if the driver accidentally / absentmindedly focuses his gaze on the current road trajectory and drifts off the road, and there is ONE obstacle or ONE warning message regarding an obstacle for the section of road currently being traveled, since in this case it can be assumed that the driver is still driving the vehicle safely despite the absentminded look, since the driving behavior corresponds exactly to what is expected as a result of the detected / present obstacle.

Claims

[1] Method for a lane assistance function for the situation-appropriate warning of a driver when the vehicle leaves the lane and / or drifting from the lane is predicted, using and taking into account route-relevant obstacles as a criterion, comprising at least a) an environment detection system directed in the direction of the forward driving trajectory, and b) an optical detection system located inside the vehicle, designed for an eye-tracking function of the driver, characterized by , that c) whenever the vehicle leaves the lane and / or drifts from the lane are predicted, d) a signal is given to assist and / or warn the driver, where e) however, notwithstanding the foregoing, no signal for assistance and / or warning is given to the driver when f) the vehicle is leaving the lane and / or is predicted to drift out of the lane, and g) a possible reason for the departure and / or predicted drifting from the lane is determined in a temporal connection with this event, and h) the eye-tracking function detects that the driver has directed his gaze in the direction in which the vehicle is leaving the lane and / or that drifting from the lane is predicted, where the possible reason for the departure and / or drifting from the lane is determined from the detection of a route-relevant obstacle by the environment detection system directed in the direction of travel or by an additional environment detection system, and a route-relevant obstacle detected by the environment detection system directed in the direction of travel or by an additional environment detection system is only used as a criterion and taken into account if the original driving trajectory would represent a collision course with the obstacle, whereby the possible reason for the departure and / or drifting from the lane is determined from a large reduction in speed and / or a steering torque input by the driver, whereby the large reduction in speed and / or the application of steering torque is only considered and taken into account as a possible reason if this action / actions was / are initiated by the driver, whereby the possible reason for the departure and / or drifting from the lane is determined from the presence of a route-relevant TCM accident report and / or a Car-to-X accident report and where an existing route-relevant TCM accident report and / or c2x accident report is only used and taken into account as a criterion if it corresponds to the route section currently being traveled by the vehicle. [2] Lane assistance device for warning a driver in a situation-appropriate manner of a departure and / or predicted drifting of the vehicle from the lane, comprising at least a) an environment detection system directed in the direction of the forward driving trajectory, and b) an optical detection system located inside the vehicle, designed for an eye-tracking function of the driver, and c) an evaluation device for evaluating data from the environment detection system, the internal optical detection system, vehicle sensor data, TCM data, Car-to-X data and / or additional environment detection systems, characterized by , that the evaluation device is designed to d) whenever the vehicle leaves the lane and / or drifts from the lane are predicted, (e) to give a signal of assistance and / or warning to the driver, where f) however, not to issue a signal of assistance and / or warning to the driver when g) the vehicle is leaving the lane and / or is predicted to drift out of the lane, and h) a possible reason for the departure and / or predicted drifting from the lane is determined in a temporal connection with this event, and i) the eye-tracking function detects that the driver has directed his gaze in the direction in which the vehicle is leaving the lane and / or that drifting from the lane is predicted, where the possible reason for the departure and / or drifting from the lane is determined from the detection of a route-relevant obstacle by the environment detection system directed in the direction of travel or by an additional environment detection system, and a route-relevant obstacle detected by the environment detection system directed in the direction of travel or by an additional environment detection system is only used as a criterion and taken into account if the original driving trajectory would represent a collision course with the obstacle, whereby the possible reason for the departure and / or drifting from the lane is determined from a large reduction in speed and / or a steering torque input by the driver, whereby the large reduction in speed and / or the application of steering torque is only considered and taken into account as a possible reason if this action / actions was / are initiated by the driver, whereby the possible reason for the departure and / or drifting from the lane is determined from the presence of a route-relevant TCM accident report and / or a Car-to-X accident report and where an existing route-relevant TCM accident report and / or c2x accident report is only used and taken into account as a criterion if it corresponds to the route section currently being traveled by the vehicle. [3] Motor vehicle provided with a lane assistance device according to claim 2.

Citation Information

Patent Citations

  • Method and device for warning a driver in the event of lane departure

    DE102005018697A1

  • Method and device for adapting parameters of a driver assistance system

    DE102009041187A1

  • Method and apparatus for determining and analyzing a location of visual interest

    WO2007145566A1