Method for fatigue detection, device and computer program product

Abstract

Method for detecting fatigue in a driver of a vehicle, comprising the following steps: Receiving (231) information (141, 142) about a steering movement of the vehicle via a first interface; Receiving (233) environmental information (144, 145, 146) via a second interface, wherein the environmental information (144, 145, 146) enables a prediction of the occurrence of a lateral movement of the vehicle, wherein the lateral movement is caused by a disturbance acting on the vehicle from the outside, wherein the environmental information (144, 145, 146) represents information provided by a navigation system (122), and wherein the environmental information (144, 145, 146) is stored in and read from a database, a map material of the navigation system (122); and Evaluating (235) the information (141, 142) about the steering movement, taking into account the environmental information (144, 145, 146), to decide whether the information about the steering movement indicates driver fatigue, characterized by the fact that the environment information (144, 145, 146) - information provided by the navigation system (122) about position, direction of travel and speed, information provided about current and wind-relevant route positions, such as tunnel entrances and tunnel exits or bridges, as well as a provided indication of a wind-critical road section that the vehicle is passing through, and information provided about a signal from a gyroscope arranged in the vehicle as well as information provided about a wind direction and strength, and - includes information from an environmental sensor system that provides object lists from networked environmental sensors (121) containing objects, their position, direction of movement and their speed.

Description

State of the art

[0001] The present invention relates to a method for fatigue detection, to a corresponding device and to a corresponding computer program product.

[0002] Scientific studies, such as the one described at http: / / www.angurten.de / news / 321-muedigkeit-am-steuer, suggest that between 10 and 20 percent of serious traffic accidents are due to fatigue and inattention. Driver condition monitoring systems are therefore now standard equipment in luxury vehicles.

[0003] Today's driver state monitoring systems are often based on methods that use a front camera to detect lane markings or a steering angle sensor to observe the driver's steering behavior. One such method is described, for example, in "Proceedings of the 2007 IEEE Intelligent Vehicles Symposium Istanbul, Turkey, June 13-15, 2007, Evaluation of a Smart Algorithm for Commercial Vehicle Driver Drowsiness Detection" by Azim Eskandarian and Ali Mortazavi. Both methods analyze impulsive irregularities ("dead-band events").

[0004] DE 10 2005 026 456 A1 discloses a method and a device for fatigue detection.

[0005] DE 102 41 624 A1 discloses a method and a device for detecting the level of attention of a vehicle driver.

[0006] US 6 919 821 B1 discloses a method and a system for collecting meteorological data.

[0007] DE 10 2009 009 975 A1 (post-publication prior art pursuant to Section 3 (2) PatG) discloses a method for determining the attention of a driver during a journey. Disclosure of the invention

[0008] Against this background, the present invention presents a method for fatigue detection, a device that uses this method, and a corresponding computer program product according to the independent claims. Advantageous embodiments are described in the respective dependent claims and the following description.

[0009] The invention is based on the understanding that fatigue detection can be supported by a navigation system and environmental sensors. This reduces the sensitivity of driver condition detection to external disturbances affecting the vehicle. Such external disturbances can include, in particular, wind and turbulence. According to the invention, the susceptibility of fatigue detection methods to interference can be reduced by using the vehicle's navigation system as a sensor and also evaluating the vehicle's environmental sensors. This approach leverages the fact that current navigation systems calculate the most probable path of the vehicle even when route guidance is not actively being used.

[0010] A fatigue detection system according to the invention can be less sensitive to environmental disturbances and thus operate more reliably by using a navigation system and additional environmental sensors, such as radar, video, or lidar. For example, the navigation system can identify locations with strong wind pressure changes, such as bridges, tunnel entrances and exits, and incorporate these into a fatigue detection algorithm. If the navigation device has a telematics interface, wind direction and speed can also be considered. The evaluation of the gyro signal in the navigation system can also be advantageously used in the algorithm.

[0011] The environmental sensors can detect and take into account the entering and exiting of a slipstream or turbulence of a vehicle, especially a truck.

[0012] The present invention provides a method for detecting driver fatigue in a vehicle, comprising the following steps: receiving information about a steering movement of the vehicle via a first interface; receiving environmental information via a second interface, wherein the environmental information enables a prediction of the occurrence of a lateral movement of the vehicle, the lateral movement being caused by a disturbance acting externally on the vehicle; and evaluating the information about the steering movement, including the environmental information, to decide whether the information about the steering movement indicates driver fatigue.

[0013] This method can be used, for example, in a driver information system or driver assistance system. Using this method, the driver's steering movements, the resulting vehicle movements, and the resulting lane position can be evaluated during driving, allowing conclusions to be drawn about the driver's concentration, fatigue, or drowsiness. The steering movement information can include the temporal progression of the steering angle, steering wheel deflection, the speed at which the steering wheel is moved, and / or the temporal progression of the lane position. This steering movement information indicates how quickly, how strongly, and in what sequence the driver makes steering movements to keep the vehicle in its lane.The disturbance can be a force acting laterally on the vehicle, triggered, for example, by a gust of wind or a sudden change in air pressure. The disturbance can be such that it requires the driver to make a compensating steering input to keep the vehicle in its lane. Alternatively, the disturbance can be a predictable event. For example, it can occur regularly at a predetermined location along a road. This might be the case at bridges, tunnels, or specific landscape features. Such locations, along with the disturbances that typically occur there, such as a crosswind of a specific direction and magnitude, can be provided as environmental information.In this case, relevant environmental information can be stored in a database, such as map data from a navigation system, and retrieved from it. The disturbance can also be caused by events that are not stationary. This can occur, for example, during an overtaking maneuver and the associated pressure change. Such events can be detected by environmental sensors mounted on the vehicle. Using suitable evaluation methods, a disturbance that is likely to be triggered by a detected event can be estimated. Factors that can be considered include, for example, the speed, dimensions, and mass of the vehicle and any object detected by the environmental sensors, as well as the distance and relative speed at which the vehicle passes the object.The disturbance can then be predictively determined using a suitable algorithm or a lookup table before it actually occurs. The environmental information can additionally include information about a time point or time window at which the occurrence of the lateral movement or disturbance is predicted or detected. Depending on how the lateral movement is determined, the associated time can be in the future or the present. The environmental information can be used to adjust one or more threshold values ​​used to evaluate the steering movements performed by the driver. This prevents a driver's reaction to a disturbance from being incorrectly interpreted, for example, as inattention or as an impulsive irregularity in the steering movement caused by fatigue.For the actual assessment of the driver's condition, known methods can be used, such as those described in "Proceedings of the 2007 IEEE Intelligent Vehicles Symposium Istanbul, Turkey, June 13-15, 2007, Evaluation of a Smart Algorithm for Commercial Vehicle Driver Drowsiness Detection" by Azim Eskandarian and Ali Mortazavi. According to the invention, the threshold values ​​used in known methods can be adapted depending on the situation in order to reduce the disruptive influence of disturbances or to utilize the influence of disturbances to enable a more meaningful assessment of the driver's condition. Steering movements can also be ignored, not considered, or attenuated if, during a time period associated with these steering movements, the occurrence of a lateral movement or disturbance is detected or predicted.

[0014] Accordingly, information about the steering movement cannot be evaluated, or can only be evaluated to a limited extent, if the surrounding information indicates the occurrence of a lateral movement. This can prevent misinterpretation of the steering movements performed by the driver.

[0015] According to one embodiment, continuous comparisons of the steering movement over time with a fatigue pattern can be performed to determine continuous fatigue indicators. Each fatigue indicator can serve as an indicator of driver fatigue during a time interval of the steering movement associated with that indicator. Based on these fatigue indicators and incorporating environmental information, it can be determined whether the steering movement indicates driver fatigue. The fatigue pattern can include a characteristic steering movement pattern that typically occurs when the driver is tired or inattentive. For example, the fatigue pattern can define different thresholds with associated time windows that allow for an evaluation of the steering movement over time.The ongoing comparisons can be performed sequentially or overlapping. A new comparison can also be started only after a fatigue indicator has been determined that, according to the fatigue pattern, indicates driver fatigue.

[0016] According to the invention, a malfunctioning fatigue indicator used to determine whether information about the steering movement indicates driver fatigue can be disregarded or only partially considered if the specific fatigue indicator is assigned to a time period for which the environmental information indicates the occurrence of a lateral movement. A malfunctioning fatigue indicator can thus be an indicator that, based on the fatigue pattern, suggests driver fatigue. In reality, however, the driver is not tired but has merely reacted to the occurrence of the lateral movement.

[0017] For example, fatigue indicators can be combined to create a fatigue index. A driver's level of fatigue can then be determined by comparing the fatigue index to a threshold value. For instance, fatigue indicators can be summed or integrated over a specific period, and the driver can be classified as tired if the fatigue index reaches or exceeds a predetermined value.

[0018] The method according to the invention can include a step of providing a warning signal to alert the driver when the steering movement indicates driver fatigue. The warning signal can, for example, be a visual, acoustic, or haptic signal.

[0019] According to the invention, the information about the steering movement can be based on a steering angle, a steering speed, and / or a lane position of the vehicle. The lane position can be determined, for example, by detecting the distance to a lane boundary.

[0020] Environmental information can enable predictions about changes in wind pressure acting on the vehicle. In particular, it can predict a gust of wind acting laterally on the vehicle. This environmental information includes an indication of a wind-prone section of road that the vehicle will traverse. The environmental information is provided by a navigation system. Using the navigation system's data, it is possible to predict when the vehicle will pass through the wind-prone section of road.

[0021] Alternatively or additionally, the environmental information can include an indication of an object passing by. This object could be another vehicle or a road boundary. The environmental information represents data provided by environmental sensors. Data provided by these sensors can be analyzed to perform object detection and to determine distances and relative speeds to detected objects.

[0022] The environmental information represents data provided by a gyroscope located in the vehicle. Using a signal provided by the gyroscope, for example, impulsive accelerations acting on the vehicle can be detected and taken into account when evaluating the steering movement.

[0023] The present invention further provides a device designed to carry out or implement the steps of the method according to the invention. This embodiment of the invention in the form of a device also allows the problem underlying the invention to be solved quickly and efficiently.

[0024] In this context, a device can be understood as an electrical device that processes sensor signals and outputs control signals accordingly. The device may have an interface, which can be implemented in hardware and / or software. In the case of a hardware-based interface, the interfaces can, for example, be part of a so-called system ASIC, which incorporates various functions of the device. However, it is also possible that the interfaces are separate integrated circuits or consist at least partially of discrete components. In the case of a software-based interface, the interfaces can be software modules, which, for example, are present on a microcontroller alongside other software modules.

[0025] A computer program product with program code stored on a machine-readable medium such as semiconductor memory, hard disk memory or optical memory, and used to carry out the method according to one of the embodiments described above, is also advantageous if the program is executed on a control unit.

[0026] The invention is explained in more detail below with reference to the accompanying drawings. These show: Fig. 1 a block diagram of an embodiment of the present invention; Fig. 2 a flowchart of an embodiment of the present invention; and Fig. 3 a monitoring of a driver's steering behavior, according to an embodiment of the present invention.

[0027] In the following description of preferred embodiments of the present invention, the same or similar reference numerals are used for the elements shown in the various figures and having a similar effect, without repeating these elements.

[0028] Fig. Figure 1 shows a basic structure of a system for improving lane- or steering-angle-based fatigue detection by means of navigation data and / or data from environmental sensors, according to an embodiment of the present invention.

[0029] The system comprises a module 102 for drowsiness detection with a core algorithm 104 for fatigue detection and a driver interface 106, which can be designed as a human-machine interface (HMI). The module 102 is coupled with a display device 112, which can be a screen (display), speaker, or haptic device.

[0030] The system further comprises an extension 120 of a known fatigue detection system with one or more environmental sensors 121, a navigation system 122, and map data 123, for example, from a GPS system. Environmental sensors 121 can include, for example, a distance sensor 131, which can determine the distance to a vehicle ahead, and / or sensors 132 for object detection. The navigation system 122 can include a unit 135 for localization and map matching and a telematics module 137. Data can be provided to the navigation system 122 from the map data 123.

[0031] The drowsiness detection module 102 can be configured to receive a steering angle 141 and a shoulder position 142. The shoulder position 142 can, for example, be determined by the environmental sensors 121 and provided to the drowsiness detection module 102.

[0032] The environmental sensors 121 can provide an object list 144 with information about position, direction, speed, and vehicle type to the module 102 for drowsiness detection. The object list 144 can include objects and their characteristics detected by the environmental sensors 121. The navigation system 122 can include a gyro signal 145 and information about potentially wind-relevant environmental objects (points of interest), such as tunnel or bridge entrances or exits.

[0033] The in Fig. The system setup shown in Figure 1 can thus consist of a part 102, which corresponds to the prior art, and an inventive addition 120. The part 102 associated with the algorithm can consist of a control unit with the core module 104 for fatigue detection, which evaluates the steering angle and / or the lane position of the vehicle, and the module 106, which prepares the output for the user. The output is then typically displayed on a screen, via a loudspeaker, or a haptic output device 112.

[0034] The invention supplement 120 consists of the navigation system 122, which provides the algorithm with information about position, direction of travel and speed, information 146 about current and wind-relevant route positions (points of interest), such as tunnel entrances / exits or bridges, information 145 about a signal from the gyro and optionally information about wind direction and strength, and from networked environmental sensors 121, also referred to as environment sensors, which provide object lists 144 with objects, their position, direction of movement and their speed.

[0035] Algorithm 104 for fatigue detection is adapted in such a way that the decision threshold is adjusted in the case of a predicted external disturbance by wind, for example a change in wind pressure due to wind-relevant route positions or a change in wind pressure due to an environmental object. Fig. Figure 3 shows an example of a dead zone event, with corresponding decision thresholds.

[0036] According to one embodiment of the invention, the object lists 144 are generated using a radar or video system that detects slow-moving vehicles along with their relative positions, relative speeds, and sizes. Depending on the relative speed, distance, and size of the object, the duration and magnitude of a potential disturbance are predicted. Dead-zone events detected within this period are either attenuated or not considered at all for the calculation of a fatigue index. For example, the value of such a detected dead-zone event can be deleted or reduced before further processing.

[0037] Information about tunnels, bridges, or wind-sensitive terrain is extracted from digital map 123, and the timeframe of a potential disturbance is also predicted. As with disturbances caused by vehicles, dead-zone events occurring during this period are either mitigated or not considered at all. Knowing the current wind direction and speed can further improve the prediction.

[0038] The gyro can be used alternatively or additionally to detect disturbances, such as impulse-like accelerations acting on the vehicle, and to allow correlated, time-shifted reactions at the steering wheel to be included in the calculation of the fatigue index in a correspondingly attenuated manner.

[0039] Fig. Figure 2 shows a flowchart of a method for detecting driver fatigue in a vehicle, according to an embodiment of the present invention. The method can be, for example, derived from the methods described in Figure 2. Fig. The devices shown in Figure 1 are implemented. In step 231, information about a steering movement of the vehicle is received. This could be a steering angle or the vehicle's position within the lane. In step 233, environmental information can be received. This environmental information can be evaluated to enable a prediction of a lateral movement of the vehicle caused by a disturbance. The lateral movement could, for example, be caused by a gust of wind. The environmental information could, for example, be provided by a navigation system and include information about a specific section of the route that the vehicle will subsequently travel. Alternatively, the environmental information could be provided by environmental sensors and include data about objects that could cause a disturbance acting on the vehicle.The environmental information can be evaluated and used in step 235 to assess the driver's steering movement. Based on this assessment, a decision can be made as to whether the driver is classified as fatigued. If so, a warning signal, such as activating a loudspeaker, can be issued. The environmental information can be incorporated into step 235 of the evaluation process in such a way that a period during which the steering movement is influenced by external disturbances is not considered, or only partially considered, in the assessment of the driver's condition. A fatigue pattern, as described in [reference to relevant document / reference], can be used to assess the driver's condition. Fig. 3 is described.

[0040] Fig.Figure 3 shows a monitoring of a driver's steering behavior according to an embodiment of the present invention. A diagram is shown in which time is plotted on the abscissa and steering wheel speed on the ordinate. The steering wheel speed indicates how fast the driver turns the steering wheel. First thresholds 301 and second thresholds 302 are shown in the diagram. The first thresholds 301 define the width of a tolerance band 305. The second thresholds 302 define a minimum reaction amplitude. The diagram shows a time course of the steering wheel speed 310. Furthermore, a first state 311, a second state 312, and a third state 313 are shown.

[0041] At an initial point in time, marking the first state 311, the steering wheel speed 310 falls below the first threshold 301. The first state 311 defines that the steering wheel speed 310 is within the tolerance 305. This corresponds to the beginning of a potential dead zone. A minimum duration 321 for a dead zone may be specified. If the steering wheel speed 310 is still within the tolerance band 305 after the duration 321 has elapsed, the second state 312 is triggered. The second state 312 defines that a minimum duration has been reached and thus a dead zone has been detected. At a later point in time, the steering wheel speed 310 leaves the tolerance band 305 by exceeding the first threshold 301. This triggers the third state 313. The third state 313 indicates that the steering wheel speed 310 exceeds a tolerance defined by the first threshold 301.After exceeding the first threshold 301, the system waits for a reaction amplitude. The reaction amplitude is defined by the second threshold 302. A maximum waiting time is defined by a maximum duration 323 of the reaction phase. At time 325, the reaction amplitude is exceeded. This indicates that a dead zone event has occurred.

[0042] Thus, to detect the dead zone event, it can first be checked whether the steering wheel speed 310 enters the tolerance band 305, i.e., falls below the threshold value 301 according to this embodiment. If this is the case, it can then be checked in a second step whether the steering wheel speed 310 leaves the tolerance band 305 again before the duration 321 has elapsed, i.e., exceeds the first threshold value 301 according to this embodiment. If this is the case, the first step can be started again. If the steering wheel speed 310 does not leave the tolerance band 305 before the duration 321 has elapsed, it can be checked in a third step when the steering wheel speed 310 leaves the tolerance band 305.If the steering wheel speed 310 leaves the tolerance band 305, a fourth step can be performed to check whether the steering wheel speed 310 passes the second threshold 302 before the maximum duration 323 has elapsed, i.e., whether it exceeds this threshold according to this embodiment. The time durations 321 and 323 can be recorded using a suitable timing device, which is started when the steering wheel speed 310 passes through the respective thresholds 301 and 302.

[0043] Between the first state 311 and the second state 312, the driver may be in a semi-sleep phase, during which they do not make any corrective steering movements. From the third state 313 onward, the driver may be in a phase where they recognize the need for corrective steering movements but can no longer execute them smoothly, potentially resulting in large steering wheel deflections. The dead zone event detected at time 325 may indicate driver fatigue or drowsiness. The occurrence of the described dead zone events can be accumulated over time and integrated into a fatigue index. If the fatigue index exceeds a threshold, a fatigue event is detected, and a warning signal can be provided to alert the driver to their condition.

[0044] The embodiments described and shown in the figures are chosen only as examples. Different embodiments can be combined completely or with respect to individual features. An embodiment can also be supplemented by features of another embodiment. Furthermore, process steps according to the invention can be repeated and carried out in a different sequence than described. If an embodiment includes an "and / or" connection between a first feature and a second feature, this can be interpreted to mean that the embodiment according to one embodiment has both the first feature and the second feature, and according to another embodiment has either only the first feature or only the second feature.

Claims

[1] Method for detecting fatigue in a driver of a vehicle comprising the following steps: Receiving (231) information (141, 142) about a steering movement of the vehicle via a first interface; Receiving (233) environmental information (144, 145, 146) via a second interface, wherein the environmental information (144, 145, 146) enables a prediction of the occurrence of a lateral movement of the vehicle, wherein the lateral movement is caused by a disturbance acting on the vehicle from the outside, wherein the environmental information (144, 145, 146) represents information provided by a navigation system (122), and wherein the environmental information (144, 145, 146) is stored in and read from a database, a map material of the navigation system (122); and Evaluating (235) the information (141, 142) about the steering movement, taking into account the environmental information (144, 145, 146), to decide whether the information about the steering movement indicates driver fatigue, characterized by , that the environment information (144, 145, 146) - information provided by the navigation system (122) about position, direction of travel and speed, information provided about current and wind-relevant route positions, such as tunnel entrances and tunnel exits or bridges, as well as a provided indication of a wind-critical road section that the vehicle is passing through, and information provided about a signal from a gyroscope arranged in the vehicle as well as information provided about a wind direction and strength, and - includes information from an environmental sensor system that provides object lists from networked environmental sensors (121) containing objects, their position, direction of movement and their speed. [2] Method according to claim 1, wherein the information (141, 142) about the steering movement is not evaluated or is only weakened when the environmental information (144, 145, 146) indicates the occurrence of the lateral movement. [3] A method according to any of the preceding claims, comprising carrying out continuous comparisons of a temporal progression of the steering movement with a fatigue pattern to determine continuous fatigue indicators, wherein each of the fatigue indicators serves as an indicator of the driver's fatigue in a time period of the temporal progression of the steering movement associated with the respective fatigue indicator, and wherein, based on the fatigue indicators and taking into account the environmental information (144, 145, 146), it is determined whether the information (141, 142) about the steering movement indicates driver fatigue. [4] Method according to claim 3, wherein the fatigue indicators are combined to form a fatigue index and the fatigue of the driver is determined depending on a comparison of the fatigue index with a threshold value. [5] Method according to any of the preceding claims, comprising a step of providing a warning signal to warn the driver when the steering movement indicates driver fatigue. [6] Method according to one of the preceding claims, wherein the information (141, 142) about the steering movement is based on a steering angle, a steering speed and / or a track position of the vehicle. [7] Method according to one of the preceding claims, wherein the environmental information (144, 145, 146) enables the prediction of a change in wind pressure acting on the vehicle. [8] Method according to one of the preceding claims, wherein the environment information (144) includes an indication of the passing of a moving object. [9] Device configured to perform the steps of a method according to any one of claims 1 to 8. [10] Computer program product comprising program code stored on a machine-readable medium for carrying out the method according to any one of claims 1 to 8 when the program is executed on a control device.

Images