Driver monitoring system for vehicle and method for driver monitoring

Abstract

The embodiment of the invention relates to a driver monitoring system for a vehicle and a method for driver monitoring. A driver monitoring system for a vehicle includes at least an output device, a monitoring module, and a control module. The output device is arranged to output at least one attention alert. The monitoring module is configured to monitor a driver of the vehicle and identify an inattention state of the driver. The control module is arranged to accumulate a duration of the identified inattention state within a finite time window having a predetermined length to determine an accumulated inattention duration for the finite time window, and manipulating at least one output device for outputting at least one attention alert based on the accumulated attention deficit duration and the rate metric of the increase of the accumulated attention deficit duration.

Description

Technical Field

[0001] The present disclosure relates to a driver monitoring system for a vehicle, a vehicle having such a driver monitoring system, a method for driver monitoring, and a storage medium for executing the method. The present disclosure relates in particular to the intelligent output of driver prompts, in particular attention warnings. Background Art

[0002] Driver monitoring systems (DMS) are increasingly being installed in modern vehicles. These systems monitor the driver via an in-car camera and output a warning if the driver does not look at the road for too long, for example. Such driver monitoring systems are intended to increase safety in road traffic.

[0003] According to the EuroNCAP protocol, driver distraction can be identified, for example, if the driver looks away from the windscreen for more than 3 seconds or has looked away from the windscreen for 10 seconds in the past 30 seconds. There are various ways to design appropriate acoustic and / or visual warning messages. For example, a visual display can be shown during the accumulation process so that the display is initially only vaguely visible and becomes visually clearer as the threshold for triggering the acoustic warning increases. The display can be hidden again after the driver has refocused his attention on the road and this is recognized by the driver monitoring system. Displays or warning messages are shown and hidden according to preset rules. Summary of the invention

[0004] The object of the present disclosure is to provide a driver monitoring system for a vehicle, a vehicle having such a driver monitoring system, a method for driver monitoring, and a storage medium for executing the method, which enable intelligent output of attention warnings. In particular, the object of the present disclosure is to execute the output of attention warnings in a manner adapted to the situation and specific to the driver.

[0005] This object is achieved by the subject matter of the independent claim. Advantageous embodiments are specified in the dependent claims.

[0006] According to an independent aspect of the present disclosure, a driver monitoring system for a vehicle, in particular a motor vehicle, is described. The driver monitoring system comprises at least one output device, which is configured to output at least one attention warning; a monitoring module, which is configured to monitor a driver of the vehicle and identify a state of inattention of the driver; and a control module, which is configured to

[0007] - accumulating durations of the identified inattention states within a finite time window having a predetermined length to determine an accumulated inattention duration for the finite time window; and

[0008] Based on the accumulated inattention duration and the rate measure of increase of the accumulated inattention duration, at least one output device for outputting at least one attention warning is actuated.

[0009] According to the present invention, the output of the attention warning is adapted based on a rate measure of increase of the accumulated inattention duration. In particular, the resetting of the accumulated inattention duration and thus, for example, the disappearance of the attention warning can be adapted to the increasing speed of the accumulated inattention duration, so that the disappearance of the attention warning does not occur similar to the accumulated inattention duration. If the accumulation occurs very quickly, for example due to multiple inattention states identified in a short period of time, it can be considered that the driver tends to be distracted. In this case, the resetting of the accumulated inattention duration can be adapted so that the accumulation is reset more slowly and thus the disappearance of the attention warning is delayed. Therefore, the attention warning can be output in a manner adapted to the situation and for the driver, so that the driver can be assisted better and more sustainably when guiding the vehicle. In addition, better warning report effectiveness can be achieved, thereby increasing traffic safety.

[0010] Preferably, the driver monitoring system, in particular the monitoring module, comprises at least one sensor, which is arranged to detect the driver and provide corresponding detection data. The at least one sensor may be an optical sensor, such as a camera, in particular an interior space camera. Optionally, the driver monitoring system, in particular the monitoring module, may comprise at least one light source. In some embodiments, the light source may be an infrared light source, and the at least one sensor may be an infrared sensor, such as an infrared camera.

[0011] The monitoring module may be configured to analyze and evaluate detection data, such as image data, provided by at least one sensor. For example, the driver's gaze direction may be detected or determined by analyzing and evaluating the detection data.

[0012] Typically, the monitoring module can be configured to detect the gaze direction by means of eye tracking. The term "eye tracking" is to be understood in particular as the detection of eye movements, for example, detected by a camera of an interior spatial sensor system. This allows for the acquisition of knowledge about where a person is looking.

[0013] The monitoring module can be configured to determine whether the driver's gaze direction deviates from a reference gaze direction. Preferably, the reference gaze direction is a gaze direction pointing to a road and / or a windshield located in front of the vehicle. If the driver's gaze direction deviates from the reference gaze direction, that is, if the driver moves his eyes away from the road and / or the windshield, the event can be identified as an inattention state. The gaze duration can indicate: how long the driver has not looked toward the reference gaze direction or how long the driver has moved his eyes away from the reference gaze direction. The gaze duration can be added for the identified inattention state to determine the accumulated inattention duration.

[0014] Preferably, the at least one attention warning comprises or is an attention prompt.

[0015] Preferably, the at least one attention warning comprises or is a visual and / or acoustic attention prompt.

[0016] At least one output device may include at least one display device for outputting at least one optical or visual attention prompt and / or at least one speaker for outputting at least one auditory attention prompt. The at least one display device may include a display, in particular an LCD display, a plasma display or an OLED display. In addition or alternatively, at least one display device may include a projection device, which is arranged to fade in information directly in the driver's field of view, in particular onto a windshield. In some embodiments, at least one display device may be a central information output device of an infotainment system, such as an audio head unit or a pillar-to-pillar display. Preferably, at least one output device is fixedly mounted in the vehicle.

[0017] Preferably, the control module is configured to control at least one output device to output or fade in at least one attention warning when the accumulated inattention duration reaches or exceeds a first threshold.

[0018] Preferably, the control module is configured to operate at least one output device to hide at least one attention warning when the accumulated inattention duration subsequently reaches or falls below a first threshold or a second threshold (different from the first threshold).

[0019] The first threshold may be in the range of 5 seconds to 15 seconds. In some embodiments, the first threshold may be 10 seconds.

[0020] The second threshold may be in the range of 5 seconds to 15 seconds. In some embodiments, the second threshold may be 10 seconds.

[0021] Preferably, the finite time window is a sliding time window. "Sliding" means that new values ​​for the accumulated inattention duration are continuously added, while old values ​​are deleted or removed from the accumulated inattention duration ("first in, first out").

[0022] Preferably, the end of the sliding time window corresponds to the current time. Thus, the start of the sliding time window may be a point in time corresponding to the current time minus a predetermined length of the time window.

[0023] A time window may also be referred to as a "time interval".

[0024] Preferably, the predetermined length of the limited time window is in the range of 10 seconds to 50 seconds or in the range of 20 seconds to 40 seconds. In some embodiments, the predetermined length of the limited time window may be 30 seconds.

[0025] Preferably, the rate brightness of the increase of the accumulated inattention duration is equivalent to the slope of the accumulated inattention duration over time. In other words, the continuity of the accumulated inattention duration can be tested. In particular, the slope of the accumulated inattention duration can be equivalent to the first-order derivative of the accumulated inattention duration over time. The larger the value of the first-order derivative, the faster or steeper the increase and the higher the driver's distraction. The smaller the value of the first-order derivative, the smaller or flatter the increase and the smaller the driver's distraction.

[0026] Preferably, the control module is configured to perform a reset of the accumulated inattention duration based on a rate measure of increase of the accumulated inattention duration. The term "reset" as used in the context of the present disclosure refers to removing old values ​​from the accumulated inattention duration. Removing old values ​​from the accumulated inattention duration may cause the accumulated inattention duration to decrease and, for example, fall below a warning threshold, thereby suppressing the warning message.

[0027] Preferably, the control module is arranged such that the larger the rate metric is, the slower the reset of the accumulated inattention duration is performed. In other words, the steeper the increase in the accumulated inattention duration is, the slower the reset occurs. Thus, if the driver is prone to inattention, the at least one attention warning may be displayed for a longer period of time.

[0028] Preferably, the control module is configured such that the smaller the rate metric is, the faster the reset of the accumulated inattention duration is performed. In other words, the flatter the slope of the accumulated inattention duration is, the faster the reset will occur. Thus, if the driver is not prone to inattention and reacts quickly to the attention warning, the at least one attention warning may be displayed for a shorter time.

[0029] Preferably, the control module is arranged to variably adjust the predetermined duration of the limited time window based on a measure of the rate of increase of the accumulated inattention duration, in particular (variably) adjust the speed of resetting the accumulated inattention duration.

[0030] Preferably, the control module is arranged to increase the predetermined duration of the limited time window for high speed measures and thereby slow down the resetting of the accumulated inattention duration. Thus, if the driver is prone to inattention, at least one attention warning may be displayed for a longer time.

[0031] Preferably, the control module is arranged to reduce the predetermined duration of the limited time window for a low speed metric and / or adjust it to a reference or standard value and thereby accelerate the resetting of the accumulated inattention duration. Thus, if the driver is not prone to inattention and reacts quickly to the attention warning, at least one attention warning may be displayed for a shorter time.

[0032] Preferably, the control module is arranged to increase the predetermined duration of the limited time window when the rate metric reaches or exceeds a threshold. For example, the predetermined duration of the limited time window may be adjusted to a reference or standard value and adapted to a higher value only when the threshold is reached or exceeded.

[0033] Preferably, the control module is arranged to increase and / or decrease the predetermined duration of the limited time window corresponding to the rate metric.For example, the predetermined duration of the limited time window may be increased and decreased continuously or stepwise (eg, proportional to the rate metric).

[0034] According to another independent aspect of the present invention, a vehicle, in particular a motor vehicle, is described. The vehicle comprises a driver monitoring system according to an embodiment of the present disclosure.

[0035] The term "vehicle" includes passenger cars, trucks, vans, buses, caravans, motorcycles, etc., used for transporting people, goods, etc. In particular, the term includes motor vehicles used for the transportation of people.

[0036] According to another independent aspect of the present disclosure, a method for driver monitoring is described. The method includes detecting a driver of a vehicle and identifying an inattention state of the driver; accumulating the duration of the identified inattention state within a limited time window having a predetermined length; outputting at least one attention warning when the accumulated duration of the inattention state within the limited time window reaches or exceeds a first threshold; resetting the accumulated duration of the inattention state within the limited time window based on a rate measure of increase of the accumulated duration of the inattention state; and hiding at least one attention warning based on resetting the accumulated duration of the inattention state within the limited time window.

[0037] The method may implement aspects of the driver monitoring system described in this document.

[0038] According to another independent aspect of the present disclosure, a software (SW) program is described. The software program can be arranged to run on one or more processors and thereby perform the method for driver monitoring described in this document.

[0039] According to another independent aspect of the present disclosure, a storage medium is described. The storage medium may include a SW program, which is configured to be executed on one or more processors and in order to thereby perform the method for driver monitoring described herein.

[0040] According to another independent aspect of the disclosure, software with a program code is described. The software is configured to perform a method for driver monitoring when the software is executed on one or more software-controlled devices.

[0041] According to another independent aspect of the present disclosure, a driver monitoring system is described. The driver monitoring system includes one or more processors; and at least one memory connected to the one or more processors and containing instructions that can be executed by the one or more processors to perform the method for driver monitoring described herein.

[0042] A processor or processor module is a programmable arithmetic unit, ie a machine or electronic circuit that controls other elements according to given instructions and drives an algorithm (process) therein. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Embodiments of the present disclosure are shown in the accompanying drawings and described in more detail below. The accompanying drawings show:

[0044] Figure 1 A vehicle with a driver monitoring system according to an embodiment of the present disclosure is schematically shown.

[0045] Figure 2schematically illustrates the accumulated inattention duration according to an embodiment of the present disclosure, and

[0046] Figure 3 A flow chart of a method for driver monitoring according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0047] In the following, unless otherwise stated, the same reference numerals are used for elements that are identical and have the same effect.

[0048] Figure 1 A vehicle 10 is schematically shown having a driver monitoring system 100 according to an embodiment of the present disclosure. Figure 2 FIG. 4 schematically illustrates the accumulated inattention duration AUD according to an embodiment of the present disclosure.

[0049] The driver monitoring system 100 includes at least one output device 110 configured to output at least one attention warning, such as an attention warning prompt; a monitoring module 120 configured to monitor a driver of the vehicle 10 and detect a state of inattention of the driver; and a control module 130 configured to accumulate a duration of an identified inattention state UAZ within a limited time window having a predetermined length to determine an accumulated inattention duration AUD for the limited time window; and to manipulate the at least one output device 110 for outputting at least one attention warning based on the accumulated inattention duration AUD and a rate metric of increase of the accumulated inattention duration AUD.

[0050] At least one attention warning may include a visual and / or audible attention warning prompt. At least one output device 110 may include at least one display device for outputting at least one optical or visual attention prompt and / or at least one speaker for outputting at least one acoustic attention prompt. The at least one display device may include a display, in particular an LCD display, a plasma display or an OLED display. In addition or alternatively, at least one display device may include a projection device, which is arranged to fade in information directly in the driver's field of view, in particular onto the windshield. In some embodiments, at least one display device may be a central information output device of an infotainment system, such as a head unit or a column-to-column display. Preferably, at least one output device is fixedly mounted in the vehicle.

[0051] The monitoring module 120 may include at least one sensor configured to detect the driver and provide corresponding detection data. The at least one sensor may be an optical sensor, such as a camera, in particular an internal camera. The monitoring module 120 may also be configured to analyze and evaluate the detection data provided by the at least one sensor, such as image data. By analyzing and evaluating the detection data, for example, the driver's gaze direction can be detected or determined. Typically, the monitoring module 120 may be configured to sense the gaze direction by means of eye tracking.

[0052] Monitoring module 120 can be configured to determine whether the driver's gaze direction deviates from a reference gaze direction. The reference gaze direction is preferably a gaze direction pointing to the road and / or windshield in front of vehicle 10. If the driver's line of sight deviates from the reference line of sight, i.e., if the driver's line of sight leaves the road and / or windshield, the event can be identified as an inattention state UAZ. The gaze duration can indicate how long the driver has not looked at the reference gaze direction or how long the driver has deviated from the reference gaze direction. The gaze duration can be added to the identified inattention state UAZ to determine the accumulated inattention duration AUD.

[0053] The control module 130 may be configured to control at least one output device 110 for outputting or displaying at least one attention warning when the accumulated inattention duration AUD reaches or exceeds a first threshold. Figure 2 In the example of , the first threshold is 10 seconds. The control module 130 can also be configured to manipulate at least one output device 110 to hide at least one attention warning when the accumulated inattention duration AUD reaches or falls below the first threshold or a second threshold (different from the first threshold).

[0054] The finite time window may be a sliding time window. "Sliding" means that new values ​​are continuously added to the accumulated inattention duration AUD, while old values ​​are removed or deleted from the accumulated inattention duration ("first in, first out"). The end of the sliding time window may be equivalent to the current time. Therefore, the beginning of the sliding time window may be a time point equivalent to the current time minus a predetermined length of the time window.

[0055] The predetermined length of the limited time window may be in the range between 10 seconds and 50 seconds or in the range between 20 seconds and 40 seconds. In some embodiments, the predetermined length of the limited time window may be 30 seconds.

[0056] The rate measurement of the increase of the accumulated inattention duration AUD can correspond to the slope of the accumulated inattention duration AUD over time. In other words, the continuity of the accumulated inattention duration AUD can be tested. In particular, the slope of the accumulated inattention duration AUD can correspond to the first-order derivative of the accumulated inattention duration AUD over time. The larger the value of the first-order derivative, the faster or steeper the increase, and the higher the driver's distraction. The smaller the value of the first-order derivative, the smaller or flatter the increase, and the smaller the driver's distraction.

[0057] The control module 130 can be configured to perform the reset of the accumulated inattention duration AUD based on the rate metric of the increase of the accumulated inattention duration AUD. For example, the larger the rate metric, the slower the reset of the accumulated inattention duration AUD can occur. Therefore, if the driver tends to be inattentive, at least one attention warning can be displayed longer. Similarly, the smaller the rate metric, the faster the reset of the accumulated inattention duration AUD can occur. Therefore, if the driver is not inclined to be inattentive and reacts quickly to the attention warning, at least one attention warning can be displayed shorter.

[0058] Preferably, the control module 130 is configured to variably adjust the predetermined duration of the limited time window based on the rate metric of the increase of the accumulated inattention duration AUD, and in particular (variably) adjust the speed of resetting the accumulated inattention duration AUD. In particular, the predetermined duration of the limited time window can be increased for high rate metrics, and the reset of the accumulated inattention duration AUD can therefore be slowed down. Therefore, if the driver tends to be inattentive, at least one attention warning can be displayed for a longer time. Similarly, the predetermined duration of the limited time window can be reduced for low rate metrics, and the reset of the accumulated inattention duration AUD can therefore be accelerated.

[0059] In some embodiments, the control module 130 may be configured to increase the predetermined duration of the limited time window when the rate metric reaches or exceeds a threshold. For example, the predetermined duration of the limited time window may be set to a reference value or a standard value and adapted to a higher value only when the threshold is reached or exceeded. Alternatively, the control module 130 may be configured to increase and / or decrease the predetermined duration of the limited time window continuously and in proportion to the rate metric.

[0060] Figure 3The flowchart of the method 300 for driver monitoring according to the embodiment of the present disclosure is schematically shown. The method 300 may be implemented by corresponding software executable by one or more processors (eg, CPU).

[0061] Method 300 includes, in box 310, detecting a driver of a vehicle and identifying an inattention state of the driver; in box 320, accumulating the duration of the identified inattention state within a finite time window having a predetermined length; in box 330, outputting at least one attention warning if the accumulated duration of the inattention state within the finite time window reaches or exceeds a first threshold; in box 340, resetting the accumulated duration of the inattention state within the finite time window based on a rate metric of an increase of the accumulated inattention duration; and in box 350, fading out at least one attention warning based on resetting the accumulated duration of the inattention state within the finite time window.

[0062] According to the present invention, the output of the attention warning is adapted based on a rate measure of increase of the accumulated inattention duration. In particular, the resetting of the accumulated inattention duration and thus, for example, the disappearance of the attention warning can be adapted to the speed of increase of the accumulated inattention duration, so that the disappearance of the attention warning does not occur similarly to the accumulated inattention duration. If the accumulation occurs very quickly, for example due to the recognition of multiple inattention states in a short period of time, it can be considered that the driver tends to be distracted. In this case, the resetting of the accumulated inattention duration can be adapted so that the accumulation is reset more slowly, thereby delaying the disappearance of the attention warning. Therefore, the attention warning can be output adaptively according to the situation and for the driver, so that the driver can be assisted better and more sustainably when guiding the vehicle. In addition, better warning report effectiveness can be achieved, thereby also improving traffic safety.

[0063] Although the present invention has been shown and explained in detail by preferred embodiments, the present invention is not limited to the disclosed examples, and those skilled in the art can derive other variations therefrom without departing from the scope of the present invention. It is therefore apparent that there are a large number of possible variations. It is also clear that the exemplary embodiments mentioned are in fact only examples and should not be interpreted as limiting the scope, possible applications or configurations of the present invention in any way. On the contrary, the foregoing description and the accompanying drawings enable those skilled in the art to specifically implement the exemplary embodiments, wherein those skilled in the art know that the disclosed inventive concept can make various changes, such as the functions or arrangements of the various elements mentioned in the exemplary embodiments without departing from the scope of protection defined by the claims and their legal equivalents, such as further explanations in the specification.

Claims

1. A driver monitoring system (100) for a vehicle (10), comprising: at least one output device (110) configured to output at least one attention warning; a monitoring module (120) configured to monitor a driver of the vehicle (10) and to detect a state of inattention (UAZ) of the driver; and A control module (130), the control module being configured to: accumulating durations of the identified inattention states (UAZ) within a finite time window having a predetermined length to determine an accumulated inattention duration (AUD) for the finite time window; and Based on the accumulated inattention duration (AUD) and a rate metric of increase of the accumulated inattention duration (AUD), the at least one output device (110) for outputting the at least one attention warning is controlled.

2. The driver monitoring system (100) according to claim 1, wherein the control module (130) is configured to: When the accumulated inattention duration (AUD) reaches or exceeds a first threshold, controlling the at least one output device (110) to output the at least one attention warning; and When the accumulated inattention duration (AUD) reaches or falls below the first threshold or the second threshold, the at least one output device (110) is controlled to hide the at least one attention warning.

3. The driver monitoring system (100) according to any one of the preceding claims, wherein the finite time window is a sliding time window.

4. The driver monitoring system (100) according to claim 3, wherein the end of the sliding time window corresponds to the current time and / or the predetermined length of the limited time window is in the range between 10 seconds and 50 seconds.

5. A driver monitoring system (100) according to any one of the preceding claims, wherein the rate measure of the increase of the accumulated inattention duration (AUD) is equivalent to the slope of the accumulated inattention duration (AUD) over time, wherein the slope of the accumulated inattention duration (AUD) is equivalent to the first derivative of the accumulated inattention duration (AUD) over time.

6. A driver monitoring system (100) according to any one of the preceding claims, wherein the control module (130) is arranged to perform a reset of the accumulated inattention duration (AUD) based on the rate metric of the increase of the accumulated inattention duration (AUD).

7. The driver monitoring system (100) of claim 6, wherein the control module (130) is configured such that the greater the rate metric, the slower the resetting of the accumulated inattention duration (AUD) is performed.

8. A driver monitoring system (100) according to any of the preceding claims, wherein the control module (130) is configured to variably adjust the predetermined duration of the limited time window based on the rate metric of the increase of the accumulated inattention duration (AUD) so as to adjust the speed of the resetting of the accumulated inattention duration (AUD).

9. The driver monitoring system (100) according to claim 8, wherein the control module (130) is configured to: When the rate metric reaches or exceeds a threshold, increasing the predetermined duration of the limited time window; and / or The predetermined duration of the limited time window is increased and / or decreased corresponding to the rate metric.

10. A vehicle (10) comprising a driver monitoring system (100) according to any one of claims 1 to 9.

11. A method (300) for driver monitoring, comprising: Detecting (310) a driver of a vehicle (10) and identifying an inattention state (UAZ) of the driver; accumulating (320) the duration of the identified inattention state (UAZ) within a finite time window having a predetermined length; When the accumulated duration of the inattention state (AUD) within the limited time window reaches or exceeds a first threshold, outputting (330) at least one attention warning; resetting (340) the accumulated duration (AUD) of the inattention state (UAZ) within the limited time window based on a rate metric of increase of the accumulated inattention duration (AUD); and The at least one attention warning is suppressed (350) based on resetting the duration (AUD) of the accumulated inattention state (UAZ) within the limited time window. 12 . A storage medium comprising a software program, which is configured to be executed on one or more processors and thereby to perform the method ( 300 ) for driver monitoring according to claim 11 .

Images