Driver monitoring system for vehicle and method for monitoring driver

Abstract

To provide a driver monitoring system for a vehicle that enables intelligent output of an attention warning, a vehicle having such a driver monitoring system, a method for monitoring a driver, and a storage medium for executing the method.SOLUTION: A driver monitoring system 100 for a vehicle 10 includes: an output device 110 configured to output at least one attention warning; a monitoring module 120 configured to monitor a driver of the vehicle 10 and recognize a careless state of the driver; and a control module 130 which accumulates a duration time of a recognized careless state to specify an inattention duration time accumulated for the finite time frame within a finite time frame having a predetermined length, and which is configured to control the output device 110 to output the attention warning based on an accumulated inattention duration time and an increase pace of the accumulated inattention duration time.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a driver monitoring system for a vehicle, a vehicle having such a driver monitoring system, a method for monitoring a driver, and a storage medium for carrying out said method. The present disclosure relates in particular to the intelligent output of driver instructions, in particular attentiveness warnings. [Background technology]

[0002] Modern vehicles are increasingly equipped with "Driver Monitoring Systems" (DMS), which monitor the driver via an interior camera and, for example, issue a warning if the driver does not keep his or her eyes on the road for a significant period of time. Such driver monitoring systems increase safety in road traffic.

[0003] According to Euro NCAP protocols, driver inattention can be recognized, for example, if the driver looks away from the windshield for more than 3 seconds or if the driver looks away from the windshield for a cumulative 10 seconds within the last 30 seconds. There are various approaches to configure suitable acoustic and / or visual warning notifications. For example, it is possible to fade in the visual indication in the course of accumulation, so that the indication is only slightly perceptible at first and becomes increasingly clear visually as the threshold for the acoustic warning activation is approached. After the driver directs his / her attention back to the road and this is recognized by the driver monitoring system, the indication can be faded out again. In this case, the fading in and out of the indication or warning notification is performed according to predefined rules. Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure aims to provide a driver monitoring system for a vehicle, which enables intelligent output of an attention warning, a vehicle having such a driver monitoring system, a method for monitoring a driver, and a storage medium for executing said method. In particular, the present disclosure aims to provide an attention warning that is individual to the driver and adapted to the situation. [Means for solving the problem]

[0005] The problem is solved by the subject matter of the independent claims. Preferred embodiments are set forth in the dependent claims.

[0006] According to one independent aspect of the present disclosure, there is provided a driver monitoring system for a vehicle, particularly a motor vehicle, comprising at least one output device configured to output at least one attentiveness warning, a monitoring module configured to monitor a driver of the vehicle and recognize an inattentive state of the driver, and - accumulating durations of recognized inattentive states within a finite time window having a predetermined length to identify an accumulated inattentive duration for the finite time window; - controlling (activating) an output device to output at least one attention warning based on the accumulated inattention duration and the pace of increase of the accumulated inattention duration. and a control module configured to:

[0007] According to the present invention, the output of the attention warning is adapted based on the rate of increase of the accumulated inattention duration. In particular, the reset of the accumulated inattention duration and thus the fade-out of the attention warning, for example, can be adapted to the rate of increase of the accumulated inattention duration, so that the fade-out of the attention warning is not performed at the same time as the accumulated inattention duration. If the accumulation is performed very quickly, for example based on a large number of inattention states recognized in a short time, it is considered that the driver is prone to distraction (prone to distraction). In this case, the reset of the accumulated inattention duration can be adapted so that the accumulation is reset more slowly and thus the fade-out of the attention warning is extended. As a result, it is possible to perform a situation-adapted and driver-specific attention warning output, so that the driver can be better and sustainably supported when driving the vehicle. In addition, a better effectiveness of the warning notification can be achieved, which can also improve road safety.

[0008] Preferably, the driver monitoring system, in particular the monitoring module, includes at least one sensor configured to detect the driver and provide corresponding detection data. The at least one sensor may be an optical sensor, such as, for example, a camera, in particular an interior camera. Optionally, the vehicle monitoring system, in particular the monitoring module, may include at least one light source. In some embodiments, the light source may be an infrared light source and the at least one sensor may be an infrared sensor, such as, for example, an infrared camera.

[0009] The monitoring module can be configured to evaluate detection data provided by the at least one sensor, such as image data, such that, for example, a driver's gaze direction can be detected or determined by evaluating the detection data.

[0010] In general, the monitoring module can be configured to detect the gaze direction using eye tracking, where the term "eye tracking" is understood in particular to mean the detection of eye movements detected, for example by a camera of a room sensor type, so that knowledge can be obtained as to where a person is looking.

[0011] The monitoring module can be configured to determine whether the driver's gaze direction is different from a reference gaze direction. Preferably, the reference gaze direction is a gaze direction to the roadway and / or windshield in front of the vehicle. If the driver's gaze direction is different from the reference gaze direction, i.e., the driver is looking away from the road and / or windshield, the event can be recognized as an inattentive state. The gaze duration can then indicate how long the driver is not looking at the reference gaze direction or how long the driver is looking away from the reference gaze direction. The gaze duration can be summed for the recognized inattentive state to determine an accumulated inattentive duration.

[0012] Preferably, at least one assessment module includes or is a warning indicator.

[0013] Preferably, the at least one assessment module includes or is a visual and / or an audible warning indication.

[0014] The at least one output device may include at least one display device for outputting at least one visual (optical) or visual warning indication and / or at least one speaker for outputting at least one audible (acoustic) warning indication. The at least one display device may include a display, in particular an LCD display, a plasma display or an OLED display. Additionally or alternatively, the at least one display device may include a projection device, in particular configured for projecting onto the windscreen, to fade in information directly in the driver's field of vision. In some embodiments, the at least one display device may include a central information output device of an infotainment system, for example a head unit or a pillar-to-pillar display. Preferably, the at least one output device is fixed to the vehicle.

[0015] Preferably, the control module is configured to control at least the output device to output or fade in at least one attention warning when the accumulated inattention duration reaches or exceeds a first threshold.

[0016] Preferably, the control module is configured to control at least the output device to fade out the at least one attention warning when the accumulated inattention duration subsequently reaches or falls below the first threshold or a second threshold (different from the first threshold).

[0017] The first threshold may be in the range of 5 to 15 seconds. In some embodiments, the first threshold may be 10 seconds.

[0018] The second threshold may be in the range of 5 to 15 seconds. In some embodiments, the second threshold may be 10 seconds.

[0019] Preferably, the finite time window is a sliding time window, where "sliding" means that new values ​​are continually added to the accumulated inattention duration while old values ​​are removed or erased from the accumulated inattention duration ("first in, first out").

[0020] Preferably, the end of the sliding time window corresponds to the current time, and therefore the beginning of the sliding time window may be a point in time corresponding to the current time minus a predetermined length of the time window.

[0021] A time period may also be called a "time interval."

[0022] Preferably, the predetermined length of the finite time window is in the range of 10 to 50 seconds, or in the range of 20 to 40 seconds, In some embodiments, the predetermined length of the finite time window may be 30 seconds.

[0023] Preferably, the pace of the rise of the accumulated inattention duration corresponds to the gradient of the accumulated inattention duration over time. In other words, it is possible to check the continuity of the accumulated inattention duration. In particular, the gradient of the accumulated inattention duration can correspond to the first derivative of the accumulated inattention duration over time. The larger the value of the first derivative, the more rapid or steep the rise, and the more distracted the driver is. The smaller the value of the first derivative, the more gentle or flatter the rise, and the less distracted the driver is.

[0024] Preferably, the control module is configured to perform a reset of the accumulated inattention duration based on the rate of increase of the accumulated inattention duration, where the term "reset" as used in this disclosure refers to subtracting an older value from the accumulated inattention duration, which may lead to the accumulated inattention duration being reduced, with a reasonable level of driver attention, to fall below a warning threshold, for example, which causes the warning indication to fade out.

[0025] Preferably, the control module is configured to reset the accumulated inattention duration more slowly the faster the pace is, in other words, the steeper the rise in the accumulated inattention duration is, the slower the reset can be, so that at least one attention warning can be displayed for a longer period if the driver is inattentive.

[0026] Preferably, the control module is configured to perform the reset of the accumulated inattention duration more quickly the slower the pace, in other words, the flatter the increase in the accumulated inattention duration, the more quickly the reset can be performed, so that at least one attention warning can be displayed for a shorter period of time if the driver is not prone to inattention and reacts quickly to the attention warning.

[0027] Preferably, the control module is configured to variably adjust the predetermined duration of the finite time frame based on the pace of increase in the accumulated inattention duration, in particular to (variably) adjust the speed of resetting the accumulated inattention duration.

[0028] Preferably, the control module is configured to extend the predetermined duration of the finite time window for a fast pace, and thus to reset the accumulated inattention duration more slowly, so that the at least one attention warning can be displayed for a longer period of time if the driver is more inattentive.

[0029] Preferably, the control module is configured to shorten the predetermined duration of the finite time window for a slower pace and / or adjust (set) it to a baseline or standard value, thus hastening the reset of the accumulated inattention duration, so that at least one attention warning can be displayed for a shorter period if the driver is less prone to inattention and reacts quickly to the attention warning.

[0030] Preferably, the control module is configured to extend the predetermined duration of the finite time window when the pace reaches or exceeds a threshold value, for example, the predetermined duration of the finite time window can be adjusted to a baseline or standard value and adapted to a higher value only once the threshold value is met.

[0031] Preferably, the control module is configured to lengthen and / or shorten the predetermined duration of the finite time window in accordance with the pace, for example, the predetermined duration of the finite time window can be lengthened and shortened continuously or in stages (e.g., proportional to the pace).

[0032] According to another independent aspect of the present disclosure, there is provided a vehicle, in particular a motor vehicle, including a driver monitoring system according to an embodiment of the present disclosure.

[0033] The term vehicle includes cars, lorries, vans, buses, campers, motorbikes, etc., used for the transport of people, goods, etc. In particular, the term includes motor vehicles for transporting people.

[0034] According to another independent aspect of the present disclosure, a method for monitoring a driver is provided, the method includes: detecting a driver of a vehicle and recognizing an inattentive state of the driver, accumulating a duration of the recognized inattentive state within a finite time frame having a predetermined length, outputting at least one attention warning when the accumulated duration of the inattentive state within the finite time frame reaches or exceeds a first threshold, resetting the accumulated duration of the inattentive state within the finite time frame based on a pace of increase of the accumulated inattentive duration, and fading out the at least one attention warning based on the resetting of the accumulated duration of the inattentive state within the finite time frame.

[0035] The method may implement aspects of the driver monitoring system described herein.

[0036] According to another independent aspect of the present disclosure, there is provided a software (SW) program that can be configured to run on one or more processors and thereby perform the driver monitoring method described herein.

[0037] According to another independent aspect of the present disclosure, a storage medium is provided, which may include a software (SW) program configured to be executed on one or more processors and thereby perform the driver monitoring method described herein.

[0038] According to another independent aspect of the present disclosure, there is provided software having program code configured to perform a method of driver monitoring when the software is executed in one or more software controlled devices.

[0039] According to another independent aspect of the present disclosure, there is provided a driver monitoring system including one or more processors and at least one memory coupled to the one or more processors for receiving instructions executable by the one or more processors to perform the driver monitoring methods described herein.

[0040] A processor or processor module is a programmable computing unit, ie a machine or electronic circuit that controls other elements according to commands transmitted to it and in so doing executes an algorithm (process).

[0041] The disclosed embodiments are illustrated in the drawings and are described in detail below. [Brief description of the drawings]

[0042] [Figure 1] FIG. 1 is a schematic diagram of a vehicle having a driver monitoring system according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a schematic diagram illustrating accumulated inattention duration according to an embodiment of the present disclosure. [Diagram 3] FIG. 2 illustrates a flowchart of a method for monitoring a driver according to an embodiment of the disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0043] In the following, unless otherwise stated, the same reference numerals are used for the same and similarly acting elements.

[0044] 1, a vehicle 10 having a driver monitoring system 100 according to an embodiment of the present disclosure is shown generally at 1. FIG. 2, a cumulative inattention duration AUD according to an embodiment of the present disclosure is shown generally at 2.

[0045] The driver monitoring system 100 includes at least one output device 110 configured to output at least one attention warning, e.g., an attention indication, a monitoring module 120 configured to monitor the driver of the vehicle 10 and recognize an inattention state of the driver, and a control module 130 configured to accumulate the duration of the recognized inattention state UAZ within a finite time window having a predetermined length to identify an accumulated inattention duration AUD for the finite time window, and to control (activate) the output device 110 to output at least one attention warning based on the accumulated inattention duration AUD and the pace of increase of the accumulated inattention duration AUD.

[0046] The at least one attention warning may include a visual and / or audible warning indication. The at least one output device 110 may include at least one display device for outputting at least one visual (optical) or visual warning indication and / or at least one speaker for outputting at least one audible (acoustic) warning indication. The at least one display device may include a display, in particular an LCD display, a plasma display or an OLED display. Additionally or alternatively, the at least one display device may include a projection device, in particular configured for projecting on the windshield, to fade in information directly in the driver's field of vision. In some embodiments, the at least one display device may include a central information output device of an infotainment system, for example a head unit or a pillar-to-pillar display. Preferably, the at least one output device is fixedly mounted in the vehicle.

[0047] The monitoring module 120 may include at least one sensor configured to detect a driver and provide corresponding detection data. The at least one sensor may be an optical sensor, such as a camera, in particular an interior camera. Furthermore, the monitoring module 120 may be configured to evaluate detection data provided by the at least one sensor, such as image data. By evaluating the detection data, for example, a gaze direction of the driver may be detected or determined. In general, the monitoring module 120 may be configured to detect the gaze direction using eye tracking.

[0048] The monitoring module 120 can be configured to determine whether the driver's gaze direction is different from the reference gaze direction. Preferably, the reference gaze direction is a gaze direction to the roadway and / or windshield in front of the vehicle 10. If the driver's gaze direction is different from the reference gaze direction, i.e., the driver is looking away from the road and / or windshield, the event can be recognized as an inattentive state UAZ. The gaze duration can then indicate how long the driver is not looking at the reference gaze direction or how long the driver is looking away from the reference gaze direction. The gaze duration can be summed for the recognized inattentive state UAZ to determine an accumulated inattentive duration AUD.

[0049] The control module 130 can be configured to control at least the output device 110 to output or fade in at least one attention warning when the accumulated inattention duration AUD reaches or exceeds a first threshold. In the example of Fig. 2, the first threshold is 10 seconds. Furthermore, the control module 130 can be configured to control at least the output device 110 to fade out at least one attention warning when the accumulated inattention duration AUD subsequently reaches or falls below the first threshold or a second threshold (different from the first threshold).

[0050] The finite time window may be a sliding time window. By "sliding" we mean that new values ​​are continually added to the accumulated inattention duration AUD while old values ​​are removed or erased from the accumulated inattention duration ("first in, first out"). The end of the sliding time window may correspond to the current time. Thus, the beginning of the sliding time window may be a point in time that corresponds to the current time minus a predetermined length of the time window.

[0051] The predetermined length of the finite time window may be in the range of 10 to 50 seconds, or in the range of 20 to 40 seconds, In some embodiments, the predetermined length of the finite time window may be 30 seconds.

[0052] The pace of the rise of the accumulated inattention duration AUD may correspond to the slope of the accumulated inattention duration AUD over time. In other words, it is possible to check the continuity of the accumulated inattention duration AUD. In particular, the slope of the accumulated inattention duration AUD may correspond to the first derivative of the accumulated inattention duration AUD over time. The larger the value of the first derivative, the more rapid or steep the rise, and the more distracted the driver is. The smaller the value of the first derivative, the more gentle or flatter the rise, and the less distracted the driver is.

[0053] The control module 130 may be configured to perform a reset of the accumulated inattention duration AUD based on the rate of increase of the accumulated inattention duration AUD. For example, the greater the rate, the slower the reset of the accumulated inattention duration AUD may be performed. This allows the driver to display at least one attention warning for a longer period if the driver is inattentive. Similarly, the slower the rate, the quicker the reset of the accumulated inattention duration AUD may be performed. This allows the driver to display at least one attention warning for a shorter period if the driver is not inattentive and responds quickly to the attention warning.

[0054] Preferably, the control module 130 is configured to variably adjust the predetermined duration of the finite time frame based on the pace of the rise of the accumulated inattention duration AUD, in particular to (variably) adjust the speed of resetting the accumulated inattention duration AUD. In particular, the predetermined duration of the finite time frame can be extended for a fast pace, thereby slowing down the resetting of the accumulated inattention duration AUD. This allows at least one attention warning to be displayed for a longer period if the driver is inattentive. Similarly, the predetermined duration of the finite time frame can be shortened for a slow pace, thereby hastening the resetting of the accumulated inattention duration AUD.

[0055] In some embodiments, the control module 130 can be configured to extend the predetermined duration of the finite time window when the pace reaches or exceeds a threshold. For example, the predetermined duration of the finite time window can be adjusted to a baseline or standard value and adapted to a higher value only once the threshold is met. Alternatively, the control module 130 can be configured to extend and / or shorten the predetermined duration of the finite time window continuously and in proportion to the pace.

[0056] A flow chart of a method 300 for monitoring a driver according to an embodiment of the present disclosure is generally shown in Figure 3. The method 300 may be implemented by corresponding software executable by one or more processors (e.g., CPUs).

[0057] The method 300 includes, in block 310, detecting a driver of the vehicle and recognizing an inattentive state of the driver, in block 320, accumulating a duration of the recognized inattentive state within a finite time frame having a predetermined length, in block 330, outputting at least one attention warning when the accumulated duration of the inattentive state within the finite time frame reaches or exceeds a first threshold, in step 340, resetting the accumulated duration of the inattentive state within the finite time frame based on a pace of increase of the accumulated duration of the inattentive state within the finite time frame, in block 350, fading out the at least one attention warning based on the resetting of the accumulated duration of the inattentive state within the finite time frame.

[0058] According to the present invention, the output of the attention warning is adapted based on the rate of increase of the accumulated inattention duration. In particular, the reset of the accumulated inattention duration and thus the fade-out of the attention warning, for example, can be adapted to the rate of increase of the accumulated inattention duration, so that the fade-out of the attention warning is not performed at the same time as the accumulated inattention duration. If the accumulation is performed very quickly, for example based on a large number of inattention states recognized in a short time, it is considered that the driver is prone to distraction (prone to distraction). In this case, the reset of the accumulated inattention duration can be adapted so that the accumulation is reset more slowly and thus the fade-out of the attention warning is extended. As a result, it is possible to perform a situation-adapted and driver-specific attention warning output, so that the driver can be better and sustainably supported when driving the vehicle. In addition, a better effectiveness of the warning notification can be achieved, which can also improve road safety.

[0059] Although the present invention has been illustrated and described in detail by preferred embodiments, the present invention is not limited to the disclosed examples, and other modifications can be derived by those skilled in the art without departing from the scope of protection of the present invention. Therefore, it is clear that there are many possible modifications. Similarly, it is clear that the exemplary embodiments are merely examples, and the examples should not be understood as limiting the scope of protection, application or configuration of the present invention in any manner, for example. Rather, the above description and the description of the drawings (description by the drawings) enable those skilled in the art to specifically implement the exemplary embodiments, and those skilled in the art can make various modifications, for example, regarding the function or arrangement of individual elements listed in the exemplary embodiments, without departing from the scope of protection defined in the claims and their legal counterparts, such as further descriptions in the specification, after knowing the concept of the present invention disclosed.

Claims

1. A driver monitoring system (100) for a vehicle (10), comprising: at least one output device (110) configured to output at least one attentiveness warning; a monitoring module (120) configured to monitor a driver of the vehicle (10) and recognize an inattentive state (UAZ) of the driver; a control module (130) configured to accumulate durations of recognized inattention states (UAZ) within a finite time frame having a predetermined length to identify an accumulated inattention duration (AUD) for the finite time frame, and to control the at least one output device (110) to output at least one attention alert based on the accumulated inattention duration (AUD) and a pace of increase of the accumulated inattention duration (AUD); A driver monitoring system (100) comprising:

2. The control module (130): to control said at least output device (110) to output at least one attention alert when an accumulated inattention duration (AUD) reaches or exceeds a first threshold; and and controlling the at least one output device (110) to fade out at least one attention warning when the accumulated inattention duration (AUD) subsequently reaches or falls below a first threshold or a second threshold.

2. The driver monitoring system (100) of claim 1, configured as follows:

3. The driver monitoring system (100) of claim 1 or 2, wherein the finite time window is a sliding time window.

4. 4. The driver monitoring system (100) of claim 3, wherein an end of the sliding time window corresponds to a current time and / or the predetermined length of the finite time window is in the range of 10 to 50 seconds.

5. The driver monitoring system (100) of any one of claims 1 to 4, characterized in that the pace of increase of the accumulated inattention duration (AUD) corresponds to a slope of the accumulated inattention duration (AUD) over time, and the slope of the accumulated inattention duration (AUD) corresponds to a first derivative with respect to time of the accumulated inattention duration (AUD).

6. The driver monitoring system (100) of any one of claims 1 to 5, wherein the control module (130) is configured to perform a reset of the accumulated inattention duration (AUD) based on a pace of increase of the accumulated inattention duration (AUD).

7. 7. The driver monitoring system (100) of claim 6, wherein the control module (130) is configured to perform a reset of accumulated inattention duration (AUD) more slowly the faster the pace is.

8. The driver monitoring system (100) of any one of claims 1 to 7, wherein the control module (130) is configured to variably adjust the predetermined duration of the finite time window based on a pace of increase in the accumulated inattention duration (AUD) to adjust a rate of resetting the accumulated inattention duration (AUD).

9. 9. The driver monitoring system (100) of claim 8, wherein the control module (130) is configured to extend the predetermined duration of the finite time window when the pace reaches or exceeds a threshold, and / or to extend and / or shorten the predetermined duration of the finite time window in accordance with pace.

10. A vehicle (10) including a driver monitoring system (100) according to any one of the preceding claims.

11. A method (300) for monitoring a driver, comprising: detecting (310) a driver of a vehicle (10) and recognizing an inattentive state (UAZ) of the driver; accumulating (320) the duration of a recognized inattention state (UAZ) within a finite time window having a predetermined length; outputting (330) at least one attention alert when an accumulated duration of inattention (AUD) within the finite time frame reaches or exceeds a first threshold; resetting (340) the accumulated duration of inattention (UAZ) within the finite time frame based on the pace of increase of the accumulated duration of inattention (AUD); fading out (350) at least one attention warning based on a reset of an accumulated duration (AUD) of an inattention state (UZA) within the finite time frame; The method (300), comprising:

12. A storage medium containing a software program configured to be executed on one or more processors and thereby to perform the method (300) of driver monitoring according to claim 11.

Images