Method for determining a measure representing the attention of a driver and device for monitoring the attention of a driver

Abstract

Method for determining a measure (A) representing the attention of a driver, comprising the steps - Detecting the driver's direction of gaze, - Determining an actual frequency distribution (H Ist ) the direction of view, - Determining the measure as a function of a deviation (A) of the actual frequency distribution (H) Ist ) from a target frequency distribution (H Soll ) the direction of view, - Recording the driver's gaze direction for a specified period of time, - Determining the actual frequency distribution (H Ist ) the direction of view from the direction of view recorded for the predetermined duration - Determining the current driving situation, - Specifying the time duration depending on the current driving situation.

Description

[0001] The present invention relates to a method for determining a metric representing a driver's level of attention. The invention further relates to a device for monitoring a driver's level of attention in a vehicle, and to a vehicle equipped with such a device.

[0002] DE 197 34 307 A1 discloses a method for monitoring the vehicle behavior of a motor vehicle, wherein the driver's current direction of gaze is determined. This direction of gaze is compared with the vehicle's current direction of travel. Depending on the comparison result, a warning signal can be generated.

[0003] DE 10 2013 002 686 A1 relates to a method for operating a motor vehicle. In this method, images of a driver are captured using a recording device. An attention signal is generated based on the captured images. It is proposed that the driver's gaze direction also be preferably captured and evaluated to generate the attention signal.

[0004] DE 100 24 227 A1 describes a device for electronically monitoring a driver's level of attention. The device uses a camera to capture the driver's gaze direction, followed by a pattern matching evaluation unit. The aim is to recognize patterns or pattern combinations that represent acute drowsiness or fatigue. Fuzzy logic methods are to be used for this purpose.

[0005] DE 10 2008 015 986 A1 proposes to generate a gaze distribution pattern from the behavior of a driver's eyes, recorded by a camera during a predetermined period of time, and to estimate the driver's condition based on the distribution pattern.

[0006] DE 10 2013 213 236 A1 relates to determining the level of attention of a driver while driving his vehicle, wherein a sensor of the vehicle detects the direction of the driver's gaze towards an area of ​​the environment visible to him, and a probability distribution is assigned to the area visible to the driver, by means of which the level of attention of the driver can be determined on the basis of the detected direction of the driver's gaze.

[0007] DE 10 2011 121 260 A1 teaches a method for assisting a driver of a motor vehicle, whereby a deviation of actual driver behavior with target driver behavior is determined. If a predetermined deviation is exceeded, an error counter is incremented.

[0008] DE 10 2014 201 036 A1 discloses a method for determining the state of a vehicle driver. A control unit for a vehicle is described. The control unit is configured to receive image data from one or more image sensors. The one or more image sensors are configured to detect the driver's position within the vehicle. The control unit is further configured, based on the image data, to assign the driver's gaze direction to a specific viewing sector from a plurality of viewing sectors for a sequence of time points, thereby determining a temporal progression of viewing sectors as a representation of the driver's gaze direction over time.

[0009] Based on the state of the art, the task is to specify an improved method and an improved device for determining the attention of a vehicle driver.

[0010] The problem is solved by a method, a device, and a vehicle with the features of the independent claims. Advantageous embodiments of the invention are the subject of the dependent claims.

[0011] The method according to the invention serves to determine a numerical value representing a driver's level of attention. Such a numerical value can, for example, be binary, i.e., zero (0) or one (1). A binary numerical value can express whether the driver is attentive or not. The numerical value can also be a number on a two-sided scale. For example, the numerical value can take values ​​between 0 and 1 (or between 0 and 100 percent), where 0 represents complete inattention and 1 represents maximum attention. The numerical value can also be a number on a one-sided, open-ended scale. For example, the numerical value can take values ​​greater than or equal to 0. It can be provided that only whole numbers can be determined as the numerical value.

[0012] In a first step of the inventive process, the driver's gaze direction is detected. For this purpose, gaze direction detection devices known in the prior art, also known as "eye trackers" or "gaze trackers," can be used. The gaze direction is initially understood to be the spatial axis along which the driver looks (gaze direction axis). Since the gaze direction axis intersects with the vehicle surrounding the driver, the gaze direction can also be understood as the point of intersection of the gaze direction axis with the vehicle or the element of the vehicle encompassing the point of intersection.

[0013] In a further step of the process, the actual frequency distribution of the driver's gaze direction is determined. For this purpose, the driver's current gaze direction and that recorded at at least one previous time point can be evaluated. The actual frequency distribution of the gaze direction thus serves as a measure of the driver's gaze behavior.

[0014] In a further procedural step, the metric representing the driver's attention is determined as a function of the deviation of the actual frequency distribution from a target frequency distribution of the gaze direction. The target frequency distribution of the gaze direction reflects the ideal gaze behavior of a highly attentive driver. In other words, the metric is determined by comparing the driver's actual gaze behavior with the gaze behavior of an ideal (i.e., highly attentive) driver.

[0015] Studies have shown that drivers do not keep their gaze fixed on the road ahead, but rather scan their entire visible area (field of vision). The frequency with which they look in different directions is generally not uniform, but follows a characteristic pattern. For example, the driver most frequently looks at the road ahead. With decreasing frequency, they then look at other objects such as the side and rearview mirrors, the dashboard, an information display, a radio, and a side window. Therefore, simply looking away from the road ahead is not, in itself, an indication of a lack of attention on the part of the driver. On the contrary, an attentive driver will be able to keep their entire field of vision in view, to which they will frequently change their gaze direction.The inventor has determined that the frequency distribution of the driver's gaze direction depends significantly on their level of attention. For example, an inattentive driver may look away from the road ahead much less frequently than an attentive driver. By comparing the actual frequency distribution and the ideal frequency distribution of the gaze direction, the invention allows the determination of whether and / or to what extent the driver is inattentive.

[0016] In a preferred embodiment, the actual frequency distribution of the gaze direction and the target frequency distribution of the gaze direction are relative frequency distributions. Thus, the frequencies of each frequency distribution add up to one (1) or 100 percent. This has the advantage that the frequency distributions are directly comparable and the function of the deviation of the actual frequency distribution from the target frequency distribution of the gaze direction can be determined without first having to normalize one or both frequency distributions. Various methods for determining a numerical value as a function of a deviation between two frequency distributions, in particular two relative frequency distributions, are known to those skilled in the art. It should be noted that the invention encompasses each of these methods and is not limited to any one particular method.Exemplary methods for determining the measure are described in more detail below with reference to the figures using an embodiment.

[0017] In a particularly advantageous embodiment, the method includes an additional step in which the direction of gaze is assigned to a sub-area of ​​the driver's field of vision. The actual frequency distribution of the direction of gaze and the target frequency distribution of the direction of gaze are then frequency distributions of the sub-areas. Areas of the field of vision that are relevant to driving the vehicle can be particularly advantageous as sub-areas. A sub-area can be a vehicle element, such as a display or a mirror. A sub-area can also be an area of ​​the field of vision defined by the traffic situation. For example, a sub-area can be the area of ​​the vehicle's windshield in which the lane ahead is visible. A sub-area can also be referred to as a sector of the field of vision.As described above, gaze direction detection allows for the identification of a specific element or sub-area of ​​the vehicle by determining the intersection of the gaze direction axis with the vehicle. Sub-areas of the driver's field of vision in a motor vehicle can include, for example, [various examples]. • a left side window, • a left-hand wing mirror, • a horizon visible through the windshield, • an area visible through the windshield to the left of one's own lane, • a dedicated lane, • a combined instrument panel instrument, • an area visible through the windshield to the right of one's own lane, • a control element located on the center console, • an interior or rearview mirror, • a display located in the center of the vehicle, • a right-hand wing mirror, • a right-hand side window.

[0018] By using frequency distributions of the sub-areas for the invention, an even more meaningful result can be achieved. This is because only the frequency with which the gaze is directed into the respective sub-areas is then recorded. In contrast, changes in the direction of gaze within a sub-area are no longer recorded. For example, one driver might keep their gaze fixed on the lane ahead, while another driver lets their gaze wander within the sub-area encompassing that lane. However, this does not directly allow conclusions to be drawn about the attention of these two exemplary drivers. By using frequency distributions of the sub-areas, changes in the direction of gaze within a sub-area are not included in the determination of the metric.

[0019] It is possible, and expressly encompassed by the invention, that the actual frequency distribution of the gaze direction is initially captured not with reference to the sub-areas, but with fine spatial resolution. From the actual frequency distribution of the gaze direction thus captured, the actual frequency distribution of the sub-areas can be determined in a subsequent step by spatially quantifying the actual frequency distribution of the gaze direction. Thus, it is also possible to carry out the method according to the invention with variable sub-areas without necessarily having to change the steps of capturing the driver's gaze direction and determining the actual frequency distribution of the gaze direction.

[0020] According to the invention, the step of recording the driver's gaze direction is carried out for a predetermined duration. The actual frequency distribution of the gaze direction is then determined from the gaze direction recorded for the predetermined duration. In other words, the actual frequency distribution is determined by recording the gaze direction for a predetermined duration, for example, 10 seconds. This can preferably be carried out continuously, so that an actual frequency distribution is determined at each point in time, reflecting the frequencies of the gaze directions.

[0021] According to the invention, a current driving situation is determined, and the observation period is predetermined based on this situation. This design is based on the understanding that characteristic statistical patterns, which allow conclusions to be drawn about the driver's attention, can be observed within a specific time period, which depends on the specific driving situation. In other words, in some driving situations, an assessment of the driver's attention is possible within a relatively short observation period, whereas in other driving situations, an assessment of the driver's attention requires a relatively long observation period. A current driving situation can involve a driving speed. Preferably, the observation period can be increased proportionally to the driving speed.For example, the duration might be 30 seconds at a speed of 130 km / h on a motorway and 5 seconds at a speed of 30 km / h in a traffic-calmed zone. A current driving situation can relate to a road type (e.g., motorway, rural road, urban traffic). A current driving situation can relate to a present or upcoming driving action, such as a lane change, a turn, and the like. A driver assistance system in the vehicle, which already collects this information, can preferably be used to determine the current driving situation. For example, the vehicle's navigation system may have road type information. Similarly, the vehicle's central control unit may have vehicle information such as speed, steering angle, and the like.For example, a current or upcoming turning maneuver can be determined based on the steering angle and an activated turn signal (indicator).

[0022] A further advantage of the method is that the current driving situation is determined and the target frequency distribution is specified based on this situation. The inventor has observed that the target frequency distribution—that is, the frequency distribution of a highly attentive driver's gaze direction—can vary depending on the driving situation. For example, driving at higher speeds and / or on a highway may require the driver to focus very frequently on their lane and very rarely on the side windows. Conversely, driving in dense city traffic may require the driver to frequently look at the side windows because relevant cross traffic is visible there. Even when the driving situation involves a current or upcoming driving action, it is advantageous to use different target frequency distributions, each stored for a specific driving action.For example, a lane change or a turning maneuver requires an attentive driver to frequently look in the mirrors and side windows, whereas these areas should be looked at much less frequently when driving straight ahead on the motorway.

[0023] In a preferred embodiment, the step of capturing the driver's gaze direction includes capturing the driver's head direction and / or eye direction. Both head direction and eye direction determine where the driver is looking. It is therefore advantageous to capture both values. However, head direction and eye direction are also good indicators of gaze direction on their own. Since different image processing algorithms can be used for capturing head direction and eye direction, gaze direction can be captured more reliably if both head direction and / or eye direction are captured. For example, capturing eye direction may be difficult due to ambient light conditions. In this case, gaze direction can still be determined by capturing head direction.

[0024] In a further embodiment, the measured value is transmitted to a driver assistance system in the vehicle. This driver assistance system can, for example, perform a predetermined action if the measured value exceeds a predefined threshold. For instance, a visual and / or audible alert can be triggered for the driver, or a break recommendation can be displayed. It can also be provided that the driver assistance system uses the measured value as an input parameter to control a driver assistance function. For example, the distance set by an automatic distance control system to a vehicle ahead can be increased proportionally to the measured value. It can also be provided that the measured value is transmitted to nearby vehicles via vehicle-to-vehicle communication to inform them about the driver's level of attention.It can be particularly advantageous for the vehicle's driver assistance system to evaluate the transmitted measurement data and thus learn a distribution function of the measurement data for a specific driver. It can then be particularly advantageous for the system to perform certain actions if the transmitted measurement data deviates from a normal range of measurement data for that driver.

[0025] An inventive device for monitoring the attention of a vehicle driver comprises a detection device, in particular an interior camera, which captures the area of ​​the driver's head. The interior camera may be a stereo interior camera. The device further comprises a gaze direction detection unit. This unit is connected to the detection device and evaluates the data transmitted by the detection device. In particular, image processing algorithms can be used to determine the gaze direction from the images of the interior camera. The device further comprises a processing unit for carrying out the further process steps described above. The processing unit can be connected to other control units of the vehicle, which, for example, can transmit vehicle information to the processing unit.The computing unit can be connected to a driver assistance system of the vehicle for the transmission of the measurement value.

[0026] The invention is suitable for any type of vehicle that is not fully automated but driven by a human driver. The invention is therefore suitable for motor vehicles, in particular trucks and passenger cars, motorcycles, as well as rail, water, and air vehicles.

[0027] Further embodiments of the invention are explained below with reference to exemplary illustrations. These show Fig. 1. a driver's field of vision encompassing a partial area, Fig. 2. an actual frequency distribution of the sub-areas and Fig. 3. A target frequency distribution of the sub-areas.

[0028] Identical reference numerals in the figures denote identical features of the illustrated embodiments of the invention. It should be noted that the figures and the accompanying description are merely exemplary embodiments of the invention. In particular, representations of feature combinations in the figures and / or the figure description are not to be interpreted as necessarily requiring the implementation of all mentioned features. Other embodiments of the invention may contain fewer, more, and / or different features. The scope of protection and the disclosure of the invention are set forth in the accompanying claims and the complete description. It should also be noted that the illustrations are schematic representations of embodiments of the invention.The arrangement of the individual elements shown is only exemplary and may be different in other embodiments of the invention. Furthermore, the illustration is not necessarily to scale. Individual features may be enlarged or reduced for clarity.

[0029] In Fig. Figure 1 shows a field of vision 20 of the driver of an exemplary passenger car in which the invention is used in one embodiment. The driver is referred to below as the driver. The field of vision is the area visible to the driver. It is therefore not the area that the driver actually sees at any given time, but rather the area that the driver can see. The field of vision thus also includes, for example, peripheral areas that the driver can see by turning their head. The field of vision 20 is divided into sub-areas 1-13. Each of the sub-areas 1-13 represents an area that the driver typically looks at with a certain frequency. The sub-areas are: • a left side window (1), • a left exterior mirror (2), • a horizon visible through the windshield (3), • an area visible through the windshield to the left of one's own lane (4), • a dedicated lane (5), • a combination instrument panel (6), • an area visible through the windshield to the right of one's own lane (7), • a control element located on the center console (8), • a rearview mirror (9), • a display located in the center of the vehicle (10), • a right-hand exterior mirror (11), • a right side window (12), • a passenger area and a glove compartment (13).

[0030] It should be noted that the in Fig. The viewing area shown in Figure 1 also includes areas that are not assigned to any sub-area 1-13. However, this representation was chosen solely for the sake of clarity. In implementing the invention, it is advantageous if each area of ​​the viewing area 20 is assigned to a sub-area 1-13.

[0031] The vehicle is equipped with an interior camera 30 with a downstream gaze direction detection unit (not shown), which is directed towards the driver and continuously records their gaze direction. In the illustrated embodiment, only the gaze direction towards one of the sub-areas 1-13 is recorded, but not any change in the gaze direction within a sub-area 1-13. The driver's gaze direction is continuously recorded and assigned to a sub-area. An actual frequency distribution is determined from the gaze direction recorded for a predetermined duration. This duration can be, for example, 10 seconds. It may be possible to adjust the duration depending on the driving situation. In other words, in the present example, an actual frequency distribution of the gaze direction over the past 10 seconds is continuously determined.Since the recording of the gaze direction captures sub-areas 1-13, to which the driver has directed his gaze at each time, an actual frequency distribution of the gaze direction results.

[0032] Fig. Figure 2 shows an example of the actual frequency distribution H Ist of the sub-areas. As just explained, this frequency distribution H represents Ist The graph represents the direction of gaze (on sub-areas 1-13) over the past 10 seconds. The abscissa plots sub-areas 1-13, with the variable i, representing sub-areas 1-13, labeled on the abscissa. The ordinate represents the relative frequency H. Ist (i) of subrange i is plotted. The sum of the relative frequencies is 1: ∑i=113HIst(i)=1.

[0033] Based on the current frequency distribution of Fig. 2 shows that the driver has focused his gaze on display 10 for most of the past 10 seconds. The value H Ist (10)=0.5, therefore the driver has focused his gaze on display 10 for approximately 5 seconds in the past 10 seconds. In contrast, the driver has looked at his own lane 5 for approximately more than 2 seconds.

[0034] Fig. Figure 3 shows a target frequency distribution H Soll of sub-areas 1-13. In the simplest case, this is a given frequency distribution H. Soll , which, for example, may have been created once for a specific vehicle type using observations of selected test subjects. However, it could also be a target frequency distribution H. Soll This involves actions derived from the driver's past driving behavior. It may be stipulated that the target frequency distribution H Sollis selected depending on the current driving situation. The in Fig. 3. Target frequency distribution shown H Soll This shows, for example, that the driver should ideally focus their gaze on their own lane 5 for approximately 45% of the time, while during the remaining time their gaze should cover the other sub-areas 1-4 and 6-13. The sum of the relative frequencies is 1.

[0035] In the next step, the metric representing the driver's attention is calculated as a function of a deviation A from the actual frequency distribution H. Ist from the target frequency distribution H Soll the direction of view. This can be done, for example, using the following calculation rule: A=1−∑i=113|Hset(i)−Hact(i)|.

[0036] As a result, the deviation A can take values ​​between 0 (very distracted driver) and 1 (very attentive driver). In the present example, A = 0.09 = 9%. It can also be stipulated that the value A be transmitted as a numerical value to a driver assistance system. This system can be configured, for example, so that a warning symbol is displayed when a first threshold of 50% is undershot, and an audible warning signal sounds when a second threshold of 25% is undershot. This would be the case in the present example, so that the driver would receive an immediate indication that they should change their driving behavior or take a break.

[0037] The above calculation method is to be understood as an example. Numerous other functions are known that are suitable for determining the deviation of frequency distributions. For example, a quadratic function could alternatively be chosen: A=1−∑i=113(Hset(i)−Hact(i))2.

[0038] Any function that represents a deviation of the frequency distributions is suitable for implementing the invention. In particular, functions that can take values ​​between 0 and 1 are suitable.

Claims

[1] Method for determining a measure (A) representing the attention of a driver, comprising the steps - Detecting the driver's direction of gaze, - Determining an actual frequency distribution (H Ist ) the direction of view, - Determining the measure as a function of a deviation (A) of the actual frequency distribution (H) Ist ) from a target frequency distribution (H Soll ) the direction of view, - Recording the driver's gaze direction for a specified period of time, - Determining the actual frequency distribution (H Ist ) the direction of view from the direction of view recorded for the predetermined duration - Determining the current driving situation, - Specifying the time duration depending on the current driving situation. [2] Method according to claim 1, wherein the actual frequency distribution (H Ist) the direction of gaze and the target frequency distribution (H Soll ) relative frequency distributions of the viewing direction. [3] Method according to any of the preceding claims comprising the step - Assigning the direction of gaze to a sub-area (1-13) of a visual area (20) of the driver, whereby the actual frequency distribution (H Ist ) the direction of gaze and the target frequency distribution (H Soll ) the direction of view frequency distributions of the sub-areas (1-13) are. [4] Method according to any of the preceding claims comprising the steps - Determining the current driving situation, - Specifying the target frequency distribution (H Soll ) depending on the current driving situation. [5] Method according to any of the preceding claims, wherein the step of detecting the direction of the driver's gaze includes detecting the direction of the driver's head and / or eye. [6] Method according to any one of the preceding claims comprising the step - Transmitting the measurement value (A) to a driver assistance system of the vehicle. [7] Device for monitoring the attention of a vehicle driver for a vehicle comprising a detection device, in particular an interior camera (30), a gaze direction detection unit and a computing unit for carrying out the method according to one of claims 1 to 6. [8] Vehicle with a device according to claim 7.

Images