Method and system for monitoring driver attention

Abstract

The present application discloses a method and a motor vehicle control unit configured to monitor the attention of a driver of the vehicle. For this purpose, a gaze location of a driver of the vehicle is determined. If the gaze location corresponds to a rearward-facing mirror, a field of view of the rearward-facing mirror is determined. If the field of view corresponds to a view of a driving environment of the vehicle, the driver is classified as performing a driving task. If the field of view corresponds to a view of the vehicle, the driver is classified as performing a non-driving task. Finally, an attention warning is issued to the driver based on the driver classification.

Description

TECHNICAL FIELD

[0001] The invention relates generally to monitoring the attention of a driver of a vehicle and, more particularly, to preventing a false determination that a driver is paying attention to the driving environment. BACKGROUND

[0002] To determine a driver's alertness, a modern vehicle typically employs a Driver Monitoring System (DMS) that determines whether the driver is monitoring the vehicle's driving environment. For example, if a driver looks at the vehicle's dashboard or a screen located between a driver's seat and a passenger seat, the driver can be considered, i.e., classified, as performing a non-driving task, i.e., a task unrelated to the Dynamic Driving Task (DDT).When the driver looks in a rear-facing mirror, such as a rearview mirror or a side-view mirror, the driver is typically classified by the DMS as performing a driving task because looking in the rear-facing mirror is typically considered monitoring traffic behind and to the side of the vehicle as part of the Object and Event Detection and Response (OEDR) subtask of the DDT. However, the driver may have repurposed the rear-facing mirror to monitor rear seat occupants of the vehicle, particularly children sitting in the rear seats. In such a case, classifying the driver as performing a driving task when looking in the rear-facing mirror is erroneous because the driver is not looking in the rear-facing mirror to monitor the driving environment, but rather to monitor rear seat passengers.

[0003] Therefore, it is an object of the present disclosure to prevent misclassification of a driver as performing a driving task when the driver looks into any of the rear-facing mirrors of the vehicle after repurposing any of the rear-facing mirrors of the vehicle for monitoring occupants of the rear seats of the vehicle. SUMMARY OF THE INVENTION

[0004] To achieve this goal, the present disclosure provides a method for monitoring attention of a driver of a vehicle. The method includes determining a gaze location of a driver of the vehicle. If the gaze location corresponds to a rear-facing mirror, the method determines a field of view of the rear-facing mirror. If the field of view corresponds to a view of a driving environment of the vehicle, the method classifies the driver as performing a driving task. If the field of view corresponds to a view of the vehicle, the method classifies the driver as performing a non-driving task. Finally, the method issues an attention warning to the driver based on the driver classification.

[0005] The present disclosure further provides a motor vehicle control unit. The motor vehicle control unit comprises at least one processing unit and a memory coupled to the at least one processing unit and configured to store machine-readable instructions. The machine-readable instructions cause the at least one processing unit to determine a gaze location of a driver of a vehicle. If the gaze location corresponds to a rear-facing mirror, the machine-readable instructions cause the at least one processing unit to determine a field of view of the rear-facing mirror. If the field of view corresponds to a view of a driving environment of the vehicle, the machine-readable instructions cause the at least one processing unit to classify the driver as performing a driving task.If the field of view corresponds to a view of the vehicle, the machine-readable instructions cause the at least one processing unit to classify the driver as performing a non-driving task. Finally, the machine-readable instructions cause the at least one processing unit to issue an attention warning to the driver based on the driver classification.

[0006] The present disclosure further provides a vehicle including the motor vehicle control unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Examples of the present disclosure are described with reference to the following attached drawings, in which like reference numerals refer to like elements. Fig. 1A to 1C show a rearview mirror as an example of a rear-facing mirror with different fields of view according to examples of the present disclosure. Fig. 2A and Fig. 2B show a side mirror as an example of a rear-facing mirror with different fields of view according to examples of the present disclosure. Fig. 3A and Fig. 3B shows a flowchart of a method for monitoring attention of a driver of a vehicle according to examples of the present disclosure. Fig. 4 illustrates a vehicle configured to monitor attention of a driver of the vehicle, according to examples of the present disclosure. Fig. 5 illustrates a motor vehicle control unit according to examples of the present disclosure.

[0008] It should be understood that the drawings identified above are not intended to limit the present disclosure in any way. Instead, these drawings are provided to aid in understanding the present disclosure. Those skilled in the art will readily understand that aspects of the present invention shown in one drawing may be combined with aspects in another drawing or omitted without departing from the scope of the present disclosure. DETAILED DESCRIPTION

[0009] The present disclosure generally provides a method, a motor vehicle control system, and a vehicle configured to monitor attention of a driver of the vehicle. For this purpose, two verifications are performed. First, a gaze location of the driver is determined. If the gaze location corresponds to a rear-facing mirror, second, a field of view of the rear-facing mirror is determined. Depending on whether the field of view of the rear-facing mirror corresponds to a driving environment of the vehicle, the driver is classified as either performing a driving task or performing a non-driving task. That is, the second verification checks whether the rear-facing mirror is actually being used by the driver to monitor the driving environment or has instead been repurposed to monitor a passenger in a rear seat of the vehicle.If the rear-facing mirror has been converted to monitor a passenger in a rear seat of the vehicle, the driver is classified as performing a non-driving task. Based on the classification, an attention warning, such as an audible or combined visual and audible warning, is emitted in the vehicle's passenger compartment to prompt the driver to return to performing a driving task, and in particular, to return their gaze to the road ahead.

[0010] This general concept is explained with reference to the attached drawings, where Fig. 3A and Fig. 3B provides a flowchart of a method 200 for monitoring attention of a driver of a vehicle, and Fig. 1A to 2B show various examples of rear-facing mirrors with different fields of view. Additionally, Fig. 4 a vehicle according to the present disclosure, and Fig. 5 illustrates a motor vehicle controller configured to perform method 200.

[0011] It is understood that dashed boxes in Fig. 1 illustrate optional steps of the method 100.

[0012] The method 200 is designed to monitor an attention of a driver of a vehicle, such as the vehicle 300 of Fig. 4.

[0013] Briefly with reference to Fig. 2, the vehicle 300, and more generally the term vehicle in the context of the present disclosure, refers to any type of motor vehicle configured to transport people and / or cargo. The engine of the vehicle 300 may be any type of engine, such as an electric motor or an internal combustion engine. For example, the vehicle 300 may be a passenger car, as shown in Fig. 2. However, it should be understood that the vehicle 300 may also be a bus, a truck, or any other type of vehicle that includes one or more sensors 310 and a vehicle control unit 300, enabling the vehicle 300 to provide at least driver assistance. In other words, the vehicle control unit 300 and the one or more sensors 310 may be configured to enable the vehicle 300 to provide vehicle control functionality capable of at least driver assistance, i.e., Level 1 of the driving automation taxonomy defined in the SAE International J3016 standard. That is, the vehicle 300 may be configured to control the lateral movement and / or the longitudinal movement of the vehicle 300 based on vehicle sensor data provided by the one or more sensors 310, under the supervision of a driver of the vehicle 300.

[0014] It is understood that the vehicle 300 may be configured to enable higher levels of driving automation, such as partial driving automation, i.e., Level 2 or higher of the driving automation taxonomy defined in SAE International's J3016 standard.

[0015] The one or more sensors 310 are configured to collect automotive sensor data indicative of the surroundings of the vehicle 300. Accordingly, the automotive sensor data provides environmental awareness to the one or more vehicle control modules, and thereby to the vehicle 300, to enable at least driver assistance. For example, the automotive sensor data collected by the one or more sensors 310 may provide the vehicle 300 with information about the position and size of other vehicles, road surface markings, or traffic signs. For this purpose, the one or more sensors 310 may be radar sensors configured to emit radio waves to determine a distance, angle, and speed of objects in the vicinity of the vehicle based on the reflected radio waves.The one or more sensors 310 may be LIDAR (Light Detection and Ranging) sensors configured to emit laser beams to determine a distance, angle, and speed of objects in the vicinity of the vehicle 300 based on the reflected laser beams. The one or more sensors 310 may be cameras that capture images of the surroundings of the vehicle. The one or more sensors 310 may be thermal imaging cameras that capture images of the surroundings of the vehicle 300 based on infrared radiation. It is understood that LIDAR sensors, radar sensors, or cameras are merely provided as examples of sensor types of the one or more sensors 310. For example, the one or more sensors 310 may also be ultrasonic sensors.The one or more sensors 310 may be global navigation satellite system (GNSS) sensors configured to receive position data, such as satellite signals, to determine the position of the vehicle 200. More generally, the one or more sensors 310 may be any type of sensor capable of collecting automotive sensor data indicative of the surroundings of the vehicle 200. It is further understood that the one or more sensors 310 may include multiple sensors of different types. Furthermore, the one or more sensors 310 of the same type may have different characteristics, e.g., by being configured to collect sensor data at different ranges, such as a near range, a medium range, and a far range.For example, vehicle 300 may include three near-range radar sensors, one at the front and one at the rear of vehicle 300, one mid-to-long-range radar sensor at the rear of vehicle 200, one LIDAR sensor at the front of vehicle 300, one rear-facing camera at the rear of vehicle 200, one forward-facing camera at the front of the vehicle, one forward-facing camera on the rearview mirror, and one near-to-mid-range rear-facing radar sensor in each door-mounted exterior sideview mirror. It is understood that vehicle 300 may include more or fewer automotive sensors than in . Fig. 4 shown and discussed in the example above.

[0016] The vehicle 300 further includes one or more interior sensors 330 configured to capture interior data indicative of at least one driver behavior. The driver behavior includes at least a gaze location of the driver and may include other behavioral elements indicative of the driver's attention to the vehicle 300, such as eye opening or objects in the driver's hands. For this purpose, the one or more interior sensors 330 may be cameras that capture images of at least the driver, or may be thermal imaging cameras that capture images of at least the driver based on infrared radiation, or may be any other type of sensor configured to capture the interior data.It is understood that the interior data may be collected by a single interior sensor 330, as well as a plurality of interior sensors 330, which may be the same or different sensor types, to collect the driver's interior data from different angles and / or based on a combination of sensor types. The one or more interior sensors 330 may be integrated into an instrument cluster behind the steering wheel, located next to the rearview mirror, or located in any other location that allows for the collection of the interior data.

[0017] In the context of the present disclosure, driver attention refers to the driver being fully or at least primarily focused on performing driving tasks, i.e., tasks related to the dynamic driving task (DDT) as defined in SAE International standard J3016.Accordingly, driving tasks include all of the real-time operational and tactical tasks required to operate a vehicle in road traffic, excluding strategic functions such as time-based trip planning and selection of destinations and waypoints, and including, without limitation, lateral vehicle movement control via steering, longitudinal vehicle movement control via acceleration and deceleration, monitoring of the driving environment via object and event detection, recognition, classification and response preparation, object and event response execution, maneuver planning and conspicuity enhancement via lighting, horns, signaling, gesturing, etc.The subtasks of monitoring the driving environment by means of object and event detection, recognition, classification, and response preparation, as well as object and event response execution, are collectively referred to as object and event detection and response (OEDR). Accordingly, OEDR, as defined in SAE International's J3016 standard, should be understood as monitoring the driving environment and executing an appropriate response in response to such objects and events as part of DDT. Although driver attention in the context of the present disclosure refers to the driver being fully or at least primarily focused on performing driving tasks, the present disclosure focuses on whether the driver monitors the driving environment when looking in rear-facing mirrors.Accordingly, in the context of the present disclosure, the driver's attention refers to whether the driver performs driving tasks that correspond to the OEDR subtasks of the DDT.

[0018] In step 210, the method 200 determines a gaze location of a driver of the vehicle 300. To this end, the method 200 may utilize the interior sensors 330, and more specifically, the interior data collected by the interior sensors 330. For example, the interior data may indicate one or more head position angles, e.g., yaw, pitch, and roll, of the driver's head, based on which the method 200, as part of step 210, may derive the gaze location using the known location of the one or more interior sensors 330. In addition to or instead of the head position angles, the interior data may indicate the position of the driver's eyes to derive the gaze location. For example, if both the position of the eyes and the head position angles are used to derive the gaze location, the method 200 may determine, in step 210, a gaze vector in a coordinate system that may, for example,may be centered on the driver's head or a fixed position of the passenger compartment of the vehicle 300 to determine the gaze location. If more than one interior sensor 330 is used, the gaze location may also be determined based on triangulation, i.e., based on the known positions of the various interior sensors 330 and the gaze vectors determined by each interior sensor 330. It should be understood that these examples for determining the gaze location are provided merely as examples. The gaze location may be determined in any manner suitable for monitoring the driver's attention to the one or more interior sensors 330 deployed and the data collected by the one or more sensors 330.

[0019] The method 200 then proceeds to determine, in step 211, whether the view location corresponds to a rear-facing mirror 100. In the context of the present disclosure, the rear-facing mirror refers to any type of mirror that enables the driver of the vehicle 300 to monitor the driving environment adjacent to and behind the vehicle 300, such as the side mirrors and the rearview mirror of the vehicle 300. Thus, it is understood that the vehicle 300 may typically include at least three rear-facing mirrors 100, and any reference to the rear-facing mirror 100 should equally be understood to include any subset or all of the rear-facing mirrors 100 of the vehicle 300. As discussed with reference to step 210, the method 200 may determine, in step 211, that the view location corresponds to one of the rear-facing mirrors 100, e.g.,based on the gaze vector in the coordinate system centered at a fixed location of the passenger compartment of the vehicle 300 and the known position of one of the rear-facing mirrors 100 in the coordinate system or based on inferring from the head position angles that the driver is most likely looking at one of the rear-facing mirrors.

[0020] If the method 200 determines in step 211 that the view location corresponds to the rearward-facing mirror 100, the method 100 may proceed to steps 230a through 252, as shown in Fig. 3A. However, in some examples of the present disclosure, after determining that the view location corresponds to the rearward-facing mirror 100, the method 200 may first proceed to step 220. In step 220, the method 200 may detect an occupancy state of each of a plurality of rear seats 120 of the vehicle 300. Each occupancy state may indicate occupancy of the corresponding rear seat 120. That is, the method 200 may detect whether any of the rear seats 120 are occupied by a passenger. If none of the rear seats 120 are occupied by a passenger, the method 200 may assume that there is no reason for the driver of the vehicle 300 to repurpose any of the rearward-facing mirrors 100 of the vehicle 300. Accordingly, the method 200 may proceed directly to step 251 to classify the driver as performing a driving task.In other words, the method 200 may, in particular, omit step 240 if no re-functioning of the one or more rear-facing mirrors is expected due to the absence of passengers in the rear seats 120 of the vehicle 300.

[0021] Depending on the implementation of the method 200, the method 200 may proceed to step 230a to determine a gaze duration corresponding to a time interval during which the gaze location corresponds to one of the rear-facing mirrors 100, following determining that the gaze location corresponds to one of the rear-facing mirrors in step 211 or following detecting that none of the rear seats 120 are occupied in step 221. In addition to or instead of step 230a, the method 200 may also perform step 230b, where the method 200 may determine a plurality of gaze durations during a gaze accumulation period. In other words, the method 200 may determine the duration of a single gaze into one of the rear-facing mirrors 100 or the accumulated duration of multiple gazes into one of the rear-facing mirrors 100 over a predetermined period of time, i.e., the gaze accumulation period.Because the driver is not paying attention to the road ahead of the vehicle 300 while looking into one of the rear-facing mirrors 100, a driver who looks into one of the rear-facing mirrors 100 for an extended period of time may be considered inattentive, even if the driver is monitoring the driving environment of the vehicle 300 via the one or more rear-facing mirrors 100. Accordingly, an attention warning may also be issued to the driver as part of step 260 based on the gaze duration and / or accumulated gaze durations, as discussed below.

[0022] It is understood that both steps 230a and 230b may be performed simultaneously to determine whether individual gaze durations, as well as an accumulated gaze duration as the sum of the plurality of gaze durations, may result in the driver being considered inattentive. It is further understood that steps 230a and 230b may also be performed subsequent to step 240, e.g., if the driver is classified as performing a driving task in step 251, since otherwise the method 200 may not issue the attention warning to the driver as part of step 260 with respect to the classification in step 251. It is further understood that steps 230a and 230b may be performed subsequent to steps 220 and 221, i.e., the gaze duration and / or accumulated gaze durations may be determined following the determination that none of the rear seats 120 are occupied, since the gaze duration and / or accumulated gaze durations may be used in step 260 even if the method 200 foregoes steps 240 to 242 due to the lack of detection of occupants in the rear seats 120 in step 221.

[0023] In step 240, the method 200 determines a field of view of the rearward-facing mirror 100, i.e., an environment observable in the rearward-facing mirror 100 from the perspective of the driver of the vehicle 300. The method 300 may determine the field of view of the rearward-facing mirror 100, for example, based on sensors of the vehicle 100, such as one of the interior sensors 330 if the rearward-facing mirror 100 is a rearview mirror, or one of the vehicle sensors 310 if the rearward-facing mirror 100 is a side view mirror. The method 200 may also determine the field of view based on an angle at which the rearward-facing mirror 100 is arranged with respect to a mounting surface of the rearward-facing mirror 100, e.g., based on known fields of view with the angle or based on calculating an expected field of view based on the angle.

[0024] Based on the field of view determined in step 240, the method 200 determines in step 241 whether the field of view corresponds to a view of the driving environment of the vehicle 300. The field of view corresponds to the view of the driving environment of the vehicle 300 if the field of view includes less of a view of the vehicle 300 than a vehicle view threshold. More specifically, the field of view of the rear-facing mirror 100 always includes at least some elements of the vehicle 300 that form the view of the vehicle 300, in addition to the view of the driving environment.

[0025] The concept of the field of view of the rearward facing mirror 100 is in Fig. 1A, which illustrates the field of view of the rear-facing mirror 100, which is implemented as a rearview mirror. Fig. 1A includes elements of the vehicle 300 in the form of a portion of a vehicle ceiling 140, a portion of the front seat headrests 110, a portion of the rear seats 120, and the rear seat headrests 130. The driving environment in Fig. 1A is visible through the rear window behind the rear seat headrests 130. Fig. 1A illustrates a typical field of view of a rearview mirror, i.e., the field of view used by the driver of vehicle 300 to monitor the driving environment behind vehicle 300. Based on this typical field of view, the vehicle view threshold in the context of the present disclosure may be defined as the percentage of vehicle elements visible in the typical field of view, thereby taking into account, for example, the geometry of vehicle ceiling 140, front seat headrests 110, rear seats 120, rear seat headrests 130, and the rear window of vehicle 300, as well as any other vehicle elements that may be visible in the field of view of rearward-facing mirror 100.

[0026] Accordingly, if the percentage of vehicle elements in the field of view of the rearward-facing mirror 100 exceeds the vehicle view threshold, as determined based on the typical field of view of the rearward-facing mirror 100, the field of view of the rearward-facing mirror 100 is considered to correspond to a vehicle view, as shown in Fig. 1B. In Fig. 1B, the driving environment above the rear seats 120 is in the field of view of the exemplary rearview mirror of Fig. 1B is barely visible, and the percentage of vehicle elements is greatly increased when the driver opens the rearview mirror from Fig. 1B to enable monitoring of a child in one of the rear seats 120. Accordingly, the field of view of the rearview mirror of Fig. 1B of the view of the vehicle 300.

[0027] The concept of the view of the driving environment compared to the view of the vehicle 300 is further illustrated with an exemplary side mirror in Fig. 2A and Fig. 2B illustrates. Fig. 2A shows the exemplary side mirror with a typical field of view of the side mirror, which includes door handles 170 as well as parts of the doors and side windows of the vehicle 300 as well as the driving environment adjacent to the vehicle 300. The percentage of the side mirror field of view of Fig. 2A, which shows these vehicle elements of the vehicle 300 in the typical field of view of the side mirror of Fig. 2A defines the vehicle view threshold for the side mirror of Fig. 2A. In contrast, Fig. 2B the field of view of the side mirror of Fig. 2A, after the side mirror was converted to monitor a child in one of the rear seats 120. As in Fig. 2B, more elements of the vehicle 300 are at least partially in the field of view of the side mirror of Fig. 2B, such as the rear seat headrest 130, than in the field of view, i.e., the typical field of view, of the side mirror of Fig. 2A. Consequently, the percentage of the field of view corresponding to the view of the vehicle 300 in Fig. 2B compared to Fig. 2A is greatly increased, and the field of view of the side mirror of Fig. 2B is therefore considered to correspond to the view of the vehicle 300 rather than the view of the driving environment.

[0028] In summary, in step 241, the method 200 may determine whether the field of view corresponds to the view of the driving environment or the view of the vehicle 300 based on the vehicle view threshold, which corresponds to the percentage of the typical field of view of the rear-facing mirror 100 that includes elements of the vehicle 300. The typical field of view, such as in Fig. 1A for the exemplary rearview mirror and in Fig. 2A for the example side view mirror corresponds to the typical field of view of the rear-facing mirror 100, which enables monitoring of the driving environment behind and / or to the side of the vehicle 300. The typical field of view and corresponding vehicle view threshold may be determined by a vehicle manufacturer based on an average adjustment of the rear-facing mirror 100 and potentially one or more standard deviations of the average adjustment of the rear-facing mirror 100. The typical field of view and corresponding vehicle view threshold may also be determined for the driver of the vehicle 300 after initial setup of the vehicle 300 or each time the vehicle 300 detects an unknown driver of the vehicle 300 based on the interior data from one of the interior sensors 330.

[0029] If the field of view corresponds to the view of the driving environment of the vehicle 300, the method 200 may proceed to step 251 to classify the driver as performing the driving task. If the field of view corresponds to the view of the vehicle 300, the method 200 may proceed to step 252 to classify the driver as performing the non-driving task, ie, any type of task that does not relate to any subtask of the DDT.

[0030] If the field of view corresponds to the view of the driving environment of the vehicle 300, the method 200 may perform step 242 before proceeding from step 241 to step 251. In step 242, the method 200 may detect an auxiliary mirror in the field of view of the rearward-facing mirror 100. The auxiliary mirror may have an auxiliary field of view that includes at least one rear seat 120. That is, in step 242, the method 200 may check whether the driver 200 has repurposed the rearward-facing mirror 100 to monitor occupants of the rear seats 120. This concept is described in Fig. 1C, which has the same field of view of the rearview mirror as Fig. 1A, but with an auxiliary mirror 150 attached to one of the rear seat headrests 130 by means of auxiliary mirror attachment means 150F. As in Fig. 1C, the auxiliary field of view of the auxiliary mirror 150 includes a toddler who is in a child car seat attached to one of the rear seats 120. Although the field of view of the rearview mirror of Fig. 1C the field of view of the rearview mirror of Fig. 1A and thus the typical field of view of the rearview mirror, as discussed above, the driver of the vehicle 300 nevertheless has the rearview mirror of Fig. 1C to monitor the infant. Thus, in step 242, the method 200 may verify whether the driver has repurposed the rear-facing mirror 100 without changing the field of view from the typical field of view. Accordingly, as a result of detecting the auxiliary mirror 150 in step 242, the method 200 may proceed to step 252 to classify the driver as performing the non-driving task, even though the field of view corresponds to the typical field of view and thus the view of the driving environment of the vehicle 300, as determined in step 241.

[0031] After the driver has been classified in steps 251 and 252 as performing either the driving task or the non-driving task, the method 200 proceeds to step 260, where the method 200 issues an attention warning to the driver based on the driver's classification. The attention warning may be any type of signal that at least warns the driver to return to performing the driving task. Accordingly, step 260 may include a step 261 where the method 200 issues the attention warning to the driver if the driver is classified as performing the non-driving task.

[0032] The attention warning may be any type of optical and / or visual warning emitted in the passenger compartment of the vehicle 300, such as a tone, a voice message instructing the driver to pay attention to the DDT / OEDR subtask, a flashing icon in the instrument cluster of the vehicle 300 and / or a head-up display of the vehicle 300, or any other type of warning that causes the driver to return to performing the driving task.

[0033] In addition to being a signal that at least warns the driver to return to performing the driving task, the attention warning may further warn the driver to return to monitoring the driving environment in front of the vehicle 300. To this end, issuing the attention warning to the driver in step 260 may further be based on gaze duration, and more specifically, on the gaze duration and / or the accumulated gaze duration. In other words, step 260 may include one or both of steps 262 and 263 based on the gaze duration and / or the accumulated gaze duration determined in steps 230a and 230b.

[0034] In step 262, as part of step 260, the method 200 may issue the attention warning to the driver if the driver is classified as performing the driving task and if the gaze duration exceeds a gaze duration threshold, ie, a period of time after which the driver's attention should return to monitoring the road ahead of the vehicle 300.

[0035] In step 263, as part of step 260, the method 200 may issue the attention warning to the driver if the driver is classified as performing the driving task and if the sum, i.e., the accumulated gaze duration of the plurality of gaze durations during the gaze accumulation period, exceeds a gaze accumulation threshold. In other words, the method 200 may issue the attention warning to the driver if the driver looks in the rearview mirror for short periods that are not individually considered excessive interruptions to monitoring the road ahead, but taken together over the accumulation period constitute a combined excessive interruption to monitoring the road ahead.

[0036] Method 200 may further issue different types of attention alerts as part of step 260 based on different driver distraction levels, as determined based on the driver's classification and gaze duration and / or accumulated gaze durations. The different types of attention alerts may be similar to the attention alert discussed above, but may vary in intensity depending on the level of the different types of attention alerts.

[0037] For this purpose, step 260 may include a step 264 where the method 200 may issue a basic attention warning if one of a plurality of basic attention warning conditions is met. The plurality of basic attention warning conditions may include at least a first basic warning condition and a second basic warning condition. The first basic warning condition includes the driver being classified as performing the driving task and the gaze duration exceeding the gaze duration threshold and / or the sum of the plurality of gaze durations during the gaze accumulation period exceeding the gaze accumulation threshold. The second basic warning condition may include the driver being classified as performing the non-driving task.In other words, the basic attention warning may correspond to the attention warning as issued in steps 261 and 262, ie, the basic attention warning may be issued if the driver is classified as performing the non-driving task or if the driver is classified as performing the driving task but spends too much time monitoring the driving environment behind the vehicle 300 and / or beside the vehicle 300 by means of the rear-facing mirror 100.

[0038] In addition to step 264, step 260 may further include a step 265 in which the method 200 issues a heightened attention warning if one of a plurality of heightened attention warning conditions is met. The plurality of heightened attention warning conditions may include at least a first heightened attention warning condition. The first heightened attention warning condition may include the driver being classified as performing the non-driving task and the gaze duration exceeding the gaze duration threshold or the sum of the plurality of gaze durations during the gaze accumulation period exceeding the gaze accumulation threshold. In other words, the method 200 may issue the heightened attention warning if the driver is classified as performing the non-driving task and the driver is not monitoring the road ahead of the vehicle 300 for extended periods.The increased attention warning may be justified in such a situation since it is already assumed that the driver is distracted due to the repurposing of the rear-facing mirror 100 and is further not paying attention to the road ahead for at least the period of time defined by the gaze duration threshold and / or the gaze accumulation threshold.

[0039] It will be understood that the plurality of basic alert conditions and the plurality of heightened awareness alert conditions may include additional conditions than those discussed above with respect to steps 264 and 265.

[0040] In summary, the method 200 determines whether the driver of the vehicle 300 has repurposed the rear-facing mirror 100 and, accordingly, classifies the driver as performing the driving task, i.e., a task related to the DDT / OEDR subtask, or the non-driving task, i.e., any task not related to the DDT / OEDR subtask, when looking into the rear-facing mirror 100. Additionally, the method 300 may monitor the duration of individual glances as well as the accumulated duration of multiple glances over a predefined period of time to determine whether the driver should return to monitoring the driving environment in front of the vehicle 300 even while performing the driving task, and may further increase the intensity of the warning issued to the driver not performing the driving task.

[0041] Fig.5 shows a motor vehicle control unit 400 configured to perform method 200. The motor vehicle control unit 400 may include a processor 410, a graphics processing unit (GPU) 420, a motor vehicle processing system 430, a memory 440, a removable storage 450, a storage 460, a cellular interface 470, a global navigation satellite system (GNSS) interface 480, and a communications interface 490.

[0042] Processor 410 may be any type of single-core or multi-core processing unit employing a RISC (Reduced Instruction Set) or CISC (Complex Instruction Set). Example RISC processing units include ARM-based cores or RISC-V-based cores. Example CISC processing units include x86-based cores or x86-64-based cores. Processor 410 may execute instructions that cause vehicle control unit 400 to perform method 200. Processor 410 may be directly coupled to any of the components of vehicle control unit 400 or may be directly coupled to memory 430, GPU 420, and a device bus.

[0043] The GPU 420 may be any type of processing unit optimized for processing graphics related to instructions or, more generally, for parallel processing of instructions. Accordingly, the GPU 420 may be configured to generate a display of information, such as ADAS information or telemetry data, to a driver of the vehicle, e.g., via a head-up display (HUD) or a display located in the driver's view. The GPU 420 may be coupled to the HUD and / or the display via a connection 420C. The GPU 420 may further perform at least a portion of the method 100 to enable rapid parallel processing of instructions related to the method 100. It should be noted that in some embodiments, the processor 410 may determine that the GPU 420 does not need to perform the instructions related to the method 200.The GPU 420 may be directly coupled to any of the components of the vehicle control unit 400 or may be directly coupled to the processor 410 and the memory 430. In some embodiments, the GPU 420 may also be coupled to the device bus.

[0044] The automotive processing system 430 may be any type of system-on-chip configured to provide trillions of operations per second (TOPS) to enable the automotive control unit 400 to implement one or more ADAS while driving. The automotive processing system 430 may be linked only to the processor 410 or may be linked to other devices via the system bus. For example, the automotive processing system 430 may execute the instructions related to the one or more automotive sensor data processing modules and the one or more vehicle control modules.

[0045] Memory 440 may be any type of fast storage that enables the processor 410, GPU 420, and automotive processing system 430 to store instructions for rapid retrieval during instruction processing, as well as to cache and buffer data. Memory 440 may be a unified memory coupled to the processor 410, GPU 420, and automotive processing system 430 to enable allocation of memory 440 to the processor 410, GPU 420, and automotive processing system 430 as needed. Alternatively, the processor 410, GPU 420, and automotive processing system 430 may be coupled to a separate processor memory 440a, GPU memory 440b, and automotive processing system memory 440c.

[0046] Removable storage 450 may be a storage device that can be removably coupled to vehicle control unit 400. Examples include a DVD (Digital Versatile Disc), a CD (Compact Disc), a USB (Universal Serial Bus) storage device such as an external SSD, or a magnetic tape. It should be noted that removable storage 450 may store data such as instructions of method 200, vehicle sensor data, intermediate data, and / or vehicle control data, or may be omitted.

[0047] Storage 460 may be a storage device that enables the storage of program instructions and other data. For example, storage 460 may be a hard disk drive (HDD), a solid-state drive (SSD), or another type of non-volatile memory. Storage 460 may store, for example, the instructions of method 100, vehicle sensor data, intermediate data, and / or vehicle control data.

[0048] Removable storage 450 and storage 460 may be coupled to processor 410 via the system bus. The system bus may be any type of system bus that enables processor 410 and optionally GPU 420 and automotive processing system 430 to communicate with the other devices of automotive control unit 400. Bus 440 may be, for example, a Peripheral Component Interconnect Express (PCIe) bus or a Serial AT Attachment (SATA) bus.

[0049] The cellular interface 470 may be any type of interface that enables the vehicle control unit 400 to communicate over a cellular network, such as a 4G network or a 5G network.

[0050] The GNSS interface 480 may be any type of interface that enables the vehicle control unit 300 to receive position data provided by a satellite network, such as the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), or Galileo. The position data may be one of the types of vehicle sensor data in the context of the present disclosure.

[0051] The communication interface 490 may enable the vehicle control unit 400 to interface with external devices either directly or via a network. For example, the communication interface 480 may enable the vehicle control unit 300 to interface with a wired or wireless network, such as Ethernet, Wi-Fi, a CAN (Controller Area Network) bus, or any bus system suitable for use in vehicles. For example, the vehicle control unit 400 may be coupled to the one or more vehicle sensors 310 to receive vehicle sensor data.

[0052] The motor vehicle control unit 400 may be integrated with the vehicle 300, e.g., under the passenger compartment, under the dashboard, or in the trunk of the vehicle 300.

[0053] The invention can be further illustrated by the following examples.

[0054] In one example, a method for monitoring attention of a driver of a vehicle includes determining a gaze location of a driver of the vehicle, and if the gaze location corresponds to a rear-facing mirror, determining a field of view of the rear-facing mirror if the field of view corresponds to a view of a driving environment of the vehicle, classifying the driver as performing a driving task if the field of view corresponds to a view of the vehicle, classifying the driver as performing a non-driving task, and issuing an attention warning to the driver based on the classification of the driver.

[0055] In the example method, the field of view may correspond to the view of the driving environment of the vehicle if the field of view includes less of the view of the vehicle than a vehicle view threshold.

[0056] The exemplary method may further include, if the viewing location corresponds to the rear-facing mirror, detecting an occupancy state of each of a plurality of rear seats of the vehicle, each occupancy state indicating occupancy of each corresponding rear seat, and classifying the driver as performing the driving task if each occupancy state indicates that none of the rear seats of the plurality of rear seats are occupied.

[0057] In the example method, the example method may forego determining the field of view if each occupancy condition indicates that none of the rear seats of the plurality of rear seats are occupied.

[0058] In the exemplary method, determining the field of view of the rearward-facing mirror may include detecting an auxiliary mirror in the field of view of the rearward-facing mirror, the auxiliary mirror having an auxiliary field of view including at least one rear seat, and the exemplary method may further perform classifying the driver as the non-driving task even if the field of view corresponds to the view of the driving environment of the vehicle if the auxiliary mirror is detected.

[0059] In the example method, issuing the attention warning to the driver may include issuing the attention warning to the driver if the driver is classified as performing the non-driving task.

[0060] The example method may further include, if the gaze location corresponds to a rear-facing mirror, determining a gaze duration, wherein the gaze duration corresponds to a time interval during which the gaze location may correspond to the rear-facing mirror, wherein issuing the attention warning to the driver may further be based on the gaze duration.

[0061] In the example method, issuing the attention warning to the driver may further include issuing the attention warning to the driver if the driver is classified as performing the driving task and if the gaze duration exceeds a gaze duration threshold.

[0062] The exemplary method may further comprise, if the gaze location corresponds to a rear-facing mirror, determining a plurality of gaze durations during a gaze accumulation period, wherein issuing the attention warning to the driver further includes issuing (263) the attention warning to the driver if the driver is classified as performing the driving task and if a sum of the plurality of gaze durations during the gaze accumulation period exceeds a gaze accumulation threshold.

[0063] In the exemplary method, issuing the attention warning to the driver may further include issuing a basic attention warning if one of a plurality of basic attention warning conditions is met, wherein the plurality of basic attention warning conditions comprises at least a first basic warning condition and a second basic warning condition, wherein the first basic warning condition comprises the driver being classified as performing the driving task and the gaze duration exceeds the gaze duration threshold or the sum of the plurality of gaze durations during the gaze accumulation period exceeds the gaze accumulation threshold, and the second basic warning condition comprises the driver being classified as performing the non-driving task, and issuing an increased attention warning if one of a plurality of increased attention warning conditions is met,wherein the plurality of increased attention warning conditions comprises at least a first increased attention warning condition, wherein the first increased attention warning condition comprises the driver being classified as performing the non-driving task and the gaze duration exceeding the gaze duration threshold or the sum of the plurality of gaze durations during the gaze accumulation period exceeding the gaze accumulation threshold.

[0064] An exemplary motor vehicle control unit comprises at least one processing unit and a memory coupled to the at least one processing unit and configured to store machine-readable instructions, the machine-readable instructions causing the at least one processing unit to determine a gaze location of a driver of a vehicle, and if the gaze location corresponds to a rear-facing mirror, determine a field of view of the rear-facing mirror if the field of view corresponds to a view of a driving environment of the vehicle, classify the driver as performing a driving task if the field of view corresponds to a view of the vehicle, classify the driver as performing a non-driving task, and issue an attention warning to the driver based on the driver classification.

[0065] In the exemplary motor vehicle control unit, the machine-readable instructions may further cause the at least one processing unit to perform any of the exemplary methods discussed above.

[0066] An exemplary vehicle includes any of the exemplary automotive control units discussed above.

[0067] The foregoing description has been provided to illustrate monitoring a driver's attention. It should be understood that the description is in no way intended to limit the scope of the present disclosure to the precise embodiments discussed throughout the description. Rather, those skilled in the art will appreciate that the examples of the present disclosure may be combined, modified, or combined without departing from the scope of the present disclosure as defined by the following claims. List of reference symbols 100 rear-facing mirror 110 Front seat headrest 120 rear seat(s) 130 Rear seat headrest 140 vehicle ceiling 150 auxiliary mirrors 150F auxiliary mirror fasteners 160 front seat 170 door handle 200 procedures 210-265 procedural steps 300 vehicles 310 Automotive sensor 320 light 330 interior sensor 400 Motor vehicle control unit 410 CPU 420 GPU 420c connection 430 Automotive processing system 440 memory 450 removable storage 460 Storage 470 mobile radio interface 480 GNSS interface 490 Communication interface

Claims

[1] A method (200) for monitoring an attention of a driver of a vehicle (300), comprising: Determining (210) a viewing location of a driver of the vehicle (300); if the viewing location corresponds to a rear-facing mirror (100) (211): Determining (240) a field of view of the rearward-facing mirror (100); if the field of view corresponds (241) to a view of a driving environment of the vehicle (300), classifying (251) the driver as performing a driving task; if the field of view corresponds to a view of the vehicle (300) (241), classifying (252) the driver as performing a non-driving task; and Outputting (260) an attention warning to the driver based on the driver's classification. [2] The method (200) of claim 1, wherein the field of view corresponds (241) to the view of the driving environment of the vehicle (300) if the field of view includes less of the view of the vehicle (300) than a vehicle visibility threshold. [3] Method (200) according to one of claims 1 and 2, further comprising: if the viewing location corresponds to the rear-facing mirror (100) (211): Detecting (220) an occupancy state of each of a plurality of rear seats of the vehicle (300), each occupancy state indicating an occupancy of each corresponding rear seat (120); and Classifying (251) the driver as performing the driving task if each occupancy condition indicates that none of the rear seats (120) of the plurality of rear seats are occupied. [4] The method (200) of claim 3, wherein the method (200) omits determining (240) the field of view if each occupancy state indicates that none of the rear seats (120) of the plurality of rear seats is occupied. [5] Method (200) according to one of the preceding claims, wherein: determining (240) the field of view of the rearward-facing mirror (100) comprises detecting (242) an auxiliary mirror (150) in the field of view of the rearward-facing mirror (100), the auxiliary mirror (150) having an auxiliary field of view that includes at least one rear seat (120); and the method (200) further comprises classifying (252) the driver as performing the non-driving task even if the field of view corresponds to the view of the driving environment of the vehicle if the auxiliary mirror is detected. [6] The method (200) of any preceding claim, wherein issuing (260) the attention warning to the driver includes issuing (261) the attention warning to the driver if the driver is classified as performing the non-driving task. [7] Method (200) according to any one of the preceding claims, further comprising: if the viewing location corresponds to a rearward-facing mirror (100), determining (230a) a viewing duration, wherein the viewing duration corresponds to a time interval during which the viewing location corresponds to the rearward-facing mirror (100), wherein issuing (260) the attention warning to the driver is further based on the gaze duration. [8] The method (200) of claim 7, wherein outputting (260) the attention warning to the driver further includes outputting (262) the attention warning to the driver if the driver is classified as performing the driving task and if the gaze duration exceeds a gaze duration threshold. [9] Method (200) according to one of claims 7 and 8, further comprising: if the gaze location corresponds to a rear-facing mirror, determining (230b) a plurality of gaze durations during a gaze accumulation period, wherein issuing (260) the attention warning to the driver further includes issuing (263) the attention warning to the driver if the driver is classified as performing the driving task and if a sum of the plurality of gaze durations during the gaze accumulation period exceeds a gaze accumulation threshold. [10] The method (200) of claims 7 to 9, wherein outputting (260) the attention warning to the driver further includes: Outputting (264) a basic attention warning if one of a plurality of basic attention warning conditions is met, wherein the plurality of basic attention warning conditions comprises at least a first basic warning condition and a second basic warning condition, wherein the first basic warning condition comprises the driver being classified as performing the driving task and the gaze duration exceeds the gaze duration threshold or the sum of the plurality of gaze durations during the gaze accumulation period exceeds the gaze accumulation threshold, and the second basic warning condition comprises the driver being classified as performing the non-driving task; and Outputting (265) a heightened attention warning if one of a plurality of heightened attention warning conditions is met, wherein the plurality of heightened attention warning conditions comprises at least a first heightened attention warning condition, wherein the first heightened attention warning condition comprises the driver being classified as performing the non-driving task and the gaze duration exceeds the gaze duration threshold or the sum of the plurality of gaze durations during the gaze accumulation period exceeds the gaze accumulation threshold. [11] Motor vehicle control unit (400), comprising: at least one processing unit (410, 420, 430); and a memory (440, 450) coupled to the at least one processing unit (410, 420, 430) and configured to store machine-readable instructions, wherein the machine-readable instructions cause the at least one processing unit (410, 420, 430) to: Determining a viewing location of a driver of a vehicle (300); if the viewing location corresponds to a rear-facing mirror (100): Determining a field of view of the rear-facing mirror (100); if the field of view corresponds to a view of a driving environment of the vehicle (300), classifying the driver as performing a driving task; if the field of view corresponds to a view of the vehicle (300), classifying the driver as performing a non-driving task; and Issue an attention warning to the driver based on the driver's classification. [12] The motor vehicle control unit (400) of claim 11, wherein the machine-readable instructions further cause the at least one processing unit (410, 420, 430) to perform the method of any one of claims 2 to 10. [13] Vehicle (300) comprising the motor vehicle control unit (410, 420, 430) according to one of claims 11 and 12.

Images