Method and system for monitoring driver attention

Abstract

Method (200) for monitoring the attention of a driver of a vehicle (300), comprising: Determine (210) a viewpoint of a driver of the vehicle (300); if the viewpoint corresponds to a rear-facing mirror (100) (211): Determine (240) a field of view of the rear-facing mirror (100); if the field of vision corresponds to a view of a driving environment of the vehicle (300) (241), classify (251) the driver as performing a driving task; if the field of vision corresponds to a view of the vehicle (300) (241), classifying (252) the driver as performing a non-driving task; and Issuing (260) an attention warning to the driver based on the driver's classification, where the field of view corresponds to the view of the vehicle's driving environment (300) (241), if the field of view includes less of the view of the vehicle (300) than a vehicle view threshold.

Description

TECHNICAL AREA

[0001] The invention relates generally to monitoring the attention of a driver of a vehicle and more specifically to preventing a false determination that a driver is paying attention to the driving environment. BACKGROUND

[0002] To determine a driver's alertness, a modern vehicle typically employs a Driver Monitoring System (DMS) that assesses whether the driver is monitoring the vehicle's driving environment. For example, if a driver is looking at the vehicle's dashboard or a screen located between the driver's and passenger's seats, it may be considered, i.e., classified, that the driver is performing a non-driving task, meaning a task unrelated to the dynamic driving task (DDT).When the driver looks in a rearview mirror, such as a side mirror or a rear-view mirror, the driver is typically classified by the DMS as performing a driving task, since looking in the rearview mirror is typically considered monitoring traffic behind and beside the vehicle as part of the Object and Event Detection and Response (OEDR) subtask of the DDT. However, the driver may have repurposed the rearview mirror to monitor rear-seat occupants, particularly children. In such a case, classifying the driver as performing a driving task when looking in the rearview mirror is incorrect, as the driver is not looking to monitor the driving environment but rather to monitor passengers in the rear seats.

[0003] Therefore, an objective of the present disclosure is to prevent an incorrect classification of a driver as performing a driving task when the driver looks in any of the rear-facing mirrors of the vehicle after having repurposed any of the rear-facing mirrors of the vehicle for monitoring occupants in the rear seats of the vehicle.

[0004] Document WO 2010 / 133331 A1 discloses a device for estimating the visual detection range, capable of accurately estimating an area visually detected by a driver in real time. The device calculates a visual detection range as perceived by the driver through a section for calculating the visual detection range based on the driver's line of sight, and estimates a time-serial visual detection range, which is an area currently being visually detected by the driver, through a section for generating a spatiotemporal map of visual detection based on the history of multiple visual detection ranges calculated over a predetermined period from the past to the present.

[0005] Document DE 10 2009 005 7310 A1 discloses a method for monitoring the attention of a driver of a motor vehicle comprising the following steps: determining an actual direction of gaze, i.e., the direction in which the driver is looking; determining a target direction of gaze, i.e.,the direction in which the driver should look to safely operate the vehicle, evaluating the actual direction of gaze against the target direction of gaze to determine the driver's attention or inattention, determining a time limit within which driver inattention is tolerable, recording the duration of the inattention upon detection and comparing it with the time limit, and alerting the driver if the time limit is exceeded, the method being characterized in that the target direction of gaze and / or the time limit are determined depending on the position of a turn signal indicator, information about the road ahead of the vehicle, and / or information from a lane keeping assist system.

[0006] Document DE 10 2015 206 209 A1 discloses a method and a device for determining a driver's attention. The method determines a numerical value representing the driver's attention. It comprises the steps of: recording the driver's gaze direction, determining the actual frequency distribution of the gaze direction, and determining the numerical value as a function of the deviation of the actual frequency distribution from a target frequency distribution of the gaze direction. BRIEF SUMMARY OF THE INVENTION

[0007] To achieve this goal, the present disclosure provides a method for monitoring the attention of a vehicle driver. The method includes determining the driver's gaze location. If the gaze location corresponds to a rear-facing mirror, the method determines the field of view of the rear-facing mirror. If the field of view corresponds to a view of the vehicle's driving environment, the method classifies the driver as performing a driving task. If the field of view corresponds to a view of the vehicle, the method classifies the driver as performing a non-driving task. Finally, the method issues an attention warning to the driver based on the driver's classification. The field of view corresponds to the view of the vehicle's driving environment if the field of view includes less of the vehicle's view than a vehicle vision threshold.

[0008] The present disclosure further provides a motor vehicle control unit. The motor vehicle control unit comprises at least one processing unit and a memory coupled to the at least one processing unit and designed to store machine-readable instructions. The machine-readable instructions cause the at least one processing unit to determine the viewing position of a driver of a vehicle. If the viewing position corresponds to a rear-facing mirror, the machine-readable instructions cause the at least one processing unit to determine the field of view of the rear-facing mirror. If the field of view corresponds to a view of the vehicle's driving environment, the machine-readable instructions cause the at least one processing unit to classify the driver as performing a driving task.If the field of view corresponds to a view of the vehicle, the machine-readable instructions cause the at least one processing unit to classify the driver as performing a non-driving task. Finally, the machine-readable instructions cause the at least one processing unit to issue an attention warning to the driver based on this classification. The field of view corresponds to the view of the vehicle's driving environment if it includes less of the vehicle view than a vehicle view threshold.

[0009] The present disclosure further provides a vehicle comprising the motor vehicle control unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Examples of the present disclosure are described with reference to the following attached drawings, in which the same reference symbols refer to the same elements. Fig. Figures 1A to 1C show a rearview mirror as an example of a rear-facing mirror with different fields of view according to examples in the present disclosure. Fig. 2A and Fig. 2B shows a side mirror as an example of a rear-facing mirror with different fields of view according to examples in the present disclosure. Fig. 3A and Fig. Figure 3B shows a flowchart of a method for monitoring the attention of a driver of a vehicle according to examples in the present disclosure. Fig. Figure 4 illustrates a vehicle designed to monitor the attention of the driver of the vehicle, according to examples in the present disclosure. Fig. Figure 5 illustrates a motor vehicle control unit according to examples in the present disclosure.

[0011] It should be understood that the drawings identified above are in no way intended to limit the present disclosure. Instead, these drawings are provided to aid in understanding the present disclosure. The person skilled in the art will readily understand that aspects of the present invention shown in one drawing may be combined with aspects shown in another drawing or omitted without altering the scope of protection of the present disclosure. DETAILED DESCRIPTION

[0012] The present disclosure generally provides a method, a motor vehicle control system, and a vehicle designed to monitor the attention of the vehicle's driver. For this purpose, two verifications are performed. First, the driver's viewing location is determined. If the viewing location corresponds to a rear-facing mirror, second, the field of view of the rear-facing mirror is determined. Depending on whether the field of view of the rear-facing mirror corresponds to the vehicle's driving environment, the driver is classified as either performing a driving task or not performing a driving task. That is, the second verification checks whether the rear-facing mirror is actually being used by the driver to monitor the driving environment or has instead been repurposed to monitor a passenger in a rear seat of the vehicle.If the rearview mirror has been repurposed to monitor a passenger in the back seat of the vehicle, the driver is classified as performing a non-driving task. Based on this classification, an attention warning, such as an audible or combined visual and audible warning, is emitted in the passenger compartment of the vehicle to encourage the driver to return to performing a driving task, and in particular to refocus their attention on the road ahead.

[0013] This general concept is explained with reference to the attached drawings, where Fig. 3A and Fig. 3B provide a flowchart of a procedure 200 for monitoring the attention of a driver of a vehicle and Fig. Figures 1A to 2B show various examples of rear-facing mirrors with different fields of view. Additionally, illustrations are provided. Fig. 4 a vehicle according to the present disclosure, and Fig. Figure 5 illustrates a motor vehicle control system designed to perform procedure 200.

[0014] It goes without saying that dashed boxes in Fig. Illustrating 1 optional step of procedure 100.

[0015] Procedure 200 is designed to monitor the attention of a driver of a vehicle, such as vehicle 300. Fig. 4.

[0016] Briefly, with reference to Fig. 2. In this disclosure, "vehicle 300" and, more generally, "vehicle" refer to any type of motor vehicle designed to transport persons and / or cargo. The engine of the vehicle 300 can be any type of engine, such as an electric motor or an internal combustion engine. The vehicle 300 can, for example, be a passenger car, as described in Fig. Figure 2 shows. However, it is understood that the vehicle 300 can also be a bus, a truck, or any other type of vehicle that includes one or more sensors 310 and a vehicle control unit 300, enabling the vehicle 300 to provide at least driver assistance. In other words, the vehicle control unit 300 and the one or more sensors 310 can be designed to enable the vehicle 300 to provide vehicle control functionality capable of at least driver assistance, i.e., Level 1 of the taxonomy of automated driving as defined in SAE International standard J3016. That is, the vehicle 300 can be designed to control the lateral and / or longitudinal movement of the vehicle 300 based on vehicle sensor data provided by the one or more sensors 310, under the supervision of a driver of the vehicle 300.

[0017] It is understood that the vehicle 300 can be designed to enable higher levels of driving automation, such as partial driving automation, i.e. level 2 or higher of the taxonomy of driving automation defined in the SAE International standard J3016.

[0018] The one or more sensors 310 are designed to acquire vehicle sensor data that indicates the vehicle 300's environment. Accordingly, the vehicle sensor data provides environmental awareness to the one or more vehicle control modules and thus to the vehicle 300, at least to enable driver assistance. For example, the vehicle sensor data acquired by the one or more sensors 310 can provide the vehicle 300 with information about the position and size of other vehicles, road surface markings, or traffic signs. For this purpose, the one or more sensors 310 can be radar sensors designed to emit radio waves to determine the distance, angle, and speed of objects in the vehicle's environment based on the reflected radio waves.The one or more sensors 310 can be LiDAR sensors (LIDAR: light detection and distance measurement) designed to emit laser beams to determine the distance, angle, and speed of objects in the vicinity of the vehicle 300 based on the reflected laser beams. The one or more sensors 310 can be cameras that capture images of the vehicle's surroundings. The one or more sensors 310 can be thermal imaging cameras that capture images of the vehicle's surroundings 300 based on infrared radiation. It is understood that LiDAR sensors, radar sensors, or cameras are provided only as examples of sensor types for the one or more sensors 310. For example, the one or more sensors 310 could also be ultrasonic sensors.The one or more sensors 310 can be sensors of a global navigation satellite system (GNSS) designed to receive position data, such as satellite signals, to determine the position of the vehicle 200. More generally, the one or more sensors 310 can be any type of sensor capable of acquiring motor vehicle sensor data that indicates the environment of the vehicle 200. It is further understood that the one or more sensors 310 can include several sensors of different types. Furthermore, the one or more sensors 310 of the same type can have different characteristics, for example, by being designed to acquire sensor data at different ranges, such as short range, medium range, and long range.For example, the vehicle 300 may include three short-range radar sensors, one each at the front and rear of the vehicle 300, one medium- to long-range radar sensor at the rear of the vehicle 200, one LiDAR sensor at the front of the vehicle 300, one rear-facing camera at the rear of the vehicle 200, one forward-facing camera at the front of the vehicle, one forward-facing camera on the rearview mirror, and one rear-facing short- to medium-range radar sensor in each door-mounted exterior side mirror. It is understood that the vehicle 300 may have more or fewer vehicle sensors than in [reference to specific vehicle model]. Fig. 4 shown and discussed in the example above can include.

[0019] The vehicle 300 further comprises one or more interior sensors 330 designed to capture interior data indicating at least one aspect of the driver's behavior. The driver's behavior includes at least one gaze location and may include other behavioral elements indicating the driver's attention to the vehicle 300, such as an open eye or objects in the driver's hands. For this purpose, the one or more interior sensors 330 may be cameras capturing images of at least the driver, or they may be thermal imaging cameras capturing images of at least the driver based on infrared radiation, or they may be any other type of sensor designed to capture the interior data.It is understood that the interior data can be acquired by a single interior sensor 330 as well as by a multitude of interior sensors 330, which may be of the same or different sensor types, in order to acquire the driver's interior data from different angles and / or based on a combination of sensor types. The one or more interior sensors 330 can be integrated into an instrument cluster behind the steering wheel, positioned next to the rearview mirror, or located in any other location that enables the acquisition of interior data.

[0020] In the context of the present disclosure, the driver's attention refers to the driver being fully or at least mainly focused on performing driving tasks, i.e., tasks relating to the dynamic driving task (DDT) as defined in SAE International standard J3016.Accordingly, driving tasks include all of the operational and tactical real-time tasks required to operate a vehicle in road traffic, excluding strategic functions such as time-based trip planning and selection of destinations and waypoints, and including, without limitation, lateral vehicle motion control by means of steering, longitudinal vehicle motion control by means of acceleration and deceleration, monitoring of the driving environment by means of object and event detection, recognition, classification and reaction preparation, execution of object and event reactions, maneuver planning and increasing conspicuousness by means of lights, horns, signaling, gestures, etc.The subtasks of monitoring the driving environment by means of object and event detection, recognition, classification, and response preparation, as well as object and event response execution, are collectively referred to as object and event detection and response (OEDR). Accordingly, OEDR, as defined in SAE International standard J3016, is to be understood as monitoring the driving environment and executing an appropriate response in reaction to such objects and events as part of the driving behavior task (DDT). Although the driver's attention in the context of this disclosure refers to the driver being fully or at least primarily focused on performing driving tasks, this disclosure places particular emphasis on whether the driver monitors the driving environment while looking in rearview mirrors.Accordingly, in the context of the present disclosure, the driver's attention refers to whether the driver is performing driving tasks that correspond to the OEDR sub-tasks of the DDT.

[0021] In step 210, the method 200 determines a gaze location of a driver of the vehicle 300. For this purpose, the method 200 can use the interior sensors 330 and, more precisely, the interior data acquired by the interior sensors 330. For example, the interior data can specify one or more head position angles, such as yaw, pitch, and roll, of the driver's head, based on which the method 200, as part of step 210, can derive the gaze location using the known location of the one or more interior sensors 330. In addition to or instead of the head position angles, the interior data can specify the position of the driver's eyes to derive the gaze location. For example, if both the position of the eyes and the head position angles are used to derive the gaze location, the method 200 can determine a gaze vector in a coordinate system in step 210, which, for example,The gaze location can be centered on the driver's head or on a fixed position within the passenger compartment of the vehicle 300. If more than one interior sensor 330 is used, the gaze location can also be determined based on triangulation, i.e., based on the known positions of the various interior sensors 330 and the gaze vectors determined by each interior sensor 330. It is understood that these examples for determining the gaze location are provided merely as examples. The gaze location can be determined in any way suitable for monitoring the driver's attention to the one or more interior sensors 330 used and the data acquired by the one or more sensors 330.

[0022] Method 200 then proceeds in step 211 to determine whether the viewing location corresponds to a rear-facing mirror 100. In the context of this disclosure, the rear-facing mirror refers to any type of mirror that enables the driver of the vehicle 300 to monitor the driving environment beside and behind the vehicle 300, such as the side mirrors and the rear-view mirror of the vehicle 300. It is thus understood that the vehicle 300 may typically include at least three rear-facing mirrors 100, and any reference to the rear-facing mirror 100 is equally to be understood as including any subset or all of the rear-facing mirrors 100 of the vehicle 300. As discussed with reference to step 210, method 200 may determine in step 211 that the viewing location corresponds to one of the rear-facing mirrors 100, e.g.,based on the viewing vector in the coordinate system centered at a fixed location in the passenger compartment of the vehicle 300, and the known position of one of the rear-facing mirrors 100 in the coordinate system, or based on deducing from the head position angles that the driver is most likely looking at one of the rear-facing mirrors.

[0023] If the procedure 200 determines in step 211 that the viewing position corresponds to the rear-facing mirror 100, the procedure 100 can proceed to steps 230a to 252, as described in Fig. Figure 3A illustrates this. In some examples of the present disclosure, after determining that the viewing location corresponds to the rear-facing mirror 100, the method 200 may first proceed to step 220. In step 220, the method 200 may detect an occupancy state of each of a plurality of rear seats 120 of the vehicle 300. Each occupancy state may indicate an occupancy of the corresponding rear seat 120. That is, the method 200 may detect whether any of the rear seats 120 is occupied by a passenger. If none of the rear seats 120 is occupied by a passenger, the method 200 may assume that there is no reason for the driver of the vehicle 300 to adjust any of the rear-facing mirrors 100 of the vehicle 300. Accordingly, the method 200 may proceed directly to step 251 to classify the driver as performing a driving task.In other words, procedure 200 can omit step 240 in particular if no reconfiguration of one or more rear-facing mirrors is expected due to the absence of passengers in the rear seats 120 of vehicle 300.

[0024] Depending on the implementation of Procedure 200, Procedure 200 can proceed to step 230a to determine a gaze duration corresponding to a time interval during which the gaze location corresponds to one of the rear-facing mirrors 100, following the determination that the gaze location corresponds to one of the rear-facing mirrors in step 211, or following the detection that none of the rear seats 120 are occupied in step 221. In addition to or instead of step 230a, Procedure 200 can also perform step 230b, in which Procedure 200 can determine a plurality of gaze durations during a gaze accumulation period. In other words, Procedure 200 can determine the duration of a single gaze into one of the rear-facing mirrors 100 or the accumulated duration of multiple gazes into one of the rear-facing mirrors 100 over a predetermined period, i.e., the gaze accumulation period.Since the driver of the vehicle 300 is not paying attention to the road ahead while looking in one of the rear-facing mirrors 100, a driver who looks in one of the rear-facing mirrors 100 for an extended period of time may be considered inattentive, even if they are monitoring the vehicle 300's surroundings using one or more rear-facing mirrors 100. Accordingly, an attention warning can also be issued to the driver as part of step 260 based on the gaze duration and / or accumulated gaze durations, as discussed below.

[0025] It is understood that both steps 230a and 230b can be performed simultaneously to determine whether individual gaze durations, as well as an accumulated gaze duration as the sum of the multitude of gaze durations, can lead to the driver being considered inattentive. It is further understood that steps 230a and 230b can also be performed following step 240, for example, if the driver is classified as performing a driving task in step 251, since otherwise procedure 200 might not issue the attention warning to the driver as part of step 260 regarding the classification in step 251. It is further understood that steps 230a and 230b can be performed following steps 220 and 221, i.e.,The viewing duration and / or accumulated viewing durations can be determined following the determination that none of the rear seats 120 are occupied, since the viewing duration and / or accumulated viewing durations can be used in step 260, even if the procedure 200 omits steps 240 to 242 as a result of the lack of detection of occupants in the rear seats 120 in step 221.

[0026] In step 240, the method 200 determines a field of view of the rear-facing mirror 100, i.e., an area observable in the rear-facing mirror 100 from the driver's point of view of the vehicle 300. The method 300 can determine the field of view of the rear-facing mirror 100, for example, based on sensors of the vehicle 100, such as one of the interior sensors 330 if the rear-facing mirror 100 is a rearview mirror, or one of the vehicle sensors 310 if the rear-facing mirror 100 is a side mirror. The method 200 can also determine the field of view based on an angle at which the rear-facing mirror 100 is positioned relative to a mounting surface of the rear-facing mirror 100, for example, based on known fields of view with the angle or based on calculating an expected field of view based on the angle.

[0027] Based on the field of view determined in step 240, the procedure 200 determines in step 241 whether the field of view corresponds to a view of the vehicle 300's driving environment. The field of view corresponds to the view of the vehicle 300's driving environment if the field of view includes less of a view of the vehicle 300 than a vehicle view threshold. More precisely, the field of view of the rear-facing mirror 100 always includes at least some elements of the vehicle 300 that constitute the view of the vehicle 300, in addition to the view of the driving environment.

[0028] The concept of the field of view of the rear-facing mirror 100 is in Fig. Figure 1A illustrates the field of view of the rear-facing mirror 100, which is implemented as a rearview mirror. The in Fig. The field of view shown in Figure 1A includes elements of the vehicle 300 in the form of part of a vehicle ceiling 140, part of the front seat headrests 110, part of the rear seats 120, and the rear seat headrests 130. The driving environment in Fig. 1A is visible through the rear window behind the rear seat headrests 130. Fig. Figure 1A illustrates a typical field of view of a rearview mirror, i.e., the field of view used by the driver of vehicle 300 to monitor the driving environment behind vehicle 300. Based on this typical field of view, the vehicle view threshold in the context of the present disclosure can be defined as the percentage of vehicle elements visible in the typical field of view, taking into account, for example, the geometry of the vehicle roof 140, the front seat headrests 110, the rear seats 120, the rear seat headrests 130, and the rear window of vehicle 300, as well as any other vehicle elements that may be visible in the field of view of the rear-facing mirror 100.

[0029] Accordingly, if the percentage of vehicle elements in the field of view of the rear-facing mirror 100 exceeds the vehicle view threshold, as determined based on the typical field of view of the rear-facing mirror 100, the field of view of the rear-facing mirror 100 is considered to correspond to a vehicle view, as in Fig. Shown in 1B. Fig. 1B is the driving environment above the rear seats 120 in the field of vision of the exemplary rearview mirror of Fig. 1B barely visible, and the percentage of vehicle elements is greatly increased when the driver uses the rearview mirror from Fig. 1B has been repurposed to allow monitoring of a child in one of the rear seats. Accordingly, the field of view of the rearview mirror corresponds to... Fig. 1B of the view of vehicle 300.

[0030] The concept of viewing the driving environment in comparison to viewing the vehicle 300 is further illustrated with an exemplary side mirror in Fig. 2A and Fig. 2B illustrates this. Fig. Figure 2A shows the exemplary side mirror with a typical field of view of the side mirror, which includes door handles 170, parts of the doors and the side windows of the vehicle 300, as well as the driving environment next to the vehicle 300. The percentage of the side mirror's field of view of Fig. 2A, which includes these vehicle elements of vehicle 300 in the typical field of vision of the side mirror of Fig. 2A includes, defines the vehicle view threshold for the side mirror of Fig. 2A. In contrast, it shows Fig. 2B the field of vision of the side mirror from Fig. 2A, after the side mirror was repurposed to monitor a child in one of the rear seats (120). As in Fig. 2B can be seen, more elements of the vehicle 300 are at least partially within the field of vision of the side mirror. Fig. 2B, such as the rear seat headrest 130, as being in the field of vision, i.e., the typical field of vision, of the side mirror of Fig. 2A visible. Therefore, the percentage of the field of vision corresponding to the vehicle's view is 300 in Fig. 2B compared to Fig. 2A greatly increased, and the field of vision of the side mirror from Fig. 2B is therefore considered to correspond to the view of vehicle 300 rather than the view of the driving environment.

[0031] In summary, the procedure 200 in step 241 can determine whether the field of view corresponds to the view of the driving environment or the view of the vehicle 300, based on the vehicle view threshold, which is the percentage of the typical field of view of the rear-facing mirror 100 that includes elements of the vehicle 300. The typical field of view, such as in Fig. 1A for the exemplary rearview mirror and in Fig. Figure 2A, illustrating the example side mirror, corresponds to the typical field of view of the rear-facing mirror 100, which enables monitoring of the driving environment behind and / or beside the vehicle 300. The typical field of view and the corresponding vehicle viewing threshold can be determined by a vehicle manufacturer based on an average adjustment of the rear-facing mirror 100 and potentially one or more standard deviations of the average adjustment of the rear-facing mirror 100. The typical field of view and the corresponding vehicle viewing threshold can also be determined for the driver of the vehicle 300 after the initial setup of the vehicle 300 or each time the vehicle 300 detects an unknown driver of the vehicle 300 based on the interior data from one of the interior sensors 330.

[0032] If the driver's field of vision corresponds to the view of the vehicle's driving environment (300), procedure 200 can proceed to step 251 to classify the driver as performing the driving task. If the driver's field of vision corresponds to the view of the vehicle (300), procedure 200 can proceed to step 252 to classify the driver as performing a non-driving task, i.e., any type of task not related to any subtask of the DDT.

[0033] If the field of view corresponds to the view of the vehicle's driving environment 300, the procedure 200 can perform step 242 before proceeding from step 241 to step 251. In step 242, the procedure 200 can detect an auxiliary mirror in the field of view of the rear-facing mirror 100. The auxiliary mirror can have an auxiliary field of view that includes at least one rear seat 120. That is, in step 242, the procedure 200 can check whether the driver 200 has repurposed the rear-facing mirror 100 to monitor rear seat occupants 120. This concept is described in Fig. 1C shows the same rearview mirror field of view as Fig. 1A shows, but with an auxiliary mirror 150, which is attached to one of the rear seat headrests 130 by means of auxiliary mirror mounting means 150F. As in Fig. 1C, the auxiliary field of view of the auxiliary mirror 150 includes a small child in a child car seat attached to one of the rear seats 120. Although the field of view of the rearview mirror of Fig. 1C the field of vision of the rearview mirror of Fig. 1A corresponds to the typical field of view of the rearview mirror, as discussed above; therefore, the driver of vehicle 300 nevertheless has the rearview mirror of Fig. 1C is repurposed to monitor the infant. Thus, procedure 200 in step 242 can verify whether the driver has repurposed the rear-facing mirror 100 without altering the field of view from the typical field of view. Consequently, as a consequence of the detection of the auxiliary mirror 150 in step 242, procedure 200 can proceed to step 252 to classify the driver as performing the non-driving task, even though the field of view corresponds to the typical field of view and thus to the view of the vehicle's driving environment 300, as determined in step 241.

[0034] After the driver has been classified as either performing the driving task or the non-driving task in steps 251 and 252, procedure 200 proceeds to step 260, where procedure 200 issues an attention warning to the driver based on the driver's classification. The attention warning can be any type of signal that at least alerts the driver to return to performing the driving task. Accordingly, step 260 can include a step 261, where procedure 200 issues the attention warning to the driver if the driver is classified as performing the non-driving task.

[0035] The attention warning can be any type of optical and / or visual warning emitted in the passenger compartment of the vehicle 300, such as a sound, a voice message instructing the driver to pay attention to the DDT / OEDR subtask, a flashing symbol in the instrument cluster of the vehicle 300 and / or a head-up display of the vehicle 300, or any other type of warning that causes the driver to return to performing the driving task.

[0036] In addition to being a signal that at least warns the driver to return to performing the driving task, the attention warning can further warn the driver to return to monitoring the driving environment in front of the vehicle. For this purpose, issuing the attention warning to the driver in step 260 can also be based on the gaze duration, and more specifically on the gaze duration and / or the accumulated gaze duration. In other words, step 260, based on the gaze duration and / or the accumulated gaze duration determined in steps 230a and 230b, can include one or both of steps 262 and 263.

[0037] In step 262, as part of step 260, the procedure 200 can issue the attention warning to the driver if the driver is classified as performing the driving task and if the gaze duration exceeds a gaze duration threshold, i.e., a period after which the driver's attention should return to monitoring the road in front of the vehicle 300.

[0038] In step 263, procedure 200, as part of step 260, can issue the attention warning to the driver if the driver is classified as performing the driving task and if the sum, i.e., the accumulated gaze duration of the multitude of gaze durations during the gaze accumulation period, exceeds a gaze accumulation threshold. In other words, procedure 200 can issue the attention warning to the driver if the driver looks in the rearview mirror for short periods that, individually, are not considered excessive interruptions of monitoring the road ahead, but which, taken together over the accumulation period, constitute a combined excessive interruption of monitoring the road ahead.

[0039] Procedure 200 can further issue different types of attention warnings as part of step 260 based on different levels of driver distraction, as determined based on the driver's classification and gaze duration and / or accumulated gaze durations. The different types of attention warnings may be similar to the attention warning discussed above, but may vary in intensity depending on the level of the different types of attention warning.

[0040] To this end, step 260 may include a step 264 in which procedure 200 may issue a basic attention warning if one of a set of basic attention warning conditions is met. The set of basic attention warning conditions may include at least a first basic warning condition and a second basic warning condition. The first basic warning condition includes the driver being classified as performing the driving task and the gaze duration exceeding the gaze duration threshold and / or the sum of the set of gaze durations during the gaze accumulation period exceeding the gaze accumulation threshold. The second basic warning condition may include the driver being classified as not performing the driving task.In other words, the basic attention warning can correspond to the attention warning as issued in steps 261 and 262, i.e., the basic attention warning can be issued if the driver is classified as not performing the driving task or if the driver is classified as performing the driving task but spends too much time monitoring the driving environment behind and / or beside the vehicle 300 using the rear-facing mirror 100.

[0041] In addition to step 264, step 260 may further include a step 265 in which procedure 200 issues an enhanced attention warning if one of a set of enhanced attention warning conditions is met. The set of enhanced attention warning conditions may include at least one initial enhanced attention warning condition. The initial enhanced attention warning condition may include the driver being classified as performing the non-driving task and the gaze duration exceeding the gaze duration threshold, or the sum of the set of gaze durations during the gaze accumulation period exceeding the gaze accumulation threshold. In other words, procedure 200 may issue the enhanced attention warning if the driver is classified as performing the non-driving task and fails to monitor the road ahead of vehicle 300 for extended periods.The increased attention warning may be justified in such a situation, since it is already assumed that the driver is distracted due to the repurposing of the rear-facing mirror and furthermore does not pay attention to the road ahead for at least the period defined by the gaze duration threshold and / or the gaze accumulation threshold.

[0042] It is understood that the multitude of basic warning conditions and the multitude of heightened attention warning conditions may include further conditions than those discussed above with reference to steps 264 and 265.

[0043] In summary, the procedure 200 determines whether the driver of vehicle 300 has repurposed the rear-facing mirror 100 and accordingly classifies the driver as performing the driving task (i.e., a task related to the DDT / OEDR subtask) or the non-driving task (i.e., any task not related to the DDT / OEDR subtask) when looking in the rear-facing mirror 100. Additionally, the procedure 300 can monitor the duration of individual glances as well as the accumulated duration of multiple glances over a predefined period to determine whether the driver should return to monitor the driving environment in front of vehicle 300, even if performing the driving task, and can further increase the intensity of the warning issued to the driver not performing the driving task.

[0044] Fig.Figure 5 shows a motor vehicle control unit 400 designed to perform the procedure 200. The motor vehicle control unit 400 can include a processor 410, a graphics processing unit (GPU) 420, a motor vehicle processing system 430, a memory 440, a removable storage 450, a storage 460, a cellular interface 470, a global navigation satellite system (GNSS) interface 480, and a communications interface 490.

[0045] The Processor 410 can be any type of single-core or multi-core processing unit that uses a RISC (Reduced Instruction Set) or CISC (Complex Instruction Set). Example RISC processing units include ARM-based cores or RISC-V-based cores. Example CISC processing units include x86-based cores or x86-64-based cores. The Processor 410 can execute instructions that cause the Automotive Control Unit 400 to perform the Procedure 200. The Processor 410 can be directly coupled to any of the components of the Automotive Control Unit 400, or it can be directly coupled to the Memory 430, the GPU 420, and a device bus.

[0046] The GPU 420 can be any type of processing unit optimized for processing graphics with respect to instructions or, more generally, for parallel processing of instructions. Accordingly, the GPU 420 can be designed to generate a display of information, such as FAS information or telemetry data, for a vehicle driver, for example, via a head-up display (HUD) or a display located within the driver's field of vision. The GPU 420 can be coupled to the HUD and / or display via a connection 420C. Furthermore, the GPU 420 can perform at least part of the process 100 to enable fast parallel processing of instructions relating to the process 100. It should be noted that in some embodiments, the processor 410 can determine that the GPU 420 does not need to perform the instructions relating to the process 200.The GPU 420 can be directly coupled to any of the components of the vehicle control unit 400, or it can be directly coupled to the processor 410 and the memory 430. In some embodiments, the GPU 420 can also be coupled to the device bus.

[0047] The vehicle processing system 430 can be any type of system-on-chip designed to provide trillions of operations per second (TOPS) to enable the vehicle control unit 400 to implement one or more ADAS while driving. The vehicle processing system 430 can be linked only to the processor 410 or it can be linked to other devices via the system bus. For example, the vehicle processing system 430 can execute instructions for the one or more vehicle sensor data processing modules and the one or more vehicle control modules.

[0048] Memory 440 can be any type of fast storage that allows the Processor 410, GPU 420, and Automotive Processing System 430 to store instructions for fast retrieval during instruction processing, as well as to cache and buffer data. Memory 440 can be a unified memory coupled with the Processor 410, GPU 420, and Automotive Processing System 430 to allow Memory 440 to be allocated to these components as needed. Alternatively, the Processor 410, GPU 420, and Automotive Processing System 430 can be coupled with separate Processor Memory 440a, GPU Memory 440b, and Automotive Processing System Memory 440c.

[0049] Removable storage 450 can be a storage device that can be detachably coupled to the vehicle control unit 400. Examples include a DVD (Digital Versatile Disc), a CD (Compact Disc), a USB (Universal Serial Bus) storage device such as an external SSD, or magnetic tape. It should be noted that removable storage 450 can store data such as instructions for procedure 200, vehicle sensor data, intermediate data, and / or vehicle control data, or it can be omitted.

[0050] Storage 460 can be a storage device that enables the storage of program instructions and other data. For example, storage 460 can be a hard disk drive (HDD), a solid-state drive (SSD), or another type of non-volatile storage. Storage 460 can, for example, store the instructions of procedure 100, vehicle sensor data, intermediate data, and / or vehicle control data.

[0051] The removable storage device 450 and the storage device 460 can be connected to the processor 410 via the system bus. The system bus can be any type of system bus that allows the processor 410, and optionally the GPU 420 and the automotive processing system 430, to communicate with the other devices of the automotive control unit 400. The bus 440 can, for example, be a PCIe bus (Peripheral Component Interconnect Express) or a SATA bus (Serial AT Attachment).

[0052] The mobile communication interface 470 can be any type of interface that enables the vehicle control unit 400 to communicate via a mobile communication network, such as a 4G network or a 5G network.

[0053] The GNSS interface 480 can be any type of interface that enables the vehicle control unit 300 to receive position data provided by a satellite network, such as the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), or Galileo. The position data can be any type of vehicle sensor data in the context of this disclosure.

[0054] The communication interface 490 enables the vehicle control unit 400 to connect to external devices either directly or via a network. For example, the communication interface 480 allows the vehicle control unit 300 to be connected to a wired or wireless network, such as Ethernet, WiFi, a CAN bus (Controller Area Network), or any bus system suitable for use in vehicles. For instance, the vehicle control unit 400 can be connected to one or more vehicle sensors 310 to receive vehicle sensor data.

[0055] The vehicle control unit 400 can be integrated with the vehicle 300, e.g. under the passenger compartment, under the dashboard or in the trunk of the vehicle 300.

[0056] The invention can further be illustrated by the following examples.

[0057] In one example, a procedure for monitoring the attention of a vehicle driver includes determining the driver's gaze location, and if the gaze location corresponds to a rear-facing mirror, determining the field of view of the rear-facing mirror if the field of view corresponds to a view of the vehicle's driving environment, classifying the driver as performing a driving task if the field of view corresponds to a view of the vehicle, classifying the driver as performing a non-driving task, and issuing an attention warning to the driver based on the driver's classification.

[0058] In the exemplary procedure, the field of view can correspond to the view of the vehicle's driving environment if the field of view includes less of the view of the vehicle than a vehicle view threshold.

[0059] The exemplary procedure may further include, if the viewing location corresponds to the rear-facing mirror, detecting an occupancy state of each of a plurality of rear seats of the vehicle, each occupancy state indicating an occupancy of each corresponding rear seat, and classifying the driver as performing the driving task if each occupancy state indicates that none of the rear seats of the plurality of rear seats is occupied.

[0060] In the exemplary procedure, the exemplary procedure can dispense with determining the field of view if each occupancy state indicates that none of the rear seats of the multitude of rear seats are occupied.

[0061] In the exemplary procedure, determining the field of view of the rear-facing mirror may include detecting an auxiliary mirror within the field of view of the rear-facing mirror, wherein the auxiliary mirror has an auxiliary field of view that includes at least one rear seat, and the exemplary procedure may further classify the driver as the non-driving task, even if the field of view corresponds to the view of the vehicle's driving environment, if the auxiliary mirror is detected.

[0062] In the exemplary procedure, issuing the attention warning to the driver may include issuing the attention warning to the driver if the driver is classified as performing the non-driving task.

[0063] The exemplary procedure may further include, if the viewing location corresponds to a rear-facing mirror, determining a viewing duration, wherein the viewing duration corresponds to a time interval during which the viewing location can correspond to the rear-facing mirror, and wherein the output of the attention warning to the driver may further be based on the viewing duration.

[0064] In the exemplary procedure, issuing the attention warning to the driver may also include issuing the attention warning to the driver if the driver is classified as performing the driving task and if the gaze duration exceeds a gaze duration threshold.

[0065] The exemplary procedure may further include, if the gaze location corresponds to a rear-facing mirror, determining a plurality of gaze durations during a gaze accumulation period, wherein issuing the attention warning to the driver further includes issuing (263) the attention warning to the driver if the driver is classified as performing the driving task and if a sum of the plurality of gaze durations during the gaze accumulation period exceeds a gaze accumulation threshold.

[0066] In the exemplary procedure, issuing the attention warning to the driver may further include issuing a basic attention warning if one of a plurality of basic attention warning conditions is met, wherein the plurality of basic attention warning conditions comprises at least a first basic warning condition and a second basic warning condition, wherein the first basic warning condition comprises that the driver is classified as performing the driving task and the gaze duration exceeds the gaze duration threshold or the sum of the plurality of gaze durations during the gaze accumulation period exceeds the gaze accumulation threshold, and the second basic warning condition comprises that the driver is classified as not performing the driving task, and issuing an enhanced attention warning if one of a plurality of enhanced attention warning conditions is met.wherein the multitude of enhanced attention warning conditions includes at least one initial enhanced attention warning condition, wherein the initial enhanced attention warning condition includes the driver being classified as performing the non-driving task and the gaze duration exceeds the gaze duration threshold or the sum of the multitude of gaze durations during the gaze accumulation period exceeds the gaze accumulation threshold.

[0067] An exemplary motor vehicle control unit comprises at least one processing unit and a memory coupled to the at least one processing unit and designed to store machine-readable instructions, wherein the machine-readable instructions cause the at least one processing unit to determine a driver's viewing location, and if the viewing location corresponds to a rear-facing mirror, to determine a field of view of the rear-facing mirror if the field of view corresponds to a view of the vehicle's driving environment, to classify the driver as performing a driving task if the field of view corresponds to a view of the vehicle, to classify the driver as performing a non-driving task, and to issue an attention warning to the driver based on the driver's classification.

[0068] In the exemplary motor vehicle control unit, the machine-readable instructions can also cause the at least one processing unit to carry out any of the exemplary procedures discussed above.

[0069] An example vehicle includes any of the example motor vehicle control units discussed above.

[0070] The foregoing description has been provided to illustrate the monitoring of a driver's attention. It should be understood that the description is in no way intended to limit the scope of protection of the present disclosure to the precise embodiments discussed throughout the description. Instead, those skilled in the art will be aware that the examples of the present disclosure can be combined, modified, or summarized without departing from the scope of protection of the present disclosure, as defined by the following claims. Reference symbol list 100 rear-facing mirror 110 Front seat headrest 120 rear seats 130 Rear seat headrest 140 vehicle blanket 150 auxiliary mirrors 150F Auxiliary mirror mounting hardware 160 front seat 170 door handle 200 procedures 210-265 procedural steps 300 vehicles 310 Automotive sensor 320 light 330 Interior sensor 400 Motor vehicle control unit 410 CPU 420 GPU 420c connection 430 Automotive processing system 440 storage 450 removable storage 460 storage 470 Mobile communication interface 480 GNSS interface 490 Communication interface

Claims

Method (200) for monitoring the attention of a driver of a vehicle (300), comprising: determining (210) a gaze location of a driver of the vehicle (300); if the gaze location corresponds to a rear-facing mirror (100) (211): determining (240) a field of view of the rear-facing mirror (100); if the field of view corresponds to a view of a driving environment of the vehicle (300) (241), classifying (251) the driver as performing a driving task; if the field of view corresponds to a view of the vehicle (300) (241), classifying (252) the driver as performing a non-driving task; and issuing (260) an attention warning to the driver based on the classification of the driver, wherein the field of view corresponds to the view of the driving environment of the vehicle (300) (241), if the field of view includes less of the view of the vehicle (300) than a vehicle vision threshold. Method (200) according to claim 1, further comprising: if the viewing location corresponds to the rear-facing mirror (100) (211): detecting (220) an occupancy state of each of a plurality of rear seats of the vehicle (300), wherein each occupancy state indicates an occupancy of each corresponding rear seat (120); and classifying (251) the driver as performing the driving task if each occupancy state indicates that none of the rear seats (120) of the plurality of rear seats is occupied. Method (200) according to claim 2, wherein the method (200) dispenses with determining (240) the field of view if each occupancy state indicates that none of the rear seats (120) of the plurality of rear seats is occupied. Method (200) according to one of the preceding claims, wherein: determining (240) the field of view of the rear-facing mirror (100) comprises detecting (242) an auxiliary mirror (150) in the field of view of the rear-facing mirror (100), wherein the auxiliary mirror (150) has an auxiliary field of view that includes at least one rear seat (120); and the method (200) further comprises classifying (252) the driver as performing the non-driving task, even if the field of view corresponds to the view of the vehicle's driving environment, if the auxiliary mirror is detected. Method (200) according to one of the preceding claims, wherein issuing (260) the attention warning to the driver includes issuing (261) the attention warning to the driver if the driver is classified as performing the non-driving task. Method (200) according to one of the preceding claims, further comprising: if the viewing location corresponds to a rear-facing mirror (100), determining (230a) a viewing duration, wherein the viewing duration corresponds to a time interval during which the viewing location corresponds to the rear-facing mirror (100), wherein the output (260) of the attention warning to the driver is further based on the viewing duration. Method (200) according to claim 6, wherein the output (260) of the attention warning to the driver further includes the output (262) of the attention warning to the driver if the driver is classified as performing the driving task and if the gaze duration exceeds a gaze duration threshold. Method (200) according to one of claims 6 and 7, further comprising: if the viewing location corresponds to a rear-facing mirror, determining (230b) a plurality of viewing durations during a viewing accumulation period, wherein issuing (260) the attention warning to the driver further includes issuing (263) the attention warning to the driver if the driver is classified as performing the driving task and if a sum of the plurality of viewing durations during the viewing accumulation period exceeds a viewing accumulation threshold. Method (200) according to claims 6 to 8, wherein issuing (260) the attention warning to the driver further comprises: issuing (264) a basic attention warning if one of a plurality of basic attention warning conditions is met, wherein the plurality of basic attention warning conditions comprises at least a first basic warning condition and a second basic warning condition, wherein the first basic warning condition comprises that the driver is classified as performing the driving task and the gaze duration exceeds the gaze duration threshold or the sum of the plurality of gaze durations during the gaze accumulation period exceeds the gaze accumulation threshold, and the second basic warning condition comprises that the driver is classified as performing the non-driving task;and issue (265) an enhanced attention warning if one of a plurality of enhanced attention warning conditions is met, wherein the plurality of enhanced attention warning conditions includes at least one initial enhanced attention warning condition, wherein the initial enhanced attention warning condition includes that the driver is classified as performing the non-driving task and the gaze duration exceeds the gaze duration threshold or the sum of the plurality of gaze durations during the gaze accumulation period exceeds the gaze accumulation threshold. Motor vehicle control unit (400), comprising: at least one processing unit (410, 420, 430); and a memory (440, 450) coupled to the at least one processing unit (410, 420, 430) and designed to store machine-readable instructions, wherein the machine-readable instructions cause the at least one processing unit (410, 420, 430) to: determine a view location of a driver of a vehicle (300); if the view location corresponds to a rear-facing mirror (100): determine a field of view of the rear-facing mirror (100); if the field of view corresponds to a view of a driving environment of the vehicle (300), classify the driver as performing a driving task; if the field of view corresponds to a view of the vehicle (300), classify the driver as performing a non-driving task;and issuing an attention warning to the driver based on the driver's classification, where the field of view corresponds to the view of the vehicle's driving environment (300) (241), if the field of view includes less of the view of the vehicle (300) than a vehicle view threshold. Motor vehicle control unit (400) according to claim 10, wherein the machine-readable instructions further cause the at least one processing unit (410, 420, 430) to carry out the method according to any one of claims 2 to 9. Vehicle (300) comprising the motor vehicle control unit (410, 420, 430) according to one of claims 10 and 11.

Images