Driver state determination device

Abstract

A driver state determination device is provided, capable of determining the type and stage of a driver's abnormal state and providing appropriate driving assistance based on the driver's state. The driver state determination device includes an in-vehicle camera that detects the driver's gaze and the movement of the driver's head, and a controller configured to determine the driver's state based on the driver's gaze and head movement. When the controller determines that the driver is in an abnormal state based on the movement of the driver's gaze, it further determines whether the abnormal state is due to visual obstruction based on changes in the driver's head tilt and pitch angles. When the abnormal state is determined to be visual obstruction, if there is a direction that the driver cannot look in, the driver is deemed aware of the visual obstruction; otherwise, the driver is deemed unaware of the visual obstruction.

Description

Technical Field

[0001] This invention relates to a driver state determination device for determining the state of a driver of a vehicle. Background Technology

[0002] Previously, driver state detection devices for detecting abnormalities in vehicle drivers have been proposed. For example, a driver state estimation device has been proposed that, when it detects an expansion of the distribution of the driver's line of sight from the normal state, and detects a change in steering operation from the normal state, determines that there is a suspicion of a lack of driver visibility (see Patent Document 1).

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2018-198842

[0006] The problem that the invention aims to solve

[0007] The inventors' research revealed that in diseases like glaucoma where visual impairment gradually worsens, the driver's gaze and head movements change in accordance with the stage of the disease. However, in the prior art described in Patent Document 1, the corresponding changes in gaze and head movements with the stage of the disease were not considered. Therefore, for example, it was impossible to determine the stage immediately following the onset of disease symptoms and the stage at which the driver was aware of the disease. Consequently, it may be difficult to provide appropriate driving assistance corresponding to the driver's state. Summary of the Invention

[0008] The present invention is proposed to solve such problems, and aims to provide a driver state determination device that can determine the type and stage of a driver's abnormal state and provide appropriate driving assistance corresponding to the driver's state.

[0009] Technical means for solving problems

[0010] To address the aforementioned issues, the present invention provides a driver state determination device for determining the state of a driver operating a vehicle. The device comprises: a gaze detection device for detecting the driver's gaze; a head movement detection device for detecting the movement of the driver's head; and a controller configured to determine the driver's state based on the driver's gaze and head movement. The controller is configured to: determine whether the driver is in an abnormal state based on the movement of the driver's gaze; when the driver is determined to be in an abnormal state, determine whether the abnormal state is due to visual obstruction based on changes in the driver's head yaw and pitch angles; when the abnormal state is determined to be visual obstruction, if there is a direction the driver cannot look in, determine that the driver is aware of the visual obstruction; if there is no direction the driver cannot look in, determine that the driver is unaware of the visual obstruction.

[0011] According to the present invention configured in this way, when the controller determines that the driver is in an abnormal state based on the movement of the driver's gaze, it determines whether the abnormal state is due to attention impairment or visual impairment based on changes in the driver's head tilt and pitch angles. Therefore, it is possible to distinguish the type of abnormal state by using the different head movements depending on whether the abnormal state is attention impairment or visual impairment. Furthermore, when the abnormal state is determined to be visual impairment, if there is a direction that the driver cannot look in, the controller determines that the driver is aware of the visual impairment; if there is no direction that the driver cannot look in, the controller determines that the driver is unaware of the visual impairment. Therefore, it is possible to determine whether the driver is unaware of the visual impairment immediately after the onset of symptoms of a visual impairment disease such as glaucoma, or whether the driver is aware of the visual impairment after a period of time since the onset of symptoms of the visual impairment disease. Thus, it is possible to determine the type and stage of the driver's abnormal state and to provide appropriate driving assistance corresponding to the driver's state.

[0012] In this invention, preferably, the controller is configured such that when the driver is determined to be in an abnormal state, if the amplitude of the yaw angle and pitch angle of the driver's head is greater than or equal to a first threshold, the abnormal state of the driver is determined to be visual impairment, and if the amplitude of the yaw angle and pitch angle of the driver's head is less than the first threshold, the abnormal state of the driver is determined to be attention impairment.

[0013] According to the present invention configured in this way, the abnormal state of the driver is visual impairment. When the driver tends to move their head up, down, left, and right to compensate for a conscious or unconscious lack of vision, the abnormal state can be determined to be visual impairment based on the movement of their head. In other cases, the abnormal state can be appropriately identified as attention impairment. Therefore, the type of abnormal state of the driver can be determined, and appropriate driving assistance corresponding to the driver's state can be provided.

[0014] In this invention, preferably, the controller is configured to: when it is determined that the driver is not in an abnormal state, and in the past when the abnormal state of the driver was determined to be visual impairment, determine that the driver has understood the compensatory action for visual impairment.

[0015] According to the present invention configured in this way, when the driver is in a normal state, it is possible to appropriately determine whether the driver has compensated for visual obstruction. Therefore, it is possible to determine the stage of the driver's abnormal state and to provide appropriate driving assistance corresponding to the driver's state.

[0016] In this invention, preferably, the controller is configured to: detect the frequency of the driver's saccades based on the movement of the driver's gaze, and determine that the driver is in an abnormal state if the frequency of the saccades is below a second threshold.

[0017] According to the present invention configured in this way, the controller determines whether the driver is in an abnormal state based on the frequency of scanning, and thus can accurately determine the driver's abnormal state.

[0018] In this invention, preferably, the driver state determination device further includes an information output device for outputting information to the driver, and the controller is configured to output information indicating that the driver has a visual obstruction when it is determined that the driver is unaware of the visual obstruction.

[0019] According to the present invention configured in this way, appropriate driving assistance can be provided to drivers who are unaware of visual obstruction.

[0020] Invention Effects

[0021] According to the driver state determination device of the present invention, it is possible to determine the type and stage of the driver's abnormal state and to provide appropriate driving assistance corresponding to the driver's state. Attached Figure Description

[0022] Figure 1 This is an explanatory diagram of a vehicle equipped with the driver status determination device according to an embodiment of the present invention.

[0023] Figure 2 This is a block diagram of a driver status determination device according to an embodiment of the present invention.

[0024] Figure 3 This is a flowchart of the driver state determination process according to an embodiment of the present invention.

[0025] Figure 4 This is a graph illustrating the scanning rate at different stages of visual field obstruction.

[0026] Figure 5 This is a diagram illustrating the distribution of gaze when someone is unaware of a visual impairment.

[0027] Figure 6 This is a diagram illustrating the distribution of gaze when one is aware of visual impairment.

[0028] Symbol Explanation

[0029] 1 vehicle

[0030] 10 Controllers

[0031] 100 Driver Status Determination Device

[0032] 21. External camera

[0033] 22 Radar

[0034] 23 Navigation System

[0035] 24 Positioning System

[0036] 25 Vehicle speed sensor

[0037] 26 Accelerometers

[0038] 27 Yaw rate sensor

[0039] 28. Steering Angle Sensor

[0040] 29. Steering torque sensor

[0041] 30 Throttle Sensor

[0042] 31 Brake sensor

[0043] 32 In-vehicle cameras

[0044] 36 monitors

[0045] 37 speakers Detailed Implementation

[0046] Hereinafter, the driver status determination device according to an embodiment of the present invention will be described with reference to the accompanying drawings.

[0047] [System Architecture]

[0048] First, refer to Figure 1 and Figure 2 The structure of the driver status determination device in this embodiment will be described. Figure 1 This is an explanatory diagram of a vehicle equipped with a driver status determination device. Figure 2 This is a block diagram of the driver status determination device.

[0049] The vehicle 1 of this embodiment includes: a driving force source 2 such as an engine or electric motor that outputs driving force, a gearbox 3 that transmits the driving force output from the driving force source 2 to the drive wheels, a brake 4 that applies braking force to the vehicle 1, and a steering device 5 for steering the vehicle 1.

[0050] The driver status determination device 100 is configured to determine the status of the driver of the vehicle 1 and perform vehicle 1 control and driver assistance control as needed. Figure 2 As shown, the driver status determination device 100 includes a controller 10, multiple sensor types, multiple control systems, and multiple information output devices.

[0051] Specifically, the multiple sensor classes include an external camera 21 for acquiring information about the driving environment of vehicle 1, a radar 22, a navigation system 23 for detecting the position of vehicle 1, and a positioning system 24. Furthermore, the multiple sensor classes include a vehicle speed sensor 25, an acceleration sensor 26, a yaw rate sensor 27, a steering angle sensor 28, a steering torque sensor 29, a throttle sensor 30, and a brake sensor 31 for detecting the trajectory of vehicle 1 and the driver's driving operations. Additionally, the multiple sensor classes include an in-vehicle camera 32 for detecting the driver's line of sight. The multiple control systems include a powertrain control module (PCM) 33 for controlling the drive source 2 and the transmission 3, a dynamic stability control system (DSC) 34 for controlling the drive source 2 and the brakes 4, and an electric power steering system (EPS) 35 for controlling the steering device 5. The multiple information output devices include a display 36 for outputting image information and a speaker 37 for outputting sound information.

[0052] Furthermore, other types of sensors may include: peripheral sonar for measuring the distance and position of surrounding structures relative to vehicle 1, corner radar for measuring the approach of surrounding structures at the corners of the four parts of vehicle 1, and various sensors for detecting the driver's state (e.g., pulse sensor, electrocardiogram sensor, steering wheel grip force sensor, etc.).

[0053] The controller 10 performs various calculations based on signals received from multiple sensor types, sending control signals to the PCM 33, DSC 34, and EPS 35 to ensure the proper operation of the drive power source 2, gearbox 3, brake 4, and steering device 5, and sending control signals to the display 36 and speaker 37 to output desired information. The controller 10 is composed of a computer, which has one or more processors 10a (typically a CPU), memory 10b (ROM, RAM, etc.) for storing various programs and data, input / output devices, etc.

[0054] The external camera 21 takes pictures of the area around the vehicle 1 and outputs image data. The controller 10 determines the objects (e.g., vehicles ahead, parked vehicles, pedestrians, roads, lane markings (lane boundary lines, white lines, yellow lines), traffic signals, traffic signs, stop lines, intersections, obstacles, etc.) based on the image data received from the external camera 21.

[0055] Radar 22 determines the position and speed of an object (especially a preceding vehicle, a parked vehicle, a pedestrian, or an object falling onto the road). For example, millimeter-wave radar can be used as radar 22. Radar 22 transmits radio waves in the direction of travel of vehicle 1 and receives reflected waves generated by the object reflecting the transmitted waves. Furthermore, radar 22 determines the distance between vehicle 1 and the object (e.g., vehicle-to-vehicle distance) or the relative speed of the object to vehicle 1 based on the transmitted and received waves. Alternatively, in this embodiment, lidar, ultrasonic sensors, or the like can be used instead of radar 22 to determine the distance and relative speed to the object. Moreover, multiple sensor types can be used to construct the position and speed measuring device.

[0056] The navigation system 23 stores map information internally and can provide map information to the controller 10. The controller 10 determines roads, intersections, traffic signals, buildings, etc., existing around the vehicle 1 (especially in the direction of travel) based on the map information and the current vehicle position information. The map information can also be stored within the controller 10. The positioning system 24 is a GPS system and / or a gyroscope system that detects the position of the vehicle 1 (current vehicle position information).

[0057] The vehicle speed sensor 25 detects the speed of vehicle 1, for example, based on the rotational speed of the wheels and drive shaft. The acceleration sensor 26 detects the acceleration of vehicle 1. This acceleration includes the acceleration in the longitudinal direction and the lateral acceleration (i.e., lateral acceleration) of vehicle 1. Furthermore, in this specification, the acceleration includes not only the rate of change of velocity in the direction of increasing velocity but also the rate of change of velocity in the direction of decreasing velocity (i.e., deceleration).

[0058] Yaw rate sensor 27 detects the yaw rate of vehicle 1. Steering angle sensor 28 detects the rotation angle (steering angle) of the steering wheel of steering device 5. Steering torque sensor 29 detects the torque applied to the steering shaft (steering torque) via the steering wheel. Throttle sensor 30 detects the amount of accelerator pedal depressed. Brake sensor 31 detects the amount of brake pedal depressed.

[0059] The in-vehicle camera 32 takes pictures of the driver and outputs image data. The controller 10 detects the driver's gaze direction and head movement (e.g., head yaw and pitch angles) based on the image data received from the in-vehicle camera 32. Furthermore, the in-vehicle camera 32 is an example of the "gaze detection device" and "head movement detection device" in this invention.

[0060] PCM33 controls the drive power source 2 of vehicle 1 and adjusts the drive force of vehicle 1. For example, PCM33 controls the spark plugs of the engine, fuel injection valve, throttle valve, variable valve mechanism, or inverter that supplies power to the electric motor, etc. When it is necessary to accelerate or decelerate vehicle 1, controller 10 sends a control signal to PCM33 to adjust the drive force.

[0061] The DSC34 controls the drive power source 2 and brake 4 of vehicle 1 to perform deceleration and attitude control of vehicle 1. For example, the DSC34 controls the hydraulic pump and valve unit of brake 4, and controls the drive power source 2 with the help of PCM33. When it is necessary to perform deceleration and attitude control of vehicle 1, the controller 10 sends a control signal to the DSC34 to adjust the drive force or generate braking force.

[0062] EPS35 controls the steering device 5 of vehicle 1. For example, EPS35 controls an electric motor that imparts torque to the steering shaft of steering device 5. When it is necessary to change the direction of travel of vehicle 1, controller 10 sends a control signal to EPS35 to change the steering direction.

[0063] Display 36 is positioned in front of the driver inside the vehicle cabin to display image information to the driver. Display 36 may be, for example, a liquid crystal display or a head-up display. Speaker 37 is positioned inside the vehicle cabin to output various sound information. Furthermore, display 36 and speaker 37 are examples of the "information output device" in this invention.

[0064] [Driver Status Determination and Processing]

[0065] Next, refer to Figure 3 The process of driver status determination processing of the driver status determination device 100 based on this embodiment will be described. Figure 3 This is a flowchart of the driver status determination process.

[0066] The driver status determination process begins when the vehicle 1 is powered on and is repeatedly executed according to a cycle specified by the controller 10 (e.g., every 0.05 to 0.2 seconds).

[0067] When the driver status determination process begins, firstly, the controller 10 acquires various information, including the driver's gaze and head movement, based on signals received from sensors including the external camera 21, radar 22, navigation system 23, positioning system 24, and internal camera 32 (step S1).

[0068] Next, the controller 10 detects the movement of the driver's gaze based on the signal received from the in-vehicle camera 32 in step S1 (step S2). Specifically, the controller 10 detects the driver's pupils from the image (image data) obtained by the in-vehicle camera 32, and detects the driver's gaze based on the detected pupils. Then, the controller 10 calculates the distance the driver's gaze has moved. Furthermore, the controller 10 calculates the speed of the driver's gaze based on the change in the distance the driver's gaze has moved over time. For example, the controller 10 calculates the speed of the driver's gaze by differentiating the distance the gaze has moved over time.

[0069] Next, the controller 10 determines, based on the driver's gaze movement detected in step S2, whether the frequency fs of the driver's saccades is below a threshold fth (pre-set and stored in memory 10b, corresponding to the "second threshold" of the present invention) (step S3). Specifically, the controller 10 extracts saccade candidates as candidates for saccades based on the gaze movement speed calculated in step S2. For example, the controller 10 extracts a period during which the gaze movement speed is less than a predetermined speed threshold (e.g., 40 deg / s) and the state continues for a predetermined pause time (e.g., 0.1 seconds) as a "gaze period". Furthermore, the controller 10 extracts gaze movements that are sandwiched between two adjacent gaze periods, with a movement speed of more than the speed threshold (e.g., 40 deg / s) and a movement distance of more than a predetermined distance threshold (e.g., 3 deg), as "saccade candidates". Furthermore, the controller 10 extracts saccade candidates after removing noise from the saccade candidates using known methods as saccades. Furthermore, at each predetermined period (e.g., every 10 seconds), the controller 10 calculates the value obtained by dividing the number of saccades included in its period by the time of its period as the "saccade frequency fs", and compares it with the threshold fth.

[0070] Figure 4 This is a graph illustrating the scanning rate at various stages of visual field obstruction. According to the researchers of this invention, as... Figure 4As shown, when the driver's visual field is impaired, the scan rate is below the threshold fth, regardless of whether the driver is aware of the visual impairment. On the other hand, when the driver is in a normal state, or when the driver, after experiencing visual impairment, understands the action of compensating for the visual deficit by restoring their vision, the scan rate is greater than the threshold fth. Moreover, although not in Figure 4 However, when the driver's state is one of visual impairment, the scan rate is also below the threshold fth. Therefore, by determining whether the scan rate is below the threshold fth, it is possible to determine whether the driver's state is abnormal (visual impairment or visual impairment), or normal, or a state in which the driver has compensated for visual impairment.

[0071] Therefore, the result of step S3 is that if the frequency fs of the driver's gaze is below the threshold fth (step S3: Yes), the controller 10 determines that the driver is in an abnormal state and detects the driver's head movement based on the signal received from the in-vehicle camera 32 in step S1 (step S4). Specifically, the controller 10 performs image recognition of the driver's head from the image (image data) obtained by the in-vehicle camera 32 and calculates the head yaw angle and pitch angle ap.

[0072] According to the researchers of this invention, the following insight has been obtained: by ensuring that the frequency fs of the driver's saccades is below a threshold fth, when the driver's state is determined to be abnormal, it is possible to determine whether the abnormal state is due to attentional impairment or visual impairment based on the movement of the driver's head. Specifically, when the abnormal state is due to attentional impairment, the movement of the driver's head will not be larger than in the normal state. On the other hand, when the abnormal state is due to visual impairment, the driver tends to move their head up, down, left, and right to compensate for the conscious or unconscious lack of vision. As a result, the movement of the driver's head is larger in the normal state than in the state of attentional impairment. Therefore, when the amplitude of the oscillation of the driver's head yaw angle and pitch angle ap is relatively large (e.g., above a predetermined threshold), the abnormal state of the driver can be determined to be visual impairment; when the amplitude of the oscillation of the driver's head yaw angle and pitch angle ap is relatively small (e.g., below the predetermined threshold), the abnormal state of the driver can be determined to be attentional impairment.

[0073] Therefore, based on the driver's head movement detected in step S4, the controller 10 determines whether the amplitudes of the driver's head yaw angle ay and pitch angle ap are above the threshold ath (pre-set and stored in memory 10b, corresponding to the "first threshold" of the present invention) (step S5). Specifically, the controller 10 calculates the average amplitudes of the driver's head yaw angle ay and pitch angle ap contained within its predetermined period (e.g., every 10 seconds), and uses them as the "amplitude of yaw angle ay" and "amplitude of pitch angle ap", respectively, and compares them with the threshold ath.

[0074] If the result of the determination in step S5 is that the amplitude of the driver's head yaw angle and pitch angle ap is above the threshold ath (step S5: Yes), the controller 10 determines that the driver's abnormal state is visual obstruction (step S6). At this time, the controller 10 stores the information indicating that the driver's abnormal state is visual obstruction in the memory 10b.

[0075] Next, the controller 10 detects the distribution of the driver's gaze direction based on the driver's gaze movement detected in step S2 (step S7). For example, the controller 10 determines the direction of the driver's gaze at predetermined time intervals in each predetermined cycle (e.g., every 10 seconds) and obtains the distribution of the gaze direction within its cycle.

[0076] Figure 5 and Figure 6 This is a diagram illustrating the distribution of lines of sight. Figure 5 Examples of line-of-sight distribution when someone is unaware of a visual impairment. Figure 6 This example illustrates the distribution of gaze when a person is aware of visual impairment. Figure 5 and Figure 6 The black dots indicate the direction of the driver's line of sight, determined at specified time intervals.

[0077] According to the researchers' findings, in the early stages of symptoms of glaucoma and other diseases accompanied by visual impairment, when the driver is unaware of the visual impairment, such as... Figure 5 As shown, the distribution of gaze within the visual field is unbiased, and there is no particular direction the gaze cannot be directed towards. On the other hand, given the passage of time since the onset of symptoms of the accompanying visual field impairment, and the driver's awareness of the visual field impairment, such as... Figure 6 As shown by the grid lines, corresponding to areas of visual impairment, there exists a region where the driver's line of sight is not distributed (i.e., the direction the line of sight cannot be directed towards). Therefore, by determining whether there is a region where the driver's line of sight is not distributed (i.e., the direction the line of sight cannot be directed towards), it is possible to determine whether the driver is aware of or unaware of the visual impairment.

[0078] Therefore, the controller 10 determines, based on the distribution of the driver's line of sight detected in step S7, whether there is an area where the driver's line of sight is not distributed (step S8). The result is as follows: Figure 5 As illustrated, if there is no area where the line of sight is not distributed (i.e., there is no direction that the line of sight cannot be directed towards) (step S8: No), the controller 10 determines that the driver is unaware of the visual obstruction (step S9). In this case, the controller 10 outputs information (visual obstruction notification information) to the driver from the display 36 and the speaker 37 (step S10), ending the driver state determination process.

[0079] Furthermore, in step S8, if there is an area where the driver's line of sight is not distributed (i.e., there is a direction that the line of sight cannot be directed towards) (step S8: Yes), the controller 10 determines that the driver is aware of the visual obstruction (step S11). In this case, since the driver is aware of the visual obstruction, no visual obstruction notification information is output, and the controller 10 ends the driver state determination process.

[0080] Furthermore, in step S5, if the amplitudes of the driver's head yaw angle and pitch angle ap are less than the threshold ath (step S5: No), the controller 10 determines that the driver's abnormal state is an attention impairment (step S12) and ends the driver state determination process. At this time, information informing the driver that an attention impairment has occurred can also be output from the display 36 and the speaker 37.

[0081] Furthermore, in step S3, if the frequency fs of the driver's gaze is greater than the threshold fth (step S3: No), the controller 10 determines that the past driver is in an abnormal state and determines whether the abnormal state is determined to be a visual obstruction (step S13). For example, the controller 10 determines whether the past driver's abnormal state is determined to be a visual obstruction based on whether information indicating that the driver's abnormal state is determined to be a visual obstruction is stored in the memory 10b.

[0082] As a result, if the past driver is determined to be in an abnormal state, and that abnormal state is determined to be visual impairment (step S13: Yes), as referred to Figure 4 As explained, although the driver previously experienced visual impairment, they have learned to compensate for the visual impairment through actions such as recovery. Therefore, it can be assumed that the scan rate fs has recovered to exceed the threshold fth. Thus, the controller 10 determines that the driver has learned to compensate for the visual impairment (step S14) and ends the driver state determination process.

[0083] On the other hand, if the driver's abnormal state was not determined to be an obstruction of vision in the past (step S13: No), the controller 10 determines that the driver's state is normal (step S15) and ends the driver state determination process.

[0084] Furthermore, in the driver state determination process of this embodiment, although in step S3 the controller 10 determines whether the driver is in an abnormal state (visual impairment or attention impairment) based on whether the frequency fs of the driver's saccades is below a threshold fth, other criteria can also be used to determine whether the driver is in an abnormal state. For example, if the amplitude of the driver's saccades is below a threshold, the driver can also be determined to be in an abnormal state. Moreover, as described in Japanese Patent Application Publication No. 2021-077136, the amplitude and frequency of the driver's saccades can be detected, and the level of attention in the external environment of the vehicle, which increases as the number of areas the driver should pay attention to increases while driving, can be detected. The driver can then be determined to be in an abnormal state based on the level of attention and the amplitude and frequency of the driver's saccades.

[0085] Furthermore, in the driver state determination process of this embodiment, in step S10, the controller 10 outputs information (visual obstruction notification information) to the driver from the display 36 and the speaker 37 to notify the driver that a visual obstruction has occurred. However, other information may also be output from the display 36 and the speaker 37 based on the driver state determination result. Control signals may also be sent to the PCM33, DSC34, and EPS35 to enable the drive power source 2, gearbox 3, brake 4, and steering device 5 to operate appropriately, thereby controlling the trajectory of the vehicle 1.

[0086] [Function / Effect]

[0087] Next, the effects of the driver status determination device 100 of this embodiment will be explained.

[0088] When the driver's gaze movement indicates an abnormal state, the controller 10 determines whether the abnormal state is due to attention impairment or visual impairment based on changes in the driver's head y-angle and pitch angle ap. Therefore, it can identify the type of abnormal state by utilizing the different head movements corresponding to whether the abnormal state is attention impairment or visual impairment. Furthermore, when the abnormal state is determined to be visual impairment, if there is a direction the driver cannot look in, the controller 10 determines that the driver is aware of the visual impairment; otherwise, it determines that the driver is unaware of the visual impairment. Therefore, even after the onset of symptoms of visual impairment diseases such as glaucoma, it can accurately determine whether the driver is unaware of the visual impairment or whether time has passed since the onset of symptoms. Thus, it is possible to determine the type and stage of the driver's abnormal state and provide appropriate driving assistance corresponding to the driver's state.

[0089] Furthermore, when the driver is determined to be in an abnormal state, if the amplitude of the driver's head yaw angle and pitch angle ap is above the threshold ath, the controller 10 determines that the driver's abnormal state is visual impairment; if it is below the threshold ath, the controller determines that the driver's abnormal state is attention impairment. Therefore, when the driver's abnormal state is visual impairment, and the driver tends to move their head up, down, left, and right to compensate for the conscious or unconscious lack of vision, the controller can determine that the driver's abnormal state is visual impairment based on the movement of their head. In other cases, the controller can appropriately determine that the driver's abnormal state is attention impairment. Thus, the controller can determine the type of the driver's abnormal state and provide appropriate driving assistance corresponding to the driver's state.

[0090] Furthermore, when it is determined that the driver is not in an abnormal state, in the past when the abnormal state of the driver was determined to be visual obstruction, the controller 10 determines that the driver has understood the state of compensating for visual obstruction. Therefore, it can appropriately distinguish between the state of the driver being in a normal state and the state of the driver understanding the state of compensating for visual obstruction. Thus, it is possible to determine the stage of the driver's abnormal state and to provide appropriate driving assistance corresponding to the driver's state.

[0091] Furthermore, the controller 10 determines whether the driver is in an abnormal state based on the frequency of scanning, thus enabling it to accurately determine the driver's abnormal state.

[0092] Furthermore, the driver state determination device 100 also includes a display 36 and a speaker 37 for outputting information to the driver. When the controller 10 determines that the driver is unaware of the obstruction to their field of vision, it outputs information indicating that the driver's field of vision is obstructed from the display 36 and the speaker 37. Therefore, appropriate driving assistance can be provided to drivers who are unaware of their obstruction to their field of vision.

Claims

1. A driver state determination device for determining the state of a driver of a vehicle, characterized in that, have: A gaze detection device that detects the driver's gaze; A head tracking detection device that detects the tracking of the driver's head; as well as A controller configured to determine the driver's state based on the driver's line of sight and head movement. The controller is configured as follows: The driver's gaze movement is used to determine whether the driver is in an abnormal state. When the driver is determined to be in an abnormal state, the changes in the driver's head yaw and pitch angles are used to determine whether the abnormal state is due to visual impairment. When the driver's abnormal state is determined to be visual impairment, if there is a direction that the driver's line of sight cannot be directed towards, it is determined that the driver is aware of the visual impairment. If there is no direction that the driver's line of sight cannot be directed towards, it is determined that the driver is unaware of the visual obstruction.

2. The driver status determination device as described in claim 1, characterized in that, The controller is configured as follows: When the driver is determined to be in an abnormal state, if the amplitude of the driver's head yaw angle and pitch angle is above a first threshold, the abnormal state of the driver is determined to be visual impairment.

3. The driver status determination device as described in claim 1 or 2, characterized in that, The controller is configured as follows: When it is determined that the driver is not in an abnormal state, in cases where the abnormal state of the driver was previously determined to be visual impairment, it is determined that the driver has understood the compensatory action for visual impairment.

4. The driver status determination device as described in claim 1 or 2, characterized in that, The controller is configured as follows: The frequency of the driver's gaze sweeps is detected based on the movement of the driver's gaze. If the frequency of the scanning is below the second threshold, the driver is determined to be in an abnormal state.

5. The driver status determination device as described in claim 1 or 2, characterized in that, It also has an information output device that outputs information to the driver. The controller is configured to output information indicating that the driver has a visual obstruction when it is determined that the driver is unaware of the visual obstruction through the information output device.

6. The driver status determination device as described in claim 1, characterized in that, The controller is configured as follows: When the driver is determined to be in an abnormal state, if the amplitude of the driver's head yaw angle and pitch angle is less than the first threshold, the abnormal state of the driver is determined to be attention impairment.

Images