Driver abnormality sign detection apparatus

Abstract

To provide a driver abnormality sign detection apparatus capable of highly accurately detecting an abnormality sign state of a driver at a sufficiently earlier stage than a stage at which the driver falls into an abnormal state of having difficulty in driving.SOLUTION: A driver abnormality sign detection apparatus 100 includes a controller 10 for detecting an abnormality sign state of a driver on the basis of visual line information acquired from a visual line detection device, driving operation information acquired from a driving operation detection device, and traveling environment information acquired from a traveling environment information acquisition device. The controller calculates a visual line abnormality degree representing a degree of deviation of an actual value of a visual line from a prediction value of the visual line, calculates a driving operation abnormality degree representing a degree of deviation of an actual value of driving operation from a prediction value of the driving operation, calculates a traveling risk of a vehicle, and detects the abnormality sign state of the driver when a total abnormality degree obtained by combining the visual line abnormality degree, the driving operation abnormality degree, and the traveling risk, is equal to or greater than a predetermined threshold value.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a driver abnormality sign detection device for detecting an abnormality sign state of a driver while driving a vehicle. [Background technology]

[0002] In recent years, development of driver abnormality response systems has been progressing, which detect abnormalities when the driver falls into a state where he or she cannot drive safely and automatically stop the vehicle. For example, when a driver abnormality is detected by detecting a change in the driver's posture, it is assumed that the system will gradually decelerate the vehicle while maintaining the lane, and if possible, move the vehicle to the shoulder of the road and automatically stop it.

[0003] In order to safely stop the vehicle while avoiding lane departure and contact with obstacles when an abnormality occurs in the driver, it is preferable to shorten the time from the occurrence of the abnormality in the driver to the detection as much as possible while preventing erroneous detection. Therefore, a driver abnormality determination device aimed at early determination of an abnormality in a driver who drives a vehicle has been proposed (for example, see Patent Document 1). In the device described in Patent Document 1, the abnormality determination unit receives the outputs of the involuntary function detection unit, the base function detection unit, and the predictive function detection unit, and determines the abnormality of the driver based on the detection target items and determination conditions according to the driving scene recognized by the driving scene recognition unit. Specifically, the driving functions of the driver are classified into involuntary functions, base functions, and predictive functions, and a combination of functions suitable for detecting the abnormality of the driver is determined according to the driving scene (for example, when driving on a city road or when driving on a highway). Then, the abnormality of the driver is determined based on the state of each function included in the combination. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2021-167163 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, when the present inventors conducted driving tests using actual vehicles, not using a driving simulator, with healthy and disabled people, it was found that the above-mentioned conventional technology does not always detect abnormalities in disabled people early. The reason for this is that the actual driving environment is more diverse than that of a simulator, so that the functions suitable for detecting abnormalities in the driver are not always the same even in one of the above-mentioned driving scenes, and it is difficult to detect abnormalities only with a predetermined combination of driving functions set for each driving scene. In addition, since disabled people also try to drive as carefully as possible when actually driving a vehicle, it is also considered that the change in the driver's behavior due to the deterioration of driving function is temporary and does not continue, making it difficult to accurately detect the deterioration of driving function in a short time. Therefore, in order to prevent erroneous detection and reliably detect abnormalities in the above-mentioned conventional technology, the timing of detection must be delayed until the deterioration of the driver's driving function is clearly and continuously manifested. In other words, there is room for further early and highly accurate detection of abnormal signs in the driver.

[0006] The present invention has been made to solve such problems, and aims to provide a driver abnormality precursor detection device that is capable of detecting a driver's abnormality precursor state with high accuracy at a stage sufficiently early before the driver enters an abnormal state that makes it difficult for him to drive. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, the present invention provides a driver abnormality sign detection device that detects an abnormality sign state of a driver driving a vehicle, the device comprising: a gaze detection device that detects the gaze of the driver; a driving operation detection device that detects driving operations by the driver; a driving environment information acquisition device that acquires driving environment information of the vehicle; and a controller configured to detect an abnormality sign state of the driver based on the gaze information acquired from the gaze detection device, the driving operation information acquired from the driving operation detection device, and the driving environment information acquired from the driving environment information acquisition device, the controller being configured to predict the gaze based on the driving environment information. the gaze abnormality degree indicating the degree to which the actual gaze value deviates from the predicted gaze value; the predicted driving operation value based on the driving environment information; the actual driving operation value based on the driving operation information; the driving operation abnormality degree indicating the degree to which the actual driving operation value deviates from the predicted driving operation value; the driving risk of the vehicle based on the driving environment information; the gaze abnormality degree, the driving operation abnormality degree and the driving risk are combined to calculate an overall abnormality degree; and if the overall abnormality degree is equal to or greater than a predetermined threshold, it is determined that the driver's abnormal premonition state has been detected.

[0008] According to the present invention configured as described above, the controller calculates the gaze abnormality degree indicating the degree to which the actual measured value of the gaze deviates from the predicted value of the gaze, calculates the driving operation abnormality degree indicating the degree to which the actual measured value of the driving operation deviates from the predicted value of the driving operation, calculates the driving risk of the vehicle, and detects the driver's abnormality premonition state when the overall abnormality degree obtained by combining the gaze abnormality degree, the driving operation abnormality degree, and the driving risk is equal to or greater than a predetermined threshold value. Therefore, the abnormality premonition state can be detected early and with high accuracy even at a sufficiently early stage before the driver reaches an abnormal state where driving is difficult, such that a decrease in the driver's individual driving function cannot be detected.

[0009] In the present invention, the controller is preferably configured to acquire the maximum gaze abnormality degree and the maximum driving operation abnormality degree in a most recent specified time period, and calculate an overall abnormality degree by combining the maximum gaze abnormality degree, the maximum driving operation abnormality degree, and the driving risk.

[0010] According to the present invention configured as described above, even if the change in the driver's checking behavior or operating behavior is temporary and does not continue, it is possible to detect the driver's abnormality prediction state based on the overall abnormality degree that reflects the change, thereby making it possible to more reliably detect the abnormality prediction state.

[0011] In the present invention, the controller is preferably configured to obtain an average value of driving risk over a most recent specified time period, and calculate an overall abnormality degree by combining the maximum gaze abnormality degree, the maximum driving operation abnormality degree, and the average value of the driving risk.

[0012] According to the present invention configured in this manner, it is possible to detect a driver's abnormality prediction state based on a comprehensive abnormality degree that reflects an average increase in driving risk, not a temporary increase, as a result of a change in the driver's driving behavior. Therefore, it is possible to prevent erroneous detection of an abnormality prediction state due to an accidental change in driving behavior that is not caused by a driver's illness, and it is possible to more accurately detect an abnormality prediction state.

[0013] In the present invention, the controller is preferably configured to determine that a driver's abnormality precursor state has been detected when the overall abnormality degree is equal to or greater than a predetermined threshold value and an increase in driving risk is detected.

[0014] According to the present invention configured in this manner, it is possible to detect an abnormality prediction state of a driver when a change in the driver's driving behavior results in an increased driving risk and an increased possibility of lane departure or approaching an obstacle. Therefore, it is possible to prevent erroneous detection of an abnormality prediction state due to a temporary change in driving behavior that is not caused by a driver's illness, and it is possible to more accurately detect an abnormality prediction state.

[0015] In the present invention, the controller is preferably configured to obtain a predicted value of the driver's saccade based on the driving environment information, obtain an actual value of the driver's saccade based on the gaze information, and calculate the degree to which the actual saccade value deviates from the predicted saccade value as a gaze abnormality degree.

[0016] According to the present invention configured in this manner, the controller calculates the degree of deviation of the actual saccade value from the predicted saccade value as the degree of gaze abnormality, so that changes in the driver's confirmation behavior due to a decline in their perceptual function, attention function, or motor function can be grasped using changes in saccades.

[0017] In the present invention, the controller is preferably configured to obtain predicted values ​​of the operation amounts of the steering wheel, accelerator pedal and / or brake pedal based on the driving environment information, obtain actual values ​​of the operation amounts based on the driving operation information, and calculate the degree to which the actual values ​​of the operation amounts deviate from the predicted values ​​of the operation amounts as a driving operation abnormality degree.

[0018] According to the present invention configured in this manner, changes in the driver's operating behavior due to a decline in the driver's perceptual function, attention function, or motor function can be grasped using changes in the amount of operation of the steering wheel, accelerator pedal, and / or brake pedal. Effect of the Invention

[0019] According to the driver abnormality sign detection device of the present invention, it is possible to detect an abnormality sign state of a driver with high accuracy at a stage sufficiently earlier than the driver entering an abnormal state that makes it difficult for him to drive. [Brief description of the drawings]

[0020] [Figure 1] 1 is an explanatory diagram of a vehicle equipped with a driver abnormality sign detection device according to an embodiment of the present invention. [Diagram 2] 1 is a block diagram of a driver abnormality sign detection device according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a control block diagram of abnormality sign detection according to an embodiment of the present invention. [Figure 4] 1 is a diagram showing a total abnormality degree in an embodiment of the present invention, expressed in a three-dimensional orthogonal coordinate system with the gaze abnormality degree, the driving operation abnormality degree, and the driving risk as the respective coordinate axes. [Diagram 5] 10 is a flowchart of a gaze abnormality degree calculation process according to an embodiment of the present invention. [Figure 6] 4 is a flowchart of a driving operation abnormality degree calculation process according to an embodiment of the present invention. [Figure 7] 4 is a flowchart of a driving risk calculation process according to an embodiment of the present invention. [Figure 8] 4 is a flowchart of an abnormality sign detection process according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A driver abnormality sign detection device according to an embodiment of the present invention will now be described with reference to the accompanying drawings.

[0022] [System Configuration] First, the configuration of a driver abnormality sign detection device according to the present embodiment will be described with reference to Figures 1 and 2. Figure 1 is an explanatory diagram of a vehicle equipped with a driver abnormality sign detection device, and Figure 2 is a block diagram of the driver abnormality sign detection device.

[0023] The vehicle 1 according to this embodiment is equipped with a driving force source 2 such as an engine or an electric motor that outputs driving force, a transmission 3 that transmits the driving force output from the driving force source 2 to the driving wheels, a brake 4 that applies a braking force to the vehicle 1, and a steering device 5 for steering the vehicle 1.

[0024] The driver abnormality sign detection device 100 is configured to detect an abnormality sign state of the driver of the vehicle 1 and, if necessary, control the vehicle 1 or drive assistance control. As shown in Fig. 2, the driver abnormality sign detection device 100 has a controller 10, a plurality of sensors, a plurality of control systems, and a plurality of information output devices.

[0025] Specifically, the sensors include an outside camera 21 for acquiring driving environment information of the vehicle 1, a radar 22, a navigation system (navi system) 23 for detecting the position of the vehicle 1, and a positioning system 24. The sensors also include a vehicle speed sensor 25, an acceleration sensor 26, a yaw rate sensor 27, a steering angle sensor 28, a steering torque sensor 29, an accelerator sensor 30, and a brake sensor 31 for detecting the behavior of the vehicle 1 and the driving operation by the driver. The sensors also include an in-vehicle camera 32 for detecting the line of sight of the driver. The control systems include a powertrain control module (PCM) 33 for controlling the driving force source 2 and the transmission 3, a dynamic stability control system (DSC) 34 for controlling the driving force source 2 and the brake 4, and an electric power steering system (EPS) 35 for controlling the steering device 5. The information output devices include a display 36 for outputting image information, and a speaker 37 for outputting audio information.

[0026] Other sensors may also include a surrounding sonar that measures the distance and position of surrounding structures relative to vehicle 1, a corner radar that measures the approach of surrounding structures at the four corners of vehicle 1, and various sensors that detect the driver's condition (e.g., a heart rate sensor, an electrocardiogram sensor, a steering wheel grip force sensor, etc.).

[0027] The controller 10 executes various calculations based on signals received from a plurality of sensors, transmits control signals to the PCM 33, DSC 34, and EPS 35 for appropriately operating the driving force source 2, transmission 3, brake 4, and steering device 5, and transmits control signals to the display 36 and speaker 37 for outputting desired information. The controller 10 is composed of a computer equipped with one or more processors 10a (typically a CPU), memory 10b (ROM, RAM, etc.) for storing various programs and data, input / output devices, etc.

[0028] The exterior camera 21 captures images of the surroundings of the vehicle 1 and outputs image data. The controller 10 identifies the positions and speeds of objects (e.g., preceding vehicles, parked vehicles, pedestrians, roads, dividing lines (lane boundaries, white lines, yellow lines), traffic signals, traffic signs, stop lines, intersections, obstacles, etc.) based on the image data received from the exterior camera 21. The exterior camera 21 corresponds to an example of a "driving environment information acquisition device" in the present invention, and the information acquired by the exterior camera 21 corresponds to an example of "driving environment information" in the present invention.

[0029] The radar 22 measures the position and speed of an object (particularly, a preceding vehicle, a parked vehicle, a pedestrian, a fallen object on the road, etc.). For example, a millimeter wave radar can be used as the radar 22. The radar 22 transmits radio waves in the traveling direction of the vehicle 1, and receives a reflected wave generated when the transmitted wave is reflected by the object. Then, the radar 22 measures the distance between the vehicle 1 and the object (for example, the distance between the vehicles) and the relative speed of the object with respect to the vehicle 1 based on the transmitted wave and the received wave. In this embodiment, instead of the radar 22, a laser radar, an ultrasonic sensor, etc. may be used to measure the distance to the object and the relative speed. Also, a position and speed measuring device may be configured using a plurality of sensors. The radar 22 corresponds to an example of a "driving environment information acquisition device" in the present invention, and the information acquired by the radar 22 corresponds to an example of "driving environment information" in the present invention.

[0030] The navigation system 23 stores map information internally and can provide the map information to the controller 10. The controller 10 identifies roads, intersections, traffic signals, buildings, etc. that exist around the vehicle 1 (particularly in the traveling direction) based on the map information and current vehicle position information. The map information may be stored in the controller 10. The positioning system 24 is a GPS system and / or a gyro system, and detects the position of the vehicle 1 (current vehicle position information). The navigation system 23 and the positioning system 24 also correspond to an example of the "driving environment information acquisition device" in the present invention, and the information acquired by the navigation system 23 and the positioning system 24 corresponds to an example of the "driving environment information" in the present invention.

[0031] The vehicle speed sensor 25 detects the speed of the vehicle 1 based on, for example, the rotation speed of the wheels or the drive shaft. The acceleration sensor 26 detects the acceleration of the vehicle 1. This acceleration includes the acceleration in the longitudinal direction of the vehicle 1 and the acceleration in the lateral direction (i.e., lateral acceleration). In this specification, the acceleration includes not only the rate of change of the speed in the direction in which the speed increases, but also the rate of change of the speed in the direction in which the speed decreases (i.e., deceleration). The yaw rate sensor 27 detects the yaw rate of the vehicle 1. That is, the vehicle speed sensor 25, the acceleration sensor 26, and the yaw rate sensor 27 acquire vehicle state information related to the state of the vehicle 1.

[0032] The steering angle sensor 28 detects the rotation angle (steering angle) of the steering wheel of the steering device 5. The steering torque sensor 29 detects the torque (steering torque) applied to the steering shaft via the steering wheel. The accelerator sensor 30 detects the amount of depression of the accelerator pedal. The brake sensor 31 detects the amount of depression of the brake pedal. In other words, the steering angle sensor 28, the steering torque sensor 29, the accelerator sensor 30, and the brake sensor 31 detect the driving operation of the driver and correspond to an example of a "driving operation detection device" in the present invention, and information obtained by the steering angle sensor 28, the steering torque sensor 29, the accelerator sensor 30, and the brake sensor 31 corresponds to an example of the "driving operation information" in the present invention.

[0033] The in-vehicle camera 32 captures an image of the driver and outputs image data. The controller 10 detects the behavior of the driver's head and the direction of his / her gaze based on the image data received from the in-vehicle camera 32. The in-vehicle camera 32 corresponds to an example of a "gaze detection device" in the present invention, and the information acquired by the in-vehicle camera 32 corresponds to an example of "gaze information" in the present invention.

[0034] The PCM 33 controls the driving force source 2 of the vehicle 1 to adjust the driving force of the vehicle 1. For example, the PCM 33 controls the engine's spark plugs, fuel injection valves, throttle valves, variable valve mechanisms, the transmission 3, and an inverter that supplies power to the electric motor. When it is necessary to accelerate or decelerate the vehicle 1, the controller 10 transmits a control signal to the PCM 33 to adjust the driving force.

[0035] The DSC 34 controls the driving force source 2 and the brakes 4 of the vehicle 1 to perform deceleration control and attitude control of the vehicle 1. For example, the DSC 34 controls the hydraulic pump and valve unit of the brakes 4, and controls the driving force source 2 via the PCM 33. When it is necessary to perform deceleration control or attitude control of the vehicle 1, the controller 10 transmits a control signal to the DSC 34 to adjust the driving force or generate a braking force.

[0036] The EPS 35 controls the steering device 5 of the vehicle 1. For example, the EPS 35 controls an electric motor that applies torque to a steering shaft of the steering device 5. When it is necessary to change the traveling direction of the vehicle 1, the controller 10 transmits a control signal to the EPS 35 to change the steering direction.

[0037] The display 36 is provided in front of the driver in the vehicle cabin and displays image information to the driver. For example, a liquid crystal display or a head-up display is used as the display 36. The speaker 37 is provided in the vehicle cabin and outputs various types of audio information.

[0038] [Driver Abnormality Prediction Detection Overview] Next, a basic concept of driver abnormality sign detection executed by the above-described controller 10 in this embodiment will be described with reference to Fig. 3. Fig. 3 is a control block diagram of abnormality sign detection according to this embodiment.

[0039] In the abnormality premonition state before the driver reaches an abnormal state where driving is difficult, it is considered that the driver's driving function is at least temporarily deteriorating due to a minor illness, aging, or the like. When the driver's driving function is deteriorating, the driver's driving behavior, for example, the confirmation behavior of checking the driving environment and the operation behavior of driving the vehicle 1, correspondingly changes. Furthermore, as a result of the change in driving behavior, it is considered that the risk (driving risk) of lane departure, approaching an obstacle, etc. increases. Therefore, the present inventors thought that it would be possible to detect the abnormality premonition state early and with high accuracy without detecting a clear deterioration of each individual driving function by comprehensively determining these changes in driving behavior and the increase in driving risk.

[0040] Here, the driving function of the driver includes a perception function for perceiving objects present in the driving environment, and attention functions such as a function for simultaneously viewing multiple objects in the driving environment (distributed attention function), a function for selecting and viewing one of multiple objects (selective attention function), a function for switching objects (switchable attention function), and a function for continuing to view an object (sustained attention function). The driving function of the driver also includes motor functions necessary for operating the steering wheel, accelerator pedal, brake pedal, etc. of the vehicle 1.

[0041] As a result of research by the present inventors, it was found that when the perceptual function, attention function, or motor function is deteriorated, the driver's confirmation behavior changes as a result of changes in the movement of the driver's gaze. For example, when the perceptual function is deteriorated due to visual field loss or the like, the range of gaze movement may become narrower. Also, when the attention function is deteriorated, the frequency of gaze movement decreases, and the distance of gaze movement becomes shorter. Furthermore, when the motor function is deteriorated due to paralysis of the limbs or the like, the driver may be concerned about the condition of the paralyzed limbs and may turn his / her gaze in the direction of the paralyzed limbs. Therefore, by determining the degree of abnormality in the driver's gaze movement based on the gaze movement according to the driving environment and vehicle condition in a state without a disease (healthy state), it is possible to grasp the change in the confirmation behavior due to the deterioration of the driver's perceptual function, attention function, or motor function.

[0042] It has also been found that when the perceptual function, attention function, or motor function is deteriorated, the operation behavior of the steering wheel, accelerator pedal, and brake pedal changes, and the operation of the steering wheel, accelerator pedal, and brake pedal changes. For example, when the motor function is deteriorated, the operation of the steering wheel and the pedals may be delayed compared to when the driver is in a healthy state. In addition, when the perceptual function or attention function is deteriorated, it affects the driver's recognition of the driving environment, which is the basis of the driving operation, and therefore the operation of the steering wheel and the pedals may become unstable compared to when the driver is in a healthy state, for example, due to the driver being unable to grasp the position of the vehicle in the lane well, or being late in finding or missing an object. Therefore, by calculating the abnormality degree of the driver's driving operation based on the driving operation according to the driving environment and vehicle state in the healthy state, it is possible to grasp the change in the operating behavior due to the deterioration of the driver's perceptual function, attention function, or motor function.

[0043] In addition, when the driver's perception function, attention function, or motor function is deteriorated, the driving risk such as lane departure and approach to obstacles increases due to delay or oversight of the object and unstable driving operation. According to the research by the present inventors, in a healthy driver, even if the driving risk increases temporarily due to inattention or disturbance of operation, the driving risk is immediately reduced because an appropriate corrective operation is performed after that. Therefore, the average value of the driving risk remains at a low level. In contrast, in a driver with a deteriorated driving function, it becomes gradually difficult to perform driving operations to reduce the driving risk, so the driving risk gradually increases. As a result, the average value of the driving risk becomes higher than that of a healthy person, and the trend of increasing the driving risk continues. Therefore, by determining the average value and increasing trend of the driving risk, the influence of the change in driving behavior due to the deterioration of the driver's motor function, perception function, or attention function can be grasped.

[0044] Therefore, the controller 10 of this embodiment calculates the degree of abnormality of the driver's gaze (gaze abnormality degree) based on the gaze information, driving operation information, driving environment information, vehicle state information, and head behavior information, calculates the degree of abnormality of the driver's driving operation (driving operation abnormality degree) based on the driving operation information, driving environment information, and vehicle state information, and calculates the driving risk based on the driving environment information and vehicle state information.The controller 10 is then configured to calculate the overall abnormality degree of the driver's state (overall abnormality degree) based on the gaze abnormality degree, driving operation abnormality degree, and driving risk, and to detect the driver's abnormality precursor state based on the overall abnormality degree.

[0045] Specifically, as shown in FIG. 3, the controller 10 acquires gaze information and head behavior information based on signals received from the in-vehicle camera 32, acquires driving operation information based on signals received from the steering angle sensor 28, the steering torque sensor 29, the accelerator sensor 30 and the brake sensor 31, and acquires driving environment information and vehicle state information based on signals received from sensors including the exterior camera 21, the radar 22, the navigation system 23, the positioning system 24, the vehicle speed sensor 25, the acceleration sensor 26 and the yaw rate sensor 27.

[0046] The controller 10 inputs the acquired driving operation information, driving environment information, vehicle state information, and head behavior information into the gaze model, and calculates a predicted value (gaze predicted value) of the gaze movement of a healthy driver. As a feature value representing the gaze movement, for example, the amplitude and frequency of saccades can be used.

[0047] Here, a saccade is a saccadic eye movement for capturing a visual target on the fovea of ​​the retina, and refers to an eye movement in which the line of sight moves from a fixation point where the line of sight remains for a predetermined period of time to the next fixation point. The amplitude of a saccade refers to the amount of movement when the driver's line of sight moves from a fixation point to the next fixation point, and the frequency of a saccade refers to the number of times the line of sight moves from a fixation point to the next fixation point within a predetermined period of time.

[0048] The movement of the line of sight is affected by the driver's steering operation, the vehicle state (e.g., vehicle speed, etc.), the driving environment (e.g., variations in illuminance and saliency, etc.), the driver's head behavior, etc. Here, saliency is a characteristic that indicates how easily an object attracts human attention, and refers to a visual feature determined by the temporal and spatial arrangement of color, brightness, movement, etc. In other words, an area with high saliency in the driver's visual field is an area that is likely to attract the driver's attention, for example, because it has a large color difference or brightness difference from the surrounding areas, or because it is moving significantly.

[0049] For example, when a driver is performing steering operation at an intersection or in a situation where there are successive curves, the driver focuses his / her gaze in the direction in which the vehicle is turning, so the variation in gaze tends to decrease. In addition, when the vehicle speed is relatively high on a highway, the driver's gaze tends to focus in the direction of travel. In addition, when the illumination of the driving environment is relatively low, such as at night, the driver's gaze also tends to focus in the direction of travel. In addition, when the variation in saliency around the vehicle is small (i.e., when areas with high saliency are concentrated in a part of the driver's visual field), the driver's gaze tends to focus in areas with high saliency. In addition, the driver's gaze tends to be biased in the direction in which the driver's face is facing.

[0050] Therefore, the gaze model is configured to output a gaze prediction value for a driver in a healthy state when the feature quantities such as steering operation, vehicle state, driving environment, and driver's head behavior that affect the gaze movement are input as parameters. Specifically, for example, the reference values ​​of saccade frequency and amplitude in a healthy state and correction maps for each feature quantity that affects the gaze movement, such as the standard deviation of the steering wheel operation amount (steer angle), vehicle speed, illuminance of the driving environment, saliency variation, and the standard deviation of the driver's face direction, are set in advance in the gaze model. By correcting the reference values ​​according to each of the input feature quantities based on these correction maps, a gaze prediction value that reflects the influence of the steering operation, vehicle state, driving environment, and driver's head behavior can be obtained.

[0051] The controller 10 calculates a gaze abnormality degree indicating the degree to which the gaze actual measurement value deviates from the gaze predicted value based on the actual measurement value of the driver's gaze movement (gaze actual measurement value) identified from the acquired gaze information and the gaze predicted value output from the gaze model configured as described above (gaze abnormality degree calculation).

[0052] For example, the controller 10 calculates the number of saccades per unit time as the actual value of the saccade frequency based on the number of saccades in a predetermined time (e.g., 30 seconds). The controller 10 also calculates the average value of the saccade amplitude in a predetermined time (e.g., 30 seconds) as the actual value of the saccade amplitude. Then, the controller 10 accumulates two-dimensional data in which the difference between the calculated actual value of the saccade frequency and the predicted value output from the gaze model is set as a first variable, and the difference between the calculated actual value of the saccade amplitude and the predicted value output from the gaze model is set as a second variable. The controller 10 calculates the Mahalanobis distance between the latest data point of this two-dimensional data and the center of gravity (average) of the accumulated data set. There is a correlation between the difference between the actual value and the predicted value of the saccade frequency and the difference between the actual value and the predicted value of the saccade amplitude, so by using the Mahalanobis distance in this way, the degree of deviation of the gaze movement from a normal state can be expressed by one index. Furthermore, the controller 10 calculates the gaze abnormality degree by dividing the calculated Mahalanobis distance by a preset representative value and normalizing it. As this representative value, the Mahalanobis distance when the driver is in an abnormality prediction state can be used. That is, when the gaze abnormality degree is 1, the driver is in an abnormality prediction state.

[0053] Then, the controller 10 accumulates the calculated gaze anomalous degree in a buffer (gaze anomalous degree buffer), and acquires the maximum value of the gaze anomalous degree in the most recent predetermined time period (for example, 60 seconds) from the gaze anomalous degree buffer (acquire maximum value).

[0054] The controller 10 also inputs the acquired driving environment information and vehicle state information into a driving operation prediction model to calculate a predicted value of the driving operation of a healthy driver (driving operation predicted value). As a feature quantity representing the driving operation, for example, the operation amount of the steering wheel (steer angle) and the operation amounts of the accelerator pedal and the brake pedal (accelerator pedal depression amount and brake pedal depression amount) can be used.

[0055] The driving operation prediction model is configured to output a driving operation prediction value for a healthy driver when the feature quantities of the driving environment and vehicle state required for driving operation are input as parameters. Specifically, when the feature quantities of the driving environment such as the positions of lane markings and obstacles in the traveling direction of the vehicle 1, the vehicle speed limit, the position and speed of the preceding vehicle, and the feature quantities of the vehicle state such as the vehicle speed and acceleration are input as parameters, the driving operation prediction model is configured to output, as a driving operation prediction value, the operation amount of the steering wheel required for driving in the center of the driving lane, and the operation amounts of the accelerator pedal and the brake pedal required for following the preceding vehicle or driving the vehicle 1 at the vehicle speed limit.

[0056] The controller 10 calculates a driving operation abnormality degree indicating the degree to which the actual driving operation value deviates from the predicted driving operation value based on the actual driving operation value (actual driving operation value) of the driver identified from the acquired driving operation information and the predicted driving operation value output from the driving operation prediction model (calculation of driving operation abnormality degree).

[0057] For example, the controller 10 accumulates two-dimensional data in which the difference between the actual value of the steering wheel operation amount and the predicted value output from the driving operation prediction model is set as a first variable, and the difference between the actual value of the accelerator pedal or brake pedal operation amount and the predicted value output from the driving operation prediction model is set as a second variable. The controller 10 calculates the Mahalanobis distance between the latest data point of this two-dimensional data and the center of gravity (average) of the accumulated data set. Since there is a correlation between the difference between the actual value and the predicted value of the steering wheel operation amount and the difference between the actual value and the predicted value of the pedal operation amount, by using the Mahalanobis distance in this way, the degree of deviation of the driving operation from a normal state can be expressed by one index. Furthermore, the controller 10 calculates the driving operation abnormality degree by dividing the calculated Mahalanobis distance by a preset representative value and normalizing it. As this representative value, the Mahalanobis distance when the driver is in an abnormality prediction state can be used. That is, when the driving operation abnormality degree is 1, the driver is in an abnormality prediction state.

[0058] The controller 10 then accumulates the calculated driving performance abnormality degree in a buffer (driving performance abnormality degree buffer), and obtains the maximum value of the driving performance abnormality degree in the most recent predetermined time period from the driving performance abnormality degree buffer (obtain maximum value).

[0059] The controller 10 also calculates a driving risk based on the acquired driving environment information and vehicle state information. The driving risk is a numerical representation of the possibility of lane departure, approach to an obstacle, and the like. For example, the driving risk is set to 0 when it is predicted that the vehicle 1 will be located in the center of the lane and the distance to the obstacle will be a safe distance (e.g., 1 m) or more after a predetermined time (e.g., 2 seconds), and the driving risk approaches 1 as the predicted values ​​of the distance to lane departure and the distance to the obstacle after a predetermined time become smaller, and the driving risk is set to 1 when it is predicted that lane departure will occur or the distance to the obstacle will be less than a limit distance (e.g., 0.3 m) after a predetermined time. That is, the controller 10 specifies the dividing line of the road in the traveling direction of the vehicle 1, the position of the vehicle 1 on the road, and the position and speed of the obstacle from the acquired driving environment information. Also, the controller 10 specifies the current speed and acceleration of the vehicle 1 from the acquired driving environment information or vehicle state information, and predicts the position of the vehicle 1 after a predetermined time. Then, based on the predicted position of vehicle 1 and the positions and speed of marking lines and obstacles, predicted values ​​of the distance until vehicle 1 deviates from the lane after a specified time and the distance to the obstacle are calculated, and the driving risk is calculated based on the calculated predicted values.

[0060] Then, the controller 10 accumulates the calculated driving risk in a buffer (risk buffer), and acquires an average value of the driving risk in the most recent predetermined time from the risk buffer (acquire average value).

[0061] The controller 10 calculates a total abnormality degree from the acquired maximum gaze abnormality degree, maximum driving operation abnormality degree, and average value of the driving risk (total abnormality degree calculation). For example, in a three-dimensional orthogonal coordinate system with the maximum gaze abnormality degree, maximum driving operation abnormality degree, and average value of the driving risk as axes, the controller 10 sets a three-dimensional vector having the acquired maximum gaze abnormality degree, maximum driving operation abnormality degree, and average value of the driving risk as components as a total abnormality degree vector, and calculates the magnitude of the total abnormality degree vector as the total abnormality degree.

[0062] FIG. 4 is a diagram showing the overall abnormality degree in a three-dimensional orthogonal coordinate system with the gaze abnormality degree, driving operation abnormality degree, and driving risk as the coordinate axes. As shown in FIG. 4, if the maximum acquired gaze abnormality degree is Gab_max, the maximum acquired driving operation abnormality degree is Oab_max, and the average driving risk is R_ave, the magnitude of the overall abnormality degree vector Tab is (Gab_max 2 +Oab_max 2 +R_ave 2 ) 1 / 2 For example, in the example of Fig. 4, the maximum value of the gaze abnormality degree Gab_max1, the maximum value of the driving operation abnormality degree Oab_max1, and the average value of the driving risk R_ave1 are all less than 1, but even in this case, the magnitude of the total abnormality degree vector Tab is greater than 1. Furthermore, if any one of the acquired maximum value of the gaze abnormality degree, the maximum value of the driving operation abnormality degree, and the average value of the driving risk is greater than 1, the total abnormality degree is greater than 1.

[0063] The controller 10 judges whether the calculated total abnormality degree is equal to or greater than a predetermined threshold value (threshold judgment). When the magnitude of the total abnormality degree vector obtained as described above is calculated as the total abnormality degree, the threshold value is, for example, 1, and in the example of Fig. 4, a sphere with a radius of 1 corresponds to the threshold value.

[0064] In addition, when the controller 10 determines, based on the calculated driving risk, that the driving risk is increasing and that the rate of increase will reach 1 within a specified time (for example, within 5 seconds), if that rate is maintained, it detects an increase in driving risk (detection of increased driving risk).

[0065] Then, when the total abnormality degree is equal to or greater than the threshold value and an increase in driving risk is detected, the controller 10 determines that the driver is in an abnormality premonition state, that is, detects the driver's abnormality premonition state (abnormality premonition detection).

[0066] When an abnormality predictor state is detected, the controller 10 transmits control signals to the PCM 33, the DSC 34, and the EPS 35 for appropriately operating the driving force source 2, the transmission 3, the brake 4, and the steering device 5, and transmits control signals to the display 36 and the speaker 37 for outputting desired information. For example, the controller 10 causes the EPS 35 to vibrate the steering wheel at a predetermined frequency, and determines that the driver is in an abnormal state based on the driver's response. The controller 10 also causes the display 36 to output a warning display, and determines that the driver is in an abnormal state based on the driver's response to the warning display.

[0067] [Anomaly detection process] Next, a flow of detection processing of an abnormality sign state by the driver abnormality sign detection device 100 of this embodiment will be described with reference to Fig. 5 to Fig. 8. Fig. 5 is a flowchart of gaze abnormality degree calculation processing, Fig. 6 is a flowchart of driving operation abnormality degree calculation processing, Fig. 7 is a flowchart of driving risk calculation processing, and Fig. 8 is a flowchart of abnormality sign detection processing. The gaze abnormality degree calculation processing, driving operation abnormality degree calculation processing, driving risk calculation processing, and abnormality sign detection processing are each started when the power supply of the vehicle 1 is turned ON, and are repeatedly executed in parallel by the controller 10 at a predetermined cycle (for example, every 0.05 to 0.2 seconds).

[0068] 5 is started, the controller 10 detects the driver's line of sight based on a signal received from the in-vehicle camera 32 (step S1). Next, the controller 10 calculates actual measurement values ​​of the frequency and amplitude of saccades based on the detected line of sight of the driver (step S2).

[0069] The controller 10 also acquires head behavior information based on signals received from the in-vehicle camera 32, acquires driving operation information based on signals received from the steering angle sensor 28, the steering torque sensor 29, the accelerator sensor 30, and the brake sensor 31, and acquires driving environment information and vehicle state information based on signals received from sensors including the outside-vehicle camera 21, the radar 22, the navigation system 23, the positioning system 24, the vehicle speed sensor 25, the acceleration sensor 26, and the yaw rate sensor 27 (step S3).The controller 10 then inputs the information acquired in step S3 into a gaze model and calculates predicted values ​​of the frequency and amplitude of saccades (step S4).

[0070] Next, the controller 10 calculates the gaze abnormality degree based on the actual measurement values ​​of the saccade frequency and amplitude calculated in step S2 and the predicted values ​​of the saccade frequency and amplitude calculated in step S4, and stores the degree in a buffer (step S5). As described above, for example, the controller 10 calculates the Mahalanobis distance between the latest data point of the two-dimensional data, in which the difference between the actual measurement value and the predicted value of the saccade frequency is set as a first variable and the difference between the actual measurement value and the predicted value of the saccade amplitude is set as a second variable, and the center of gravity (average) of the data set accumulated so far. Furthermore, the controller 10 calculates the gaze abnormality degree by dividing the calculated Mahalanobis distance by a preset representative value and normalizing it. After step S5, the controller 10 ends the gaze abnormality degree calculation process.

[0071] When the driving operation abnormality degree calculation process of FIG. 6 is started, the controller 10 detects the driver's driving operation, specifically, the amount of operation of the steering wheel, the amount of operation of the accelerator pedal, and the amount of operation of the brake pedal, based on signals received from the steering angle sensor 28, the steering torque sensor 29, the accelerator sensor 30, and the brake sensor 31 (step S11).

[0072] The controller 10 also acquires driving environment information and vehicle state information based on signals received from sensors including the outside camera 21, the radar 22, the navigation system 23, the positioning system 24, the vehicle speed sensor 25, the acceleration sensor 26, and the yaw rate sensor 27 (step S12).The controller 10 then inputs the information acquired in step S12 into a driving performance prediction model and calculates a predicted value of the driving performance (step S13).

[0073] Next, the controller 10 calculates a driving operation abnormality degree based on the actual measurement value of the driver's driving operation detected in step S11 and the predicted value of the driving operation calculated in step S13, and stores the degree in a buffer (step S14). As described above, for example, the controller 10 calculates the Mahalanobis distance between the latest data point of the two-dimensional data, in which the difference between the actual measurement value and the predicted value of the operation amount of the steering wheel is set as a first variable and the difference between the actual measurement value and the predicted value of the operation amount of the accelerator pedal or the brake pedal is set as a second variable, and the center of gravity (average) of the data set accumulated so far. Furthermore, the controller 10 calculates the driving operation abnormality degree by dividing the calculated Mahalanobis distance by a preset representative value and normalizing it. After step S14, the controller 10 ends the driving operation abnormality degree calculation process.

[0074] When the driving risk calculation process in Figure 7 is started, the controller 10 acquires driving environment information and vehicle state information based on signals received from sensors including the exterior camera 21, radar 22, navigation system 23, positioning system 24, vehicle speed sensor 25, acceleration sensor 26, and yaw rate sensor 27 (step S21).

[0075] Next, the controller 10 calculates a driving risk based on the driving environment information and vehicle state information acquired in step S21, and stores the calculated driving risk in a buffer (step S22). As described above, for example, the controller 10 identifies the lane markings on the road in the traveling direction of the vehicle 1, the position of the vehicle 1 on the road, and the positions and speed of obstacles from the acquired driving environment information. In addition, the controller 10 identifies the current speed and acceleration of the vehicle 1 from the acquired driving environment information or vehicle state information, and predicts the position of the vehicle 1 after a predetermined time. Then, based on the predicted position of the vehicle 1 and the positions and speed of the lane markings and obstacles, it calculates predicted values ​​of the distance to the lane departure of the vehicle 1 after a predetermined time and the distance to the obstacles, and calculates the driving risk based on the calculated predicted values.

[0076] Next, the controller 10 judges whether the driving risk is increasing based on the driving risk calculated in step S22 (step S23). As described above, for example, when the controller 10 judges that the driving risk is increasing and that the increasing rate will reach 1 within a predetermined time if it is maintained, the controller 10 judges that the driving risk is increasing.

[0077] As a result, if it is determined that the driving risk is increasing (step S23: Yes), that is, if the controller 10 detects an increase in the driving risk, the controller 10 sets the driving risk increase flag to TRUE (step S24). On the other hand, if it is not determined that the driving risk is increasing (step S23: No), the controller 10 sets the driving risk increase flag to FALSE (step S25). After the processing of step S24 or S25, the controller 10 ends the driving risk calculation processing.

[0078] When the abnormality sign detection process of FIG. 8 is started, the controller 10 acquires the maximum gaze abnormality degree in the most recent predetermined time period (for example, 60 seconds) from the gaze abnormality degree buffer (step S31).

[0079] Moreover, the controller 10 acquires the maximum value of the driving performance abnormality degree in the most recent predetermined time from the gaze abnormality degree buffer (step S32).

[0080] Furthermore, the controller 10 acquires the average value of the driving risk in the most recent predetermined time from the risk buffer (step S33).

[0081] Next, the controller 10 calculates an overall abnormality degree from the acquired maximum gaze abnormality degree, maximum driving operation abnormality degree, and average value of the driving risk (step S34). As described above, for example, in a three-dimensional orthogonal coordinate system with the maximum gaze abnormality degree, maximum driving operation abnormality degree, and average value of the driving risk as axes, the controller 10 calculates an overall abnormality degree vector and its magnitude as the overall abnormality degree.

[0082] Next, the controller 10 judges whether or not the total abnormality degree calculated in step S34 is equal to or greater than a predetermined threshold value (step S35).

[0083] When it is determined that the total abnormality degree is equal to or greater than the predetermined threshold value (step S35: Yes), the controller 10 determines whether or not the driving risk increase flag is TRUE (step S36).

[0084] When it is determined that the driving risk increase flag is TRUE (step S36: Yes), the controller 10 determines that the driver is in an abnormality premonition state, that is, detects the abnormality premonition state of the driver (step S37).

[0085] On the other hand, if the total abnormality degree is not determined to be equal to or greater than the predetermined threshold in step S35 (i.e., if the total abnormality degree is less than the threshold) (step S35: No), or if the driving risk increase flag is not determined to be TRUE in step S36 (i.e., if the driving risk increase flag is FALSE) (step S36: No), the controller 10 determines that the driver is not in an abnormality premonition state. In other words, it does not detect the driver's abnormality premonition state (step S38). After processing step S37 or S38, the controller 10 terminates the abnormality premonition detection processing.

[0086] In the above embodiment, the gaze prediction value is calculated by inputting the head behavior information, driving operation information, driving environment information, and vehicle state information into the gaze model, but the gaze prediction value may be calculated by other calculation methods. For example, the gaze prediction value may be stored in advance for each driving environment and vehicle state, and the gaze prediction value according to the current driving environment and vehicle state may be obtained. Alternatively, the gaze prediction value may be obtained by machine learning the actual measured value of the driver's gaze in a healthy state.

[0087] In the above embodiment, the gaze abnormality degree and the driving operation abnormality degree are calculated using the Mahalanobis distance, but the gaze abnormality degree and the driving operation abnormality degree may be calculated using other calculation methods. For example, the gaze abnormality degree may be a value obtained by normalizing the difference between the actual measured value and the predicted value of each of the saccade frequency and amplitude. Also, the driving operation abnormality degree may be a value obtained by normalizing the difference between the actual measured value and the predicted value of each of the steering wheel operation amount and the pedal operation amount.

[0088] Furthermore, in the above-described embodiment, the amplitude and frequency of saccades are used as the feature amount representing the movement of the line of sight. However, either the amplitude or the frequency of saccades may be used.

[0089] In addition, in the above-described embodiment, the operation amounts of the steering wheel, accelerator pedal, and brake pedal are used as feature quantities representing driving operations. However, any one or two of the operation amounts of the steering wheel, accelerator pedal, and brake pedal may also be used.

[0090] In the above embodiment, a three-dimensional vector having the maximum gaze abnormality degree, the maximum driving operation abnormality degree, and the average value of the driving risk as components is used as the overall abnormality degree vector, and the magnitude of the overall abnormality degree vector is calculated as the overall abnormality degree, but the overall abnormality degree may be calculated by other calculation methods. For example, the overall abnormality degree may be calculated as the sum or average value of the maximum gaze abnormality degree, the maximum driving operation abnormality degree, and the average value of the driving risk.

[0091] [Actions and Effects] Next, the effects of the driver abnormality sign detection device 100 of the above-described embodiment will be described.

[0092] The controller 10 calculates a gaze abnormality degree indicating the degree to which the actual measured value of the gaze deviates from the predicted value of the gaze, calculates a driving operation abnormality degree indicating the degree to which the actual measured value of the driving operation deviates from the predicted value of the driving operation, calculates the driving risk of the vehicle 1, and detects the driver's abnormality premonition state when the overall abnormality degree obtained by combining the gaze abnormality degree, the driving operation abnormality degree, and the driving risk is equal to or greater than a predetermined threshold value. Therefore, the abnormality premonition state can be detected early and with high accuracy even at a sufficiently early stage before the driver reaches an abnormal state where driving is difficult, such that a deterioration in the driver's individual driving function cannot be detected.

[0093] In addition, the controller 10 acquires the maximum gaze abnormality degree and the maximum driving operation abnormality degree in the most recent predetermined time, and calculates the overall abnormality degree by combining the maximum gaze abnormality degree, the maximum driving operation abnormality degree, and the driving risk, so that even if the change in the driver's checking behavior or operating behavior is temporary and does not continue, the abnormality premonition state of the driver can be detected based on the overall abnormality degree that reflects the change. Therefore, the abnormality premonition state can be detected more reliably.

[0094] In addition, the controller 10 obtains the average value of the driving risk in the most recent predetermined time, and calculates the overall abnormality degree by combining the maximum value of the gaze abnormality degree, the maximum value of the driving operation abnormality degree, and the average value of the driving risk, so that it is possible to detect the driver's abnormality premonition state based on the overall abnormality degree that reflects an average increase in the driving risk, not a temporary increase, as a result of a change in the driver's driving behavior. Therefore, it is possible to prevent erroneous detection of an abnormality premonition state due to an accidental change in driving behavior that is not caused by the driver's illness, and it is possible to more accurately detect an abnormality premonition state.

[0095] Furthermore, when the total abnormality degree is equal to or greater than a predetermined threshold and an increase in driving risk is detected, the controller 10 determines that an abnormality premonition state of the driver has been detected, and therefore can detect an abnormality premonition state of the driver when the driving risk increases as a result of a change in the driver's driving behavior and the possibility of lane departure or approaching an obstacle increases. Therefore, it is possible to prevent erroneous detection of an abnormality premonition state due to a temporary change in driving behavior not caused by a driver's illness, and to more accurately detect an abnormality premonition state.

[0096] In addition, the controller 10 calculates the degree of deviation of the actual saccade value from the predicted saccade value as the degree of gaze abnormality, and therefore, the change in confirmation behavior due to a decline in the driver's perceptual function, attention function, or motor function can be grasped using the change in saccade.

[0097] In addition, the controller 10 calculates the degree of deviation of the actual measured value of the operation amount of the steering wheel, accelerator pedal, and / or brake pedal from the predicted value of the operation amount as the degree of driving operation abnormality, so that changes in operating behavior due to a decline in the driver's perception function, attention function, or motor function can be grasped using the changes in the operation amount of the steering wheel, accelerator pedal, and / or brake pedal. [Explanation of symbols]

[0098] 1 vehicle 10 Controller 10a processor 10b Memory 100 Driver Abnormality Prediction Device 21 Exterior camera 22 Radar 23 Navigation System 24 Positioning System 25 Vehicle speed sensor 26 Acceleration Sensor 27 Yaw rate sensor 28 Steering angle sensor 29 Steering torque sensor 30 Accelerator Sensor 31 Brake sensor 32 In-car camera 33 PCM 34 DSC 35 EPS 36 Display 37 Speaker

Claims

1. A driver abnormality sign detection device that detects an abnormality sign state of a driver who drives a vehicle, comprising: A gaze detection device for detecting the gaze of the driver; A driving operation detection device for detecting a driving operation by the driver; A driving environment information acquisition device that acquires driving environment information of a vehicle; and a controller configured to detect an abnormality predictive state of the driver based on gaze information acquired from the gaze detection device, driving operation information acquired from the driving operation detection device, and driving environment information acquired from the driving environment information acquisition device, The controller: acquiring a predicted value of the gaze based on the driving environment information, acquiring an actual measurement value of the gaze based on the gaze information, and calculating a gaze abnormality degree indicating a degree to which the actual measurement value of the gaze deviates from the predicted value of the gaze, obtaining a predicted value of the driving operation based on the driving environment information, obtaining an actual value of the driving operation based on the driving operation information, and calculating a driving operation abnormality degree indicating a degree to which the actual value of the driving operation deviates from the predicted value of the driving operation; Calculating a driving risk of the vehicle based on the driving environment information; Calculating an overall abnormality degree by combining the gaze abnormality degree, the driving operation abnormality degree, and the driving risk; When the overall abnormality degree is equal to or greater than a predetermined threshold, it is determined that an abnormality predictor state of the driver has been detected. Driver abnormality prediction device.

2. The controller: A maximum value of the gaze abnormality degree and a maximum value of the driving operation abnormality degree in a recent predetermined time period are obtained; The maximum value of the gaze abnormality degree, the maximum value of the driving operation abnormality degree, and the driving risk are combined to calculate a total abnormality degree. The driver abnormality sign detection device according to claim 1 .

3. The controller: Obtain an average value of the driving risk for a recent specified time period; The maximum value of the gaze abnormality degree, the maximum value of the driving operation abnormality degree, and the average value of the driving risk are combined to calculate a total abnormality degree. The driver abnormality sign detection device according to claim 2 .

4. The controller is configured to determine that an abnormality predictor state of the driver has been detected when the overall abnormality degree is equal to or greater than a predetermined threshold and an increase in the driving risk is detected. The driver abnormality sign detection device according to any one of claims 1 to 3.

5. The controller: Obtaining a predicted value of the driver's saccade based on the driving environment information; acquiring an actual measurement value of the driver's saccade based on the line of sight information; A degree of deviation of the actual saccade value from the predicted saccade value is calculated as the gaze abnormality degree. The driver abnormality sign detection device according to any one of claims 1 to 3.

6. The controller: obtaining a predicted value of an operation amount of a steering wheel, an accelerator pedal, and / or a brake pedal based on the driving environment information; acquiring an actual value of the operation amount based on the driving operation information; The degree to which the actual measured value of the operation amount deviates from the predicted value of the operation amount is calculated as the driving operation abnormality degree. The driver abnormality sign detection device according to any one of claims 1 to 3.

Images