Driver abnormality omen detection device

Abstract

Provided is a driver abnormality indication detection device capable of detecting an abnormality indication state of a driver with high accuracy in a stage sufficiently earlier than the driver becomes an abnormal state in which driving is difficult. A driver abnormality indication detection device (100) is provided with a controller (10) configured so as to detect an abnormality indication state of a driver on the basis of line-of-sight information acquired from a line-of-sight detection device, driving operation information acquired from a driving operation detection device, and traveling environment information acquired from a traveling environment information acquisition device. The controller calculates a line-of-sight abnormality indicating a degree of deviation of the measured value of the line-of-sight from the predicted value of the line-of-sight, calculates a driving operation abnormality indicating a degree of deviation of the measured value of the driving operation from the predicted value of the driving operation, and calculates a driving risk of the vehicle. When a comprehensive abnormality degree obtained by combining the line-of-sight abnormality degree, the driving operation abnormality degree, and the driving risk is equal to or greater than a predetermined threshold value, an abnormality indication state of the driver is detected.

Description

Technical Field

[0001] The invention relates to a driver abnormal sign detection device for detecting an abnormal sign state of a driver during vehicle driving. Background Art

[0002] In recent years, the development of driver abnormality response systems that detect abnormalities and automatically stop the vehicle when the driver is in a state where he cannot drive safely has been promoted. For example, if the driver's abnormality is detected by detecting the driver's posture distortion, the vehicle is gradually decelerated while maintaining the driving route, and the vehicle is automatically stopped when possible by approaching the shoulder of the road.

[0003] In order to avoid leaving the driving route, contact with obstacles, etc. when the driver's abnormality occurs and to stop the vehicle safely, it is preferred to prevent false detection and shorten the time from the driver's abnormality to detection as much as possible. Therefore, a driver abnormality determination device (for example, with reference to patent document 1) for the purpose of early determination of the abnormality of the driver driving the vehicle is proposed. In the device described in patent document 1, the abnormality determination unit receives the output of the involuntary function detection unit, the basic function detection unit and the prediction function detection unit, and determines the driver's abnormality based on the detection object items and determination conditions corresponding to the driving scene identified by the driving scene recognition unit. Specifically, on the basis of classifying the driver's driving function into involuntary function, basic function and prediction function, the combination of functions suitable for detecting the driver's abnormality is determined according to the driving scene (for example, when driving on the street, when driving on the highway, etc.). And, the driver's abnormality is determined based on the state of each function included in the combination.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2021-167163

[0007] Technical problem to be solved by the invention

[0008] However, the inventors of the present application did not use a driving simulator but used an actual vehicle to conduct a driving test based on healthy people and patients, and found that in the above-mentioned prior art, the abnormality of the patient cannot always be detected early. As a reason, it can be considered that the actual driving environment is diverse compared to the simulator, so even in one of the above-mentioned driving scenes, the function suitable for detecting the abnormality of the driver is not always the same, and it is difficult to detect the abnormality only by the prescribed combination of driving functions set for each driving scene. In addition, it can also be considered that when actually driving a vehicle, the patient also drives carefully as hard as possible, so the change of the driver's behavior accompanied by the reduction of the driving function will not continue temporarily, and it is difficult to detect the reduction of the driving function with good accuracy in a short time. Therefore, in the above-mentioned prior art, in order to reliably prevent misdetection and detect abnormality, it is necessary to delay the detection time until the reduction of the driver's driving function is clearly and continuously manifested. That is, there is room for detecting the abnormal signs of the driver earlier and with high accuracy. Summary of the invention

[0009] The present invention has been made to solve such a problem, and an object of the present invention is to provide a driver abnormality sign detection device capable of detecting a driver's abnormality sign state with high accuracy at a stage sufficiently early before the driver enters an abnormal state where driving becomes difficult.

[0010] Technical means for solving technical problems

[0011] In order to solve the above technical problems, the driver abnormal sign detection device of the present invention detects the abnormal sign state of the driver driving the vehicle, and comprises: a line of sight detection device, which detects the line of sight of the driver; a driving operation detection device, which detects the driving operation of the driver; a driving environment information acquisition device, which acquires the driving environment information of the vehicle; and a controller, which is configured to detect the abnormal sign state of the driver based on the line of sight information acquired from the line of sight detection device, the driving operation information acquired from the driving operation detection device, and the driving environment information acquired from the driving environment information acquisition device. The controller is configured to detect the abnormal sign state of the driver based on the driving environment information to obtain a predicted value of the line of sight, to obtain a measured value of the line of sight based on the line of sight information, and to calculate a line of sight abnormality degree, wherein the line of sight abnormality degree indicates the degree to which the measured value of the line of sight deviates from the predicted value of the line of sight, to obtain a predicted value of the driving operation based on the driving environment information, to obtain a measured value of the driving operation based on the driving operation information, and to calculate a driving operation abnormality degree, wherein the driving operation abnormality degree indicates the degree to which the measured value of the driving operation deviates from the predicted value of the driving operation, to calculate a driving risk of the vehicle based on the driving environment information, to calculate a comprehensive abnormality degree by synthesizing the line of sight abnormality degree, the driving operation abnormality degree, and the driving risk, and to determine that an abnormal sign state of the driver is detected when the comprehensive abnormality degree is above a prescribed threshold value.

[0012] According to the present invention constructed as described above, the controller calculates the degree of sight abnormality indicating the degree of deviation of the measured value of sight from the predicted value of sight, calculates the degree of driving operation abnormality indicating the degree of deviation of the measured value of driving operation from the predicted value of driving operation, calculates the driving risk of the vehicle, and detects the driver's abnormal sign state when the comprehensive abnormality obtained by synthesizing the sight abnormality, driving operation abnormality and driving risk is above a predetermined threshold value, thereby being able to detect the abnormal sign state by comprehensively determining the changes in the driver's confirmation actions and operating actions and the increase in driving risk. Therefore, the abnormal sign state can be detected early and with high accuracy at a sufficiently early stage before the driver reaches an abnormal state where driving difficulty occurs, such as a reduction in the individual driving function of the driver cannot be detected.

[0013] In the present invention, preferably, the controller is configured to obtain the maximum value of the line of sight abnormality and the maximum value of the driving operation abnormality within the most recent specified time, and synthesize the maximum value of the line of sight abnormality, the maximum value of the driving operation abnormality and the driving risk to calculate the comprehensive abnormality.

[0014] According to the present invention thus constituted, even when the change in the driver's confirmation action or operation action does not continue temporarily, the driver's abnormality sign state can be detected based on the comprehensive abnormality degree reflecting the change. Therefore, the abnormality sign state can be detected more reliably.

[0015] In the present invention, preferably, the controller is configured to obtain an average value of driving risks within a recent predetermined time, and to calculate a comprehensive abnormality by synthesizing a maximum value of the sight line abnormality, a maximum value of the driving operation abnormality, and an average value of the driving risks.

[0016] According to the present invention constructed as described above, the driver's abnormal sign state can be detected based on the comprehensive abnormality degree that reflects that the driving risk is not temporarily but evenly increased as a result of the driver's driving behavior change. Therefore, it is possible to prevent the erroneous detection of the abnormal sign state caused by the occasional change in driving behavior that is not caused by the driver's disease, and the abnormal sign state can be detected more accurately.

[0017] In the present invention, preferably, the controller is configured to determine that the abnormality sign state of the driver is detected when the comprehensive abnormality degree is equal to or greater than a predetermined threshold value and an increase in the travel risk is detected.

[0018] According to the present invention constructed in this way, when the driving risk increases as a result of the driver's driving behavior change, and the possibility of deviating from the driving route and approaching an obstacle increases, the driver's abnormal sign state can be detected. Therefore, it is possible to prevent the erroneous detection of the abnormal sign state caused by the temporary change of driving behavior that is not caused by the driver's illness, and the abnormal sign state can be detected more accurately.

[0019] In the present invention, preferably, the controller is configured to obtain a predicted value of the driver's glance based on driving environment information, obtain an actual value of the driver's glance based on line of sight information, and calculate the degree of deviation of the actual value of the glance from the predicted value of the glance as a line of sight abnormality.

[0020] According to the present invention constructed in this way, the controller calculates the degree to which the measured value of the glance deviates from the predicted value of the glance as the degree of visual abnormality, so that the change in the glance can be used to grasp the change in confirmation action caused by the reduction of the driver's perception function, attention function or motor function.

[0021] In the present invention, preferably, the controller is configured to obtain a predicted value of the operation amount of the steering wheel, accelerator pedal and / or brake pedal based on driving environment information, obtain an actual value of the operation amount based on driving operation information, and calculate the degree of deviation of the actual value of the operation amount from the predicted value of the operation amount as the driving operation abnormality.

[0022] According to the present invention thus constituted, changes in the operation behavior due to reduction in the driver's perception function, attention function, or motor function can be grasped using changes in the operation amount of the steering wheel, accelerator pedal, and / or brake pedal.

[0023] Effects of the Invention

[0024] According to the driver abnormality sign detection device of the present invention, it is possible to detect the driver's abnormality sign state with high accuracy at a stage sufficiently early before the driver enters an abnormal state where driving becomes difficult. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is an explanatory diagram of a vehicle equipped with the driver abnormality sign detection device according to the embodiment of the present invention.

[0026] Figure 2 This is a block diagram of a driver abnormality sign detection device according to an embodiment of the present invention.

[0027] Figure 3 This is a control block diagram of abnormality sign detection according to an embodiment of the present invention.

[0028] Figure 4 This is a diagram showing the comprehensive abnormality degree in the embodiment of the present invention using a three-dimensional orthogonal coordinate system having the sight line abnormality degree, the driving operation abnormality degree, and the travel risk as coordinate axes, respectively.

[0029] Figure 5 This is a flowchart of the visual line abnormality degree calculation process according to the embodiment of the present invention.

[0030] Figure 6 This is a flowchart of a driving operation abnormality degree calculation process according to the embodiment of the present invention.

[0031] Figure 7 This is a flowchart of a travel risk calculation process according to the embodiment of the present invention.

[0032] Figure 8 This is a flowchart of abnormality sign detection processing according to the embodiment of the present invention.

[0033] Explanation of symbols

[0034] 1 Vehicle

[0035] 10 Controller

[0036] 10a processor

[0037] 10b memory

[0038] 100 Driver abnormality warning detection device

[0039] 21 External cameras

[0040] 22 Radar

[0041] 23Navigation system

[0042] 24 Positioning System

[0043] 25 Vehicle speed sensor

[0044] 26 Acceleration sensor

[0045] 27 Yaw rate sensor

[0046] 28 Steering angle sensor

[0047] 29 Steering torque sensor

[0048] 30 Acceleration sensor

[0049] 31 Brake sensor

[0050] 32 In-car cameras

[0051] 33PCM

[0052] 34DSC

[0053] 35EPS

[0054] 36 Displays

[0055] 37 speakers. DETAILED DESCRIPTION

[0056] Hereinafter, a driver abnormality sign detection device according to an embodiment of the present invention will be described with reference to the drawings.

[0057] [System Structure]

[0058] First, refer to Figure 1 and Figure 2 , the structure of the driver abnormality sign detection device based on this embodiment is described. Figure 1 This is an illustration of a vehicle equipped with a driver abnormality sign detection device. Figure 2 This is a block diagram of a driver abnormality sign detection device.

[0059] The vehicle 1 according to this embodiment includes: a driving force source 2 such as an engine or an electric motor that outputs driving force, a transmission 3 that transmits the driving force output from the driving force source 2 to driving wheels, a brake 4 that applies braking force to the vehicle 1, and a steering device 5 for steering the vehicle 1.

[0060] The driver abnormality sign detection device 100 is configured to detect the abnormality sign state of the driver of the vehicle 1 and perform control of the vehicle 1 and driving assistance control as needed. Figure 2 As shown, the driver abnormality sign detection device 100 includes a controller 10, a plurality of sensors, a plurality of control systems, and a plurality of information output devices.

[0061] Specifically, the plurality of sensors include an exterior camera 21 for acquiring information about the driving environment of the vehicle 1, a radar 22, a navigation system 23 for detecting the position of the vehicle 1, and a positioning system 24. In addition, the plurality of sensors include a vehicle speed sensor 25 for detecting the movement of the vehicle 1 based on the driving operation of the driver, an acceleration sensor 26, a yaw rate sensor 27, a steering angle sensor 28, a steering torque sensor 29, an acceleration sensor 30, and a brake sensor 31. In addition, the plurality of sensors include an interior camera 32 for detecting the driver's line of sight. The plurality of control systems include a powertrain control module (PCM) 33 for controlling the driving force source 2 and the transmission 3, a dynamic stability control system (DSC) 34 for controlling the driving force source 2 and the brake 4, and an electric power steering system (EPS) 35 for controlling the steering device 5. The plurality of information output devices include a display 36 for outputting image information and a speaker 37 for outputting sound information.

[0062] In addition, other sensor types may also include: peripheral sonar for measuring the distance and position of surrounding structures relative to the vehicle 1, corner radar for measuring the approach of surrounding structures at the four corners of the vehicle 1, and various sensors for detecting the driver's status (for example, heart rate sensors, electrocardiogram sensors, steering wheel grip force sensors, etc.).

[0063] The controller 10 performs various operations based on the signals received from the plurality of sensors, sends control signals for properly operating the driving force source 2, the transmission 3, the brake 4, and the steering device 5 to the PCM 33, the DSC 34, and the EPS 35, and sends control signals for outputting desired information to the display 36 and the speaker 37. The controller 10 is composed of a computer having one or more processors 10a (typically a CPU); a memory 10b (ROM, RAM, etc.) storing various programs and data; and an input / output device.

[0064] The vehicle exterior camera 21 captures the surroundings of the vehicle 1 and outputs image data. The controller 10 determines the position and speed of an object (e.g., a preceding vehicle, a parked vehicle, a pedestrian, a driving road, a line (driving route boundary line, white line, yellow line), a traffic signal, a traffic sign, a stop line, an intersection, an obstacle, etc.) based on the image data received from the vehicle exterior camera 21. In addition, the vehicle exterior camera 21 is equivalent to an example of the "driving environment information acquisition device" in the present invention, and the information acquired by the vehicle exterior camera 21 is equivalent to an example of the "driving environment information" in the present invention.

[0065] The radar 22 measures the position and speed of an object (especially, a preceding vehicle, a parked vehicle, a pedestrian, a fallen object on the road, etc.). As the radar 22, for example, a millimeter wave radar can be used. The radar 22 transmits radio waves in the direction of travel of the vehicle 1, and receives reflected waves generated when the transmitted waves are reflected by the object. Then, the radar 22 measures the distance between the vehicle 1 and the object (for example, the distance between the vehicles), and the relative speed of the object relative to the vehicle 1 based on the transmitted waves and the received waves. In addition, in the present embodiment, a laser radar, an ultrasonic sensor, etc. may be used instead of the radar 22 to measure the distance and relative speed to the object. In addition, a plurality of sensor types may be used to constitute a position and speed measuring device. In addition, the radar 22 is equivalent to an example of a "driving environment information acquisition device" in the present invention, and the information acquired by the radar 22 is equivalent to an example of "driving environment information" in the present invention.

[0066] The navigation system 23 stores map information internally and can provide the map information to the controller 10. The controller 10 determines the roads, intersections, traffic signals, buildings, etc. that exist around the vehicle 1 (especially in the direction of travel) based on the map information and the current vehicle position information. The map information can also be stored in the controller 10. The positioning system 24 is a GPS system and / or a gyroscope system that detects the position of the vehicle 1 (current vehicle position information). In addition, the navigation system 23 and the positioning system 24 are also equivalent to an example of the "driving environment information acquisition device" in the present invention, and the information acquired by the navigation system 23 and the positioning system 24 is equivalent to an example of the "driving environment information" in the present invention.

[0067] The vehicle speed sensor 25 detects the speed of the vehicle 1 based on, for example, the rotation speed of the wheels and the drive shaft. The acceleration sensor 26 detects the acceleration of the vehicle 1. The acceleration includes the acceleration in the front-rear direction and the acceleration in the lateral direction (i.e., lateral acceleration) of the vehicle 1. In addition, in this specification, the acceleration includes not only the rate of change of the speed in the direction of increasing speed, but also the rate of change of the speed in the direction of decreasing speed (i.e., deceleration). The yaw rate sensor 27 detects the yaw rate of the vehicle 1. That is, the vehicle speed sensor 25, the acceleration sensor 26, and the yaw rate sensor 27 obtain vehicle state information related to the state of the vehicle 1.

[0068] The steering angle sensor 28 detects the rotation angle (steering angle) of the steering wheel of the steering device 5. The steering torque sensor 29 detects the torque (steering torque) applied to the steering shaft via the steering wheel. The acceleration sensor 30 detects the amount of depression of the accelerator pedal. The brake sensor 31 detects the amount of depression of the brake pedal. That is, the steering angle sensor 28, the steering torque sensor 29, the acceleration sensor 30, and the brake sensor 31 detect the driving operation of the driver, which is equivalent to an example of the "driving operation detection device" in the present invention, and the information obtained by the steering angle sensor 28, the steering torque sensor 29, the acceleration sensor 30, and the brake sensor 31 is equivalent to an example of the "driving operation information" in the present invention.

[0069] The in-vehicle camera 32 captures the driver and outputs image data. The controller 10 detects the driver's head movement and line of sight direction based on the image data received from the in-vehicle camera 32. In addition, the in-vehicle camera 32 is equivalent to an example of the "line of sight detection device" in the present invention, and the information obtained by the in-vehicle camera 32 is equivalent to an example of the "line of sight information" in the present invention.

[0070] The PCM 33 controls the driving force source 2 of the vehicle 1 to adjust the driving force of the vehicle 1. For example, the PCM 33 controls the spark plugs, fuel injection valves, throttle valves, variable valve mechanisms, transmission 3, inverters that supply power to the motor, etc. When the vehicle 1 needs to be accelerated or decelerated, the controller 10 sends a control signal to the PCM 33 to adjust the driving force.

[0071] The DSC 34 controls the driving force source 2 and the brake 4 of the vehicle 1 to perform deceleration control and posture control of the vehicle 1. For example, the DSC 34 controls the hydraulic pump and valve unit of the brake 4, and controls the driving force source 2 via the PCM 33. When deceleration control and posture control of the vehicle 1 are required, the controller 10 sends a control signal to the DSC 34 to adjust the driving force or generate a braking force.

[0072] The EPS 35 controls the steering device 5 of the vehicle 1. For example, the EPS 35 controls a motor that applies torque to a steering shaft of the steering device 5. When the traveling direction of the vehicle 1 needs to be changed, the controller 10 sends a control signal to the EPS 35 to change the steering direction.

[0073] The display 36 is provided in front of the driver in the vehicle interior, and displays image information to the driver. For example, a liquid crystal display or a head-up display can be used as the display 36. The speaker 37 is provided in the vehicle interior, and outputs various sound information.

[0074] [Overview of driver abnormality sign detection]

[0075] Next, refer to Figure 3, the basic concept of the driver abnormality sign detection executed by the above-mentioned controller 10 in this embodiment is explained. Figure 3 This is a control block diagram of abnormality sign detection according to the present embodiment.

[0076] In the abnormal omen state before the driver reaches the abnormal state of driving difficulty, it can be considered that the driver's driving function begins to decrease at least temporarily due to a mild disease, aging, etc. If the driver's driving function decreases, the driver's driving actions, such as the confirmation action of confirming the driving environment and the operation action of performing the driving operation of the vehicle 1, will change accordingly. Moreover, it can also be considered that as a result of the change in driving action, the risk (driving risk) of deviating from the driving route and approaching obstacles increases. Therefore, the inventors of the present application believe that by comprehensively determining the changes in these driving actions and the increase in driving risks, the abnormal omen state can be detected early and with high accuracy without detecting a significant decrease in individual driving functions.

[0077] Here, the driver's driving function includes attention functions such as a perception function of perceiving objects existing in the driving environment, a function of simultaneously observing multiple objects in the driving environment (distributed attention function), a function of selecting to observe from multiple objects (selective attention function), a function of switching objects (conversion attention function), and a function of continuously observing objects (continuous attention function). In addition, the driver's driving function also includes motor functions required to operate the steering wheel, accelerator pedal, brake pedal, etc. of the vehicle 1.

[0078] According to the results of the research of the inventors of this application, the following insights are obtained: if the perception function, attention function, or motor function is reduced, the movement of the line of sight will change as a change in the driver's confirmation action. For example, if the perception function is reduced due to visual field defects, the range of moving the line of sight will become narrower. In addition, if the attention function is reduced, the frequency of moving the line of sight will decrease, or the moving distance of the line of sight will become shorter. Moreover, if the motor function is reduced due to paralysis of the hands and feet, the driver will pay attention to the condition of the paralyzed hands and feet, and therefore will turn his line of sight in the direction of the paralyzed hands and feet. Therefore, by taking the movement of the line of sight corresponding to the driving environment and the vehicle state in a state without disease (healthy state) as a reference, the abnormality of the movement of the driver's line of sight is obtained, and the change of the confirmation action caused by the reduction of the driver's perception function, attention function, or motor function can be grasped.

[0079] In addition, the following insights have been obtained: if the perception function, attention function, or motor function is reduced, the operation of the steering wheel, accelerator pedal, and brake pedal will change as the operation action of operating the steering wheel, accelerator pedal, and brake pedal changes. For example, if the motor function is reduced, the operation of the steering wheel and the operation of the pedal will be delayed compared to the healthy state. In addition, if the perception function and attention function are reduced, the driver's recognition of the driving environment, which is the basis of the driving operation, will be affected. Therefore, for example, the position of the own vehicle within the driving route cannot be well grasped or the delay or omission of the discovery of the object occurs, resulting in the operation of the steering wheel and the operation of the pedal becoming unstable compared to the healthy state. Therefore, by taking the driving operation corresponding to the driving environment and the vehicle state in the healthy state as a reference, the abnormality of the driver's driving operation can be obtained, and the change of the operation action caused by the reduction of the driver's perception function, attention function, or motor function can be grasped.

[0080] In addition, when the driver's perception function, attention function, or motor function is reduced, the risk of driving such as leaving the driving route and approaching obstacles increases due to delays / missing the discovery of the object and unstable driving operations. According to the research of the inventors of this application, in a healthy driver, even if the temporary driving risk increases due to inattention and disorder of operation, the driving risk will be reduced immediately because appropriate correction operations will be performed later. Therefore, the average value of the driving risk remains at a low level. In contrast, if the driver has reduced driving function, the driving risk gradually increases because it becomes difficult to perform driving operations for reducing the driving risk. As a result, the average value of the driving risk becomes higher than that of healthy people, and the rising trend of the driving risk continues. Therefore, by obtaining the average value and rising trend of the driving risk, the influence of the change in driving behavior caused by the reduction of the driver's motor function, perception function, or attention function can be grasped.

[0081] Therefore, the controller 10 of the present embodiment calculates the abnormality of the driver's line of sight (line of sight abnormality) based on the line of sight information, driving operation information, driving environment information, vehicle state information, and head movement information, calculates the abnormality of the driver's driving operation (driving operation abnormality) based on the driving operation information, driving environment information, and vehicle state information, and calculates the driving risk based on the driving environment information and vehicle state information. In addition, the controller 10 is configured to calculate the comprehensive abnormality of the driver's state (comprehensive abnormality) based on these line of sight abnormality, driving operation abnormality, and driving risk, and detect that the driver is in an abnormal sign state based on the comprehensive abnormality.

[0082] Specifically, if Figure 3As shown, the controller 10 obtains line of sight information and head movement information based on the signal received from the in-vehicle camera 32, obtains driving operation information based on the signal received from the steering angle sensor 28, the steering torque sensor 29, the acceleration sensor 30 and the brake sensor 31, and obtains driving environment information and vehicle status information based on the signal received from the sensor class including the external camera 21, the radar 22, the navigation system 23, the positioning system 24, the vehicle speed sensor 25, the acceleration sensor 26, and the yaw rate sensor 27.

[0083] The controller 10 inputs the acquired driving operation information, driving environment information, vehicle state information, and head movement information into the sight line model, and calculates the predicted value of the sight line movement of the driver in a healthy state (sight line prediction value). As a feature quantity representing the movement of the sight line, for example, the amplitude and frequency of the glance can be used.

[0084] Here, saccade refers to the jumping eye movement used to capture the visual target in the fovea of ​​the retina, and refers to the eye movement that moves the line of sight from the fixation point where it stays for a specified time to the next fixation point. The amplitude of the saccade refers to the amount of movement of the driver's line of sight from the fixation point to the next fixation point, and the frequency of the saccade refers to the number of times the line of sight moves from the fixation point to the next fixation point within a specified time.

[0085] The movement of sight is affected by the driver's steering operation, vehicle status (such as vehicle speed), driving environment (such as illumination, saliency (Japanese: サリエンシー) deviation, etc.), driver's head movement, etc. Here, saliency refers to the characteristics that indicate the ease of attracting people's attention, and is a visual feature determined by the temporal and spatial configuration of color, brightness, movement, etc. That is, the area with high saliency in the driver's field of vision is an area that easily attracts the driver's attention due to, for example, large color difference or brightness difference relative to the surrounding area or large movement.

[0086] For example, when the driver performs a steering operation at an intersection or in a continuous curve, the driver focuses his sight on the direction in which the vehicle turns, so the deviation of the sight tends to decrease. In addition, when the vehicle speed is high, such as on a highway, the driver's sight tends to be concentrated in the direction of travel. In addition, when the illumination of the driving environment is low, such as at night, the driver's sight also tends to be concentrated in the direction of travel. In addition, when the deviation of the saliency of the surroundings of the vehicle is small (that is, when the area with high saliency is concentrated in a part of the driver's field of vision), the driver's sight tends to be concentrated in the area with high saliency. Moreover, the driver's sight tends to be biased in the direction in which the driver's face is facing.

[0087] Therefore, the sight line model is constructed so that when the characteristic quantities such as the steering operation, vehicle state, driving environment, and driver's head movement that affect the movement of the sight line are input as parameters, the sight line prediction value of the driver in a healthy state taking these influences into account is output. Specifically, for example, the reference value of the scanning frequency and amplitude in a healthy state and the correction map of each characteristic quantity that affects the movement of the sight line are pre-set in the sight line model, and each characteristic quantity is, for example, the standard deviation of the steering wheel operation amount (steering angle), the vehicle speed, the illumination of the driving environment, the deviation of the saliency, and the standard deviation of the orientation of the driver's face. Based on these correction maps, the reference value is corrected according to each of the input characteristic quantities to obtain the sight line prediction value reflecting the influence of the steering operation, the vehicle state, the driving environment, and the driver's head movement.

[0088] The controller 10 calculates a line of sight abnormality (line of sight abnormality calculation) indicating the degree to which the line of sight measured value deviates from the line of sight predicted value based on the measured value of the movement of the driver's line of sight (line of sight measured value) determined according to the acquired line of sight information and the line of sight predicted value output from the line of sight model constructed as described above.

[0089] For example, controller 10 calculates the number of times of the sweep per unit time as the measured value of the sweep frequency based on the number of times of the sweep in the prescribed time (for example 30 seconds). In addition, controller 10 calculates the mean value of the sweep amplitude in the prescribed time (for example 30 seconds) as the measured value of the sweep amplitude. And, accumulate two-dimensional data, this two-dimensional data is the difference of the calculated measured value of the sweep frequency and the predicted value output from the line of sight model as the first variable and the difference of the calculated measured value of the sweep amplitude and the predicted value output from the line of sight model as the second variable. Controller 10 calculates the Mahalanobis distance of the center of gravity (average) of the latest data point of this two-dimensional data and the accumulated data set. There is a correlation between the measured value of the sweep frequency and the difference of the predicted value and the measured value of the sweep amplitude and the predicted value, so by using the Mahalanobis distance like this, the deviation degree of the movement of the line of sight with the healthy state can be represented by an index. And, controller 10 is standardized by dividing the representative value set in advance with the calculated Mahalanobis distance, thereby calculates the line of sight abnormality. As the representative value, the Mahalanobis distance when the driver is in a state of abnormality prediction can be used. That is, when the degree of abnormality of the sight line is 1, the driver is in a state of abnormality prediction.

[0090] Then, the controller 10 accumulates the calculated sight line abnormality in a buffer (sight line abnormality buffer), and acquires the maximum value of the sight line abnormality within a recent predetermined time (for example, 60 seconds) from the sight line abnormality buffer (maximum value acquisition).

[0091] In addition, the controller 10 inputs the acquired driving environment information and vehicle state information into the driving operation prediction model, thereby calculating the predicted value of the driving operation of the driver in a healthy state (driving operation prediction value). As the feature quantity representing the driving operation, for example, the operation amount of the steering wheel (steering angle), the operation amount of the accelerator pedal and the brake pedal (accelerator pedal depression amount and brake pedal depression amount) can be used.

[0092] The driving operation prediction model is configured to output a driving operation prediction value of a healthy driver when the characteristic quantities of the driving environment and the vehicle state required for the driving operation are input as parameters. Specifically, the driving operation prediction model is configured to output the steering wheel operation amount required to drive in the center of the driving route, the accelerator pedal and the brake pedal operation amounts required to follow the vehicle in front or make the vehicle 1 drive at the limited speed as driving operation prediction values, when the characteristic quantities of the driving environment such as the lines in the direction of travel of the vehicle 1, the position of obstacles, the speed limit, the position and speed of the preceding vehicle, and the characteristic quantities of the vehicle state such as the vehicle speed and acceleration are input as parameters.

[0093] The controller 10 calculates a driving operation abnormality degree (driving operation abnormality degree calculation) indicating the degree to which the driving operation measured value deviates from the driving operation predicted value based on the measured value of the driver's driving operation determined according to the acquired driving operation information (driving operation measured value) and the driving operation predicted value output from the driving operation prediction model.

[0094] For example, the controller 10 accumulates two-dimensional data, which uses the difference between the measured value of the steering wheel operation amount and the predicted value output from the driving operation prediction model as the first variable, and uses the difference between the measured value of the operation amount of the accelerator pedal or the brake pedal and the predicted value output from the driving operation prediction model as the second variable. The controller 10 calculates the Mahalanobis distance between the latest data point of the two-dimensional data and the centroid (average) of the accumulated data set. There is a correlation between the difference between the measured value and the predicted value of the steering wheel operation amount and the difference between the measured value and the predicted value of the pedal operation amount, so by using the Mahalanobis distance like this, the degree of deviation from the healthy driving operation can be represented by one indicator. In addition, the controller 10 calculates the driving operation abnormality by dividing the calculated Mahalanobis distance by a preset representative value for standardization. As the representative value, the Mahalanobis distance when the driver is in an abnormal sign state can be used. That is, when the driving operation abnormality is 1, the driver is in an abnormal sign state.

[0095] Then, the controller 10 accumulates the calculated driving operation abnormality in a buffer (driving operation abnormality buffer), and acquires the maximum value of the driving operation abnormality within the most recent predetermined time from the driving operation abnormality buffer (maximum value acquisition).

[0096] In addition, the controller 10 calculates the driving risk based on the acquired driving environment information and vehicle state information. The driving risk is expressed by a numerical value to indicate the possibility of deviating from the driving route, approaching an obstacle, etc. For example, it is set as follows: when it is predicted that the vehicle 1 is located in the center of the driving route and the distance to the obstacle is a safe distance (for example, 1m) or more after a predetermined time (for example, 2 seconds), the driving risk = 0. Similarly, the smaller the predicted value of the distance to the distance to the obstacle after the predetermined time, the closer the driving risk is to 1. When it is predicted that the vehicle 1 deviates from the driving route after the predetermined time and the distance to the obstacle is less than the restricted distance (for example, 0.3m), the driving risk = 1. That is, the controller 10 determines the line of the driving road in the direction of travel of the vehicle 1, the position of the vehicle 1 on the driving road, the position and speed of the obstacle based on the acquired driving environment information. In addition, the current speed and acceleration of the vehicle 1 are determined based on the acquired driving environment information or vehicle state information, and the position of the vehicle 1 after the predetermined time is predicted. Then, based on the predicted position of the vehicle 1 and the line, and the position and speed of the obstacle, the predicted values ​​of the distance until the vehicle 1 deviates from the driving route and the distance to the obstacle after a predetermined time are calculated, and the driving risk is calculated based on the calculated predicted values.

[0097] Then, the controller 10 accumulates the calculated travel risks in a buffer area (risk buffer area), and acquires the average value of the travel risks within the most recent predetermined time from the risk buffer area (average value acquisition).

[0098] The controller 10 calculates the comprehensive abnormality (comprehensive abnormality calculation) based on the maximum value of the abnormality of the line of sight, the maximum value of the abnormality of the driving operation, and the average value of the driving risk. For example, the controller 10 uses the maximum value of the abnormality of the line of sight, the maximum value of the abnormality of the driving operation, and the average value of the driving risk as axes in a three-dimensional orthogonal coordinate system, and uses the three-dimensional vector having the maximum value of the abnormality of the line of sight, the maximum value of the abnormality of the driving operation, and the average value of the driving risk as components as a comprehensive abnormality vector, and calculates the magnitude of the comprehensive abnormality vector as the comprehensive abnormality.

[0099] Figure 4 It is a graph showing the comprehensive abnormality in a three-dimensional orthogonal coordinate system with the abnormality of sight line, the abnormality of driving operation and the driving risk as coordinate axes. Figure 4 As shown in FIG. 1 , when the maximum value of the acquired sight abnormality is set to Gab_max, the maximum value of the driving operation abnormality is set to Oab_max, and the average value of the driving risk is set to R_ave, the size of the comprehensive abnormality vector Tab can be expressed as (Gab_max 2 +Oab_max 2 +R_ave 2 )1 / 2 For example, in Figure 4 In the example, any of the maximum value of the sight abnormality Gab_max1, the maximum value of the driving operation abnormality Oab_max1, and the average value of the driving risk R_ave1 is less than 1, but even in this case, the magnitude of the comprehensive abnormality vector Tab is greater than 1. In addition, as long as any of the obtained maximum value of the sight abnormality, the maximum value of the driving operation abnormality, and the average value of the driving risk is greater than 1, the comprehensive abnormality is greater than 1.

[0100] The controller 10 determines whether the calculated comprehensive abnormality is greater than a predetermined threshold value (threshold value determination). When the magnitude of the comprehensive abnormality vector obtained as described above is calculated as the comprehensive abnormality, the threshold value is, for example, 1. Figure 4 In the example, the sphere with radius 1 corresponds to the threshold.

[0101] Furthermore, when the controller 10 determines that the driving risk is increasing based on the calculated driving risk and the increasing rate is maintained, if the rate reaches 1 within a predetermined time (eg, within 5 seconds), the controller 10 detects an increase in the driving risk (driving risk increase detection).

[0102] Then, when the comprehensive abnormality degree is equal to or greater than the threshold value and an increase in the driving risk is detected, the controller 10 determines that the driver is in an abnormality sign state. In other words, the controller 10 detects that the driver is in an abnormality sign state (abnormality sign detection).

[0103] When an abnormal sign state is detected, the controller 10 sends a control signal to the PCM 33, DSC 34, and EPS 35 for properly operating the driving force source 2, the transmission 3, the brake 4, and the steering device 5, and sends a control signal to the display 36 and the speaker 37 for outputting desired information. For example, the controller 10 vibrates the steering wheel at a prescribed frequency through the EPS 35, and determines that the driver is in an abnormal state based on the driver's response. In addition, the controller 10 outputs a warning display through the display 36, and determines that the driver is in an abnormal state based on the driver's response to the warning display.

[0104] [Abnormal sign detection and processing]

[0105] Next, refer to Figures 5 to 8 , the flow of the detection process of the abnormality sign state by the driver abnormality sign detection device 100 according to the present embodiment will be described. Figure 5 This is a flowchart of the sight line abnormality calculation process. Figure 6 is a flowchart of the driving operation abnormality calculation process, Figure 7 is a flow chart of the driving risk calculation process, Figure 8The process of calculating the degree of abnormality in sight, the process of calculating the degree of abnormality in driving operation, the process of calculating the risk of driving, and the process of detecting the abnormality are respectively started when the power of the vehicle 1 is turned on, and are repeatedly executed in parallel by the controller 10 at a predetermined period (for example, every 0.05 to 0.2 seconds).

[0106] when Figure 5 When the sight line abnormality calculation process starts, the controller 10 detects the sight line of the driver based on the signal received from the in-vehicle camera 32 (step S1). Next, the controller 10 calculates the measured values ​​of the frequency and amplitude of the glance based on the detected sight line of the driver (step S2).

[0107] In addition, the controller 10 obtains head movement information based on the signal received from the in-vehicle camera 32, obtains driving operation information based on the signal received from the steering angle sensor 28, the steering torque sensor 29, the acceleration sensor 30 and the brake sensor 31, and obtains driving environment information and vehicle state information based on the signal received from the sensors including the out-vehicle camera 21, the radar 22, the navigation system 23, the positioning system 24, the vehicle speed sensor 25, the acceleration sensor 26, and the yaw rate sensor 27 (step S3). Then, the controller 10 inputs the information obtained in step S3 into the line of sight model to calculate the predicted value of the frequency and amplitude of the glance (step S4).

[0108] Then, controller 10 calculates sight line abnormality based on the measured value of the frequency of the pan calculated in step S2 and the predicted value of the amplitude and the pan calculated in step S4, and is stored in buffer (step S5).As mentioned above, for example, controller 10 calculates the Mahalanobis distance of the latest data point of the two-dimensional data of the measured value of the pan frequency and the predicted value as the first variable and the measured value of the pan amplitude and the predicted value as the second variable and the center of gravity (average) of the data set accumulated so far.And then, controller 10 standardizes the sight line abnormality with the calculated Mahalanobis distance divided by the pre-set representative value, calculates the sight line abnormality thus.After step S5, controller 10 ends the sight line abnormality calculation process.

[0109] when Figure 6 When the calculation process of the driving operation abnormality starts, the controller 10 detects the driver's driving operation based on the signals received from the steering angle sensor 28, the steering torque sensor 29, the acceleration sensor 30 and the brake sensor 31, specifically, detects the operation amount of the steering wheel, the accelerator pedal and the brake pedal (step S11).

[0110] In addition, the controller 10 obtains driving environment information and vehicle state information based on signals received from sensors including the exterior camera 21, the radar 22, the navigation system 23, the positioning system 24, the vehicle speed sensor 25, the acceleration sensor 26, and the yaw rate sensor 27 (step S12). Then, the controller 10 inputs the information obtained in step S12 into the driving operation prediction model to calculate the predicted value of the driving operation (step S13).

[0111] Next, the controller 10 calculates the driving operation abnormality based on the measured value of the driver's driving operation detected in step S11 and the predicted value of the driving operation calculated in step S13, and stores it in the buffer (step S14). As described above, for example, the controller 10 calculates the Mahalanobis distance between the latest data point of the two-dimensional data and the centroid (average) of the data set accumulated so far, using the difference between the measured value and the predicted value of the steering wheel operation amount as the first variable and the difference between the measured value and the predicted value of the accelerator pedal or the brake pedal operation amount as the second variable. Furthermore, the controller 10 calculates the driving operation abnormality by dividing the calculated Mahalanobis distance by a pre-set representative value for standardization. After step S14, the controller 10 ends the driving operation abnormality calculation process.

[0112] when Figure 7 When the driving risk calculation process starts, the controller 10 obtains the driving environment information and the vehicle status information based on the signals received from the sensors including the external camera 21, the radar 22, the navigation system 23, the positioning system 24, the vehicle speed sensor 25, the acceleration sensor 26, and the yaw rate sensor 27 (step S21).

[0113] Next, the controller 10 calculates the driving risk based on the driving environment information and vehicle status information acquired in step S21, and stores it in the buffer (step S22). As described above, for example, the controller 10 determines the lines of the driving road in the direction of travel of the vehicle 1, the position of the vehicle 1 on the driving road, and the position and speed of obstacles based on the acquired driving environment information. In addition, the current speed and acceleration of the vehicle 1 are determined based on the acquired driving environment information or vehicle status information, and the position of the vehicle 1 after a specified time is predicted. Then, based on the predicted position of the vehicle 1 and the position and speed of the lines and obstacles, the predicted values ​​of the distance of the vehicle 1 until it leaves the driving route and the distance to the obstacle after a specified time are calculated, and the driving risk is calculated based on the calculated predicted values.

[0114] Next, the controller 10 determines whether the driving risk is increasing based on the driving risk calculated in step S22 (step S23). As described above, for example, the controller 10 determines that the driving risk is increasing when it is determined that the driving risk is increasing and reaches 1 within a specified time while the increasing rate is maintained.

[0115] As a result, if it is determined that the driving risk is increasing (step S23: Yes), that is, if the controller 10 detects an increase in the driving risk, the controller 10 sets the driving risk increase flag to TRUE (step S24). On the other hand, if it is not determined that the driving risk is increasing (step S23: No), the controller 10 sets the driving risk increase flag to FALSE (step S25). After the processing of step S24 or S25, the controller 10 ends the driving risk calculation processing.

[0116] when Figure 8 When the abnormal sign detection process starts, the controller 10 obtains the maximum value of the line of sight abnormality degree within the most recent specified time (for example, 60 seconds) from the line of sight abnormality degree buffer (step S31).

[0117] Furthermore, the controller 10 acquires the maximum value of the driving operation abnormality degree within the most recent predetermined time from the sight line abnormality degree buffer (step S32 ).

[0118] Furthermore, the controller 10 acquires the average value of the travel risk within the most recent predetermined time from the risk buffer (step S33 ).

[0119] Next, the controller 10 calculates the comprehensive abnormality according to the maximum value of the abnormality of sight line, the maximum value of the abnormality of driving operation, and the average value of the driving risk (step S34). As described above, for example, the controller 10 calculates the magnitude of the comprehensive abnormality vector as the comprehensive abnormality in a three-dimensional orthogonal coordinate system with the maximum value of the abnormality of sight line, the maximum value of the abnormality of driving operation, and the average value of the driving risk as axes, respectively.

[0120] Next, the controller 10 determines whether the comprehensive abnormality degree calculated in step S34 is equal to or greater than a predetermined threshold value (step S35 ).

[0121] When it is determined that the comprehensive abnormality is equal to or greater than the predetermined threshold value (step S35 : Yes), the controller 10 determines whether the travel risk increase flag is TRUE (step S36 ).

[0122] When it is determined that the travel risk increase flag is TRUE (step S36 : YES), the controller 10 determines that the driver is in an abnormality sign state. In other words, the controller 10 detects that the driver is in an abnormality sign state (step S37 ).

[0123] On the other hand, if it is not determined in step S35 that the comprehensive abnormality is above the predetermined threshold value (i.e., if the comprehensive abnormality is less than the threshold value) (step S35: No), or if it is not determined in step S36 that the driving risk increase flag is TRUE (i.e., if the driving risk increase flag is FALSE) (step S36: No), the controller 10 determines that the driver is not in an abnormal sign state. That is, the abnormal sign state of the driver is not detected (step S38). After the processing of step S37 or S38, the controller 10 ends the abnormal sign detection processing.

[0124] In addition, in the above-mentioned embodiment, the line of sight prediction value is calculated by inputting head movement information, driving operation information, driving environment information, and vehicle state information into the line of sight model, but the line of sight prediction value can also be obtained by other calculation methods. For example, the line of sight prediction value can also be stored in advance for each driving environment and vehicle state, and the line of sight prediction value corresponding to the current driving environment and vehicle state can be obtained. Alternatively, the line of sight prediction value can also be obtained by machine learning of the measured value of the line of sight of a driver in a healthy state.

[0125] In addition, in the above-mentioned embodiment, the use of Mahalanobis distance to calculate the sight line abnormality and the driving operation abnormality is described, but the sight line abnormality and the driving operation abnormality can also be obtained by other calculation methods. For example, the difference between the actual measured value and the predicted value of the scanning frequency and amplitude can also be standardized and synthesized to obtain the value as the sight line abnormality. In addition, the difference between the actual measured value and the predicted value of the operation amount of the steering wheel and the operation amount of the pedal can also be standardized and synthesized to obtain the value as the driving operation abnormality.

[0126] Furthermore, in the above-described embodiment, the amplitude and frequency of saccade are described as feature quantities indicating the movement of the line of sight, but either the amplitude or the frequency of saccade may be used.

[0127] In the above-described embodiment, the operation amounts of the steering wheel, accelerator pedal, and brake pedal are used as feature amounts representing driving operation. However, any one or two of the operation amounts of the steering wheel, accelerator pedal, and brake pedal may be used.

[0128] In addition, in the above-mentioned embodiment, a three-dimensional vector having the maximum value of the sight abnormality, the maximum value of the driving operation abnormality, and the average value of the driving risk as components is used as a comprehensive abnormality vector, and the magnitude of the comprehensive abnormality vector is calculated as the comprehensive abnormality. However, the comprehensive abnormality can also be obtained by other calculation methods. For example, the sum of the maximum value of the sight abnormality, the maximum value of the driving operation abnormality, and the average value of the driving risk can also be calculated as the comprehensive abnormality.

[0129] [Function / Effect]

[0130] Next, the effects of the driver abnormality sign detection device 100 according to the above-mentioned embodiment will be described.

[0131] The controller 10 calculates a sight line abnormality indicating the degree to which the measured value of the sight line deviates from the predicted value of the sight line, calculates a driving operation abnormality indicating the degree to which the measured value of the driving operation deviates from the predicted value of the driving operation, calculates the driving risk of the vehicle 1, and detects the driver's abnormal sign state when the comprehensive abnormality obtained by synthesizing the sight line abnormality, the driving operation abnormality, and the driving risk is above a predetermined threshold value, thereby detecting the abnormal sign state by comprehensively determining the changes in the driver's confirmation actions and operating actions and the increase in the driving risk. Therefore, even at a sufficiently early stage before the driver reaches an abnormal state where the driver's individual driving function is reduced and cannot be detected, the abnormal sign state can be detected early and with high accuracy.

[0132] In addition, the controller 10 obtains the maximum value of the abnormality of sight and the maximum value of the abnormality of driving operation within the most recent prescribed time, and synthesizes the maximum value of the abnormality of sight, the maximum value of the abnormality of driving operation and the driving risk to calculate the comprehensive abnormality, so that even if the change of the driver's confirmation action or operation action is temporarily not continuous, the abnormal omen state of the driver can be detected based on the comprehensive abnormality reflecting the change. Therefore, the abnormal omen state can be detected more reliably.

[0133] In addition, the controller 10 obtains the average value of the driving risk within the most recent specified time, and synthesizes the maximum value of the sight line abnormality, the maximum value of the driving operation abnormality, and the average value of the driving risk to calculate the comprehensive abnormality, so that the driver's abnormal sign state can be detected based on the comprehensive abnormality that reflects that the driving risk is not temporarily but evenly increased as a result of the driver's driving behavior change. Therefore, it is possible to prevent the erroneous detection of the abnormal sign state caused by the occasional change in driving behavior that is not caused by the driver's disease, and the abnormal sign state can be detected more accurately.

[0134] In addition, the controller 10 determines that the driver's abnormal sign state is detected when the comprehensive abnormality degree is above a predetermined threshold value and an increase in driving risk is detected. Therefore, when the driving risk increases as a result of a change in the driver's driving behavior, and the possibility of deviating from the driving route and approaching an obstacle increases, it can be detected as the driver's abnormal sign state. Therefore, it is possible to prevent erroneous detection of abnormal sign states caused by temporary changes in driving behavior that are not caused by the driver's illness, and it is possible to more accurately detect abnormal sign states.

[0135] In addition, the controller 10 calculates the degree of deviation of the measured value of the glance from the predicted value of the glance as the degree of visual aberration, so that the change in the glance can be used to grasp the change in confirmation action caused by the reduction of the driver's perception function, attention function or motor function.

[0136] In addition, the controller 10 calculates the degree to which the measured value of the operation amount of the steering wheel, accelerator pedal and / or brake pedal deviates from the predicted value of the operation amount as the driving operation abnormality, so that the change in the operation amount of the steering wheel, accelerator pedal and / or brake pedal can be used to grasp the change in operational behavior caused by the reduction of the driver's perception function, attention function or motor function.

Claims

1. A driver abnormal sign detection device for detecting an abnormal sign state of a driver driving a vehicle, characterized in that: have: A sight line detection device, the sight line detection device detects the sight line of the driver; a driving operation detection device, the driving operation detection device detecting the driving operation of the driver; A driving environment information acquisition device, the driving environment information acquisition device acquires driving environment information of the vehicle; as well as a controller configured to detect an abnormal sign state of the driver based on the line of sight information obtained from the line of sight detection device, the driving operation information obtained from the driving operation detection device, and the driving environment information obtained from the driving environment information acquisition device, The controller is capable of executing a sight line model and a driving operation prediction model, wherein when a feature quantity related to the driving operation and the driving environment is input, the sight line model outputs a sight line prediction value of a healthy driver taking into account the influence of these feature quantities, and when a feature quantity related to the driving environment is input, the driving operation prediction model outputs a driving operation prediction value including a steering operation of a healthy driver, The controller calculates a degree of sight abnormality, which indicates a degree to which a measured value of a sight obtained based on the sight information deviates from a predicted value of a sight with respect to a current driving environment output by the sight model. the controller calculating a driving operation abnormality indicating a degree to which an actual value of a driving operation acquired based on the driving operation information deviates from a predicted value of the driving operation with respect to a current driving environment output by the driving operation prediction model, The controller calculates the driving risk of the vehicle based on the driving environment information, The controller combines the sight line abnormality, the driving operation abnormality, and the driving risk to calculate a comprehensive abnormality. The controller determines that the abnormality sign state of the driver is detected when the comprehensive abnormality degree is equal to or greater than a predetermined threshold value.

2. The driver abnormality sign detection device according to claim 1, characterized in that: The controller is configured as follows: Obtaining the maximum value of the degree of abnormality of the sight line and the maximum value of the degree of abnormality of the driving operation within the most recent prescribed time, The maximum value of the degree of abnormality of the sight line, the maximum value of the degree of abnormality of the driving operation, and the travel risk are synthesized to calculate a comprehensive abnormality.

3. The driver abnormality sign detection device according to claim 2, characterized in that: The controller is configured as follows: Get the average value of driving risk within the recent specified time. The maximum value of the degree of abnormality of the sight line, the maximum value of the degree of abnormality of the driving operation, and the average value of the driving risk are synthesized to calculate a comprehensive abnormality.

4. The driver abnormality sign detection device according to any one of claims 1 to 3, characterized in that: The controller is configured to determine that an abnormality sign state of the driver is detected when the comprehensive abnormality degree is equal to or greater than a predetermined threshold value and an increase in the travel risk is detected.

5. The driver abnormality sign detection device according to any one of claims 1 to 3, characterized in that: The controller is configured as follows: acquiring a predicted value of the driver's glance based on the driving environment information, obtaining a measured value of the driver's glance based on the line of sight information, The degree to which the measured value of the glance deviates from the predicted value of the glance is calculated as the degree of sight abnormality.

6. The driver abnormality sign detection device according to any one of claims 1 to 3, characterized in that: The controller is configured as follows: acquiring a predicted value of the operation amount of the steering wheel, the accelerator pedal and / or the brake pedal based on the driving environment information, acquiring a measured value of the operation amount based on the driving operation information, The degree to which the actual measurement value of the operation amount deviates from the predicted value of the operation amount is calculated as the driving operation abnormality degree.

Images