Driver state estimation device

Abstract

Provided is a driver state estimation device capable of accurately estimating that a driver is in a distraction state without depending on a traveling environment. A driver state estimation device (100) is provided with a controller (10) configured so as to estimate the state of a driver on the basis of travel environment information and the line of sight of the driver, and the controller calculates, on the basis of a relative risk, an observable time, and a speed of a vehicle, a predicted value of a gaze frequency at which the driver gazes an object to be noticed during a predetermined observation time. On the basis of the observable time, a predicted value of the watching time at which the driver continuously watches the object to be noticed during the predetermined observation time is calculated, and actual measurement values of the watching frequency and the watching time at which the driver watches the object to be noticed during the predetermined observation time are acquired. The degree of abnormality of the driver is calculated on the basis of the product of the difference between the measured value and the predicted value of the gaze frequency and the difference between the measured value and the predicted value of the gaze time, and when the degree of abnormality is equal to or greater than a predetermined threshold, it is estimated that the driver is in a distraction state.

Description

Technical Field

[0001] The present invention relates to a driver state estimating device for estimating the state of a driver driving a vehicle. Background Art

[0002] One of the main causes of traffic accidents is a driver's lack of concentration on driving, also known as a distracted state. Previously, technologies for detecting distracted states have included estimating a driver's concentration on driving based on the ratios and durations of the driver's gazes at a forward fixation point, a driving fixation point, and a non-driving fixation point (see, for example, Patent Document 1); and estimating that the current driver is in a state of inattention when the time constant of a driver model representing a temporal delay in gaze point movement is greater than the time constant of the driver's previous driver model or the time constant of a standard driver model (see, for example, Patent Document 2).

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent No. 7164275

[0006] Patent Document 2: Japanese Patent No. 6958886

[0007] Technical problem to be solved by the invention

[0008] However, in the above-mentioned prior art, since the differences in driving environments such as the vehicle speed and surrounding congestion are not taken into account, the driver's state is estimated equally in any driving environment. Therefore, when the driving environment changes significantly, such as when entering a highway from a general road, it is possible to mistakenly determine that the driver is distracted. Summary of the Invention

[0009] The present invention has been made to solve such a problem, and an object of the present invention is to provide a driver state estimating device that can accurately estimate that a driver is in a distracted state regardless of the driving environment.

[0010] Technical means for solving technical problems

[0011] In order to solve the above-mentioned technical problems, the present invention is a driver state estimation device for estimating the state of a driver driving a vehicle, comprising: a driving environment information acquisition device, which acquires the driving environment information of the vehicle; a line of sight detection device, which detects the line of sight of the driver; and a controller, which is configured to estimate whether the driver is in a distracted state based on the driving environment information and the driver's line of sight, and the controller is configured to calculate a relative risk based on the driving environment information, the relative risk indicating the risk of collision between the vehicle and surrounding attention objects of the vehicle, and calculate an observable time based on the driving environment information, the observable time indicating the presence of an attention object at a position that the driver can look at. The method comprises the following steps: according to the first embodiment of the present invention, wherein the driver is directed to the object of attention, and the second embodiment comprises the following steps: determining the time during which the driver looks at the object of attention, based on the relative risk, the observable time and the speed of the vehicle, calculating a predicted value of the gaze frequency, which represents the number of times the driver looks at the object of attention during the specified observation time, and calculating a predicted value of the gaze time during which the driver continuously looks at the object of attention during the specified observation time, and obtaining the respective measured values of the gaze frequency and gaze time of the driver looking at the object of attention during the specified observation time based on the driving environment information and the driver's line of sight, and calculating the abnormality of the driver based on the difference between the measured value and the predicted value of the gaze frequency and the product of the difference between the measured value and the predicted value of the gaze time. When the abnormality is above the specified threshold value, it is presumed that the driver is in a distracted state.

[0012] According to the present invention thus constructed, the controller calculates a predicted gaze frequency based on relative risk, observable time, and vehicle speed, and a predicted gaze duration based on the observable time. If the driver's abnormality, calculated based on the product of the difference between the measured and predicted gaze frequency values and the difference between the measured and predicted gaze duration values, exceeds a predetermined threshold, the controller infers that the driver is distracted. This allows the effects of the driving environment on gaze frequency and gaze duration to be reflected, enabling highly accurate prediction of the gaze frequency and gaze duration of a driver in a normal state. The controller can accurately infer the driver's state based on the difference between the gaze frequency and gaze duration and their respective measured values. This makes it possible to accurately infer that the driver is distracted, regardless of the driving environment.

[0013] In the present invention, preferably, the controller is configured to determine the type of attention object based on driving environment information, calculate the abnormality according to the type of attention object, synthesize the abnormality calculated according to the type of attention object to calculate the comprehensive abnormality, and when the comprehensive abnormality is above a specified threshold, it is inferred that the driver is in a distracted state.

[0014] According to the present invention thus constructed, the controller synthesizes the abnormality degrees calculated according to the type of attention object to calculate a comprehensive abnormality degree. When the comprehensive abnormality degree is above a specified threshold value, it is estimated that the driver is in a distracted state. Therefore, even if there are systematic errors in the predicted values of the gaze frequency and gaze time according to the type of attention object, the abnormality degrees calculated according to the type of attention object can be used to estimate the driver's state in a manner that does not overlap these multiple systematic errors, and the driver can be estimated to be in a distracted state more accurately.

[0015] In the present invention, preferably, the types of cautionary objects include a preceding vehicle, a side vehicle, and an unconfirmed object.

[0016] According to the present invention thus constructed, even when the predicted values of the gaze frequency and gaze time for the leading vehicle, the side vehicle, and the unconfirmed object contain different systematic errors, the controller can use the abnormality degrees calculated separately for the leading vehicle, the side vehicle, and the unconfirmed object to estimate the driver's state in a manner that does not overlap these multiple systematic errors, and can more accurately estimate that the driver is in a distracted state.

[0017] Effects of the Invention

[0018] According to the driver state estimating device of the present invention, it is possible to accurately estimate that the driver is in a distracted state regardless of the driving environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is an explanatory diagram of a vehicle equipped with a driver state estimation device according to an embodiment of the present invention.

[0020] Figure 2 This is a block diagram of a driver state estimation device according to an embodiment of the present invention.

[0021] Figure 3 This is a flowchart of the driver state estimation process according to the embodiment of the present invention.

[0022] Figure 4 This is a conceptual diagram illustrating types of attention objects to which the driver's line of sight is directed in the embodiment of the present invention.

[0023] Explanation of symbols

[0024] 1 vehicle

[0025] 10 controllers

[0026] 100 Driver status estimation device

[0027] 21Exterior camera device

[0028] 22 radar

[0029] 23 Navigation System

[0030] 24 positioning system

[0031] 25 vehicle speed sensor

[0032] 26 accelerometers

[0033] 27 yaw rate sensor

[0034] 28 steering angle sensor

[0035] 29 Steering torque sensor

[0036] 30 accelerometer sensors

[0037] 31 brake sensor

[0038] 32In-car camera

[0039] 36 displays

[0040] 37 speakers. DETAILED DESCRIPTION

[0041] Hereinafter, a driver state estimating device according to an embodiment of the present invention will be described with reference to the drawings.

[0042] [System Structure]

[0043] First, refer to Figure 1 and Figure 2 , the structure of the driver state estimation device of this embodiment is described. Figure 1 is an explanatory diagram of a vehicle equipped with a driver state estimation device. Figure 2 This is a block diagram of a driver state estimation device.

[0044] The vehicle 1 of this embodiment includes: a driving force source 2 such as an engine or an electric motor that outputs driving force; a transmission 3 that transmits the driving force output from the driving force source 2 to the drive wheels; a brake 4 that applies braking force to the vehicle 1; and a steering device 5 for steering the vehicle 1.

[0045] The driver state estimation device 100 is configured to estimate the state of the driver of the vehicle 1 and perform control of the vehicle 1 and driving support control as needed. Figure 2 As shown, the driver state estimation device 100 includes a controller 10, a plurality of sensor devices, a plurality of control systems, and a plurality of information output devices.

[0046] Specifically, the multiple sensor devices include an exterior camera 21 and a radar 22 for acquiring information about the driving environment of vehicle 1; a navigation system 23 and a positioning system 24 for detecting the position of vehicle 1. Furthermore, the multiple sensor devices include a vehicle speed sensor 25, an acceleration sensor 26, a yaw rate sensor 27, a steering angle sensor 28, a steering torque sensor 29, an accelerator sensor 30, and a brake sensor 31 for detecting the movement of vehicle 1 or the driver's driving operations. Furthermore, the multiple sensor devices include an interior camera 32 for detecting the driver's line of sight. The multiple control systems include a powertrain control module (PCM) 33 for controlling drive source 2 and transmission 3; a dynamic stability control system (DSC) 34 for controlling drive source 2 and brakes 4; and an electric power steering system (EPS) 35 for controlling steering system 5. The multiple information output devices include a display 36 for outputting image information and a speaker 37 for outputting audio information.

[0047] In addition, other sensor equipment may also include peripheral sonar for measuring the distance and position of surrounding structures relative to the vehicle 1, corner radar for measuring the approach of surrounding structures at the corners of the four parts of the vehicle 1, and various sensors for detecting the driver's status (such as heart rate sensors, electrocardiogram sensors, steering wheel grip force sensors, etc.).

[0048] The controller 10 performs various calculations based on signals received from a plurality of sensor devices, sends control signals to the PCM 33, DSC 34, and EPS 35 to properly operate the drive source 2, transmission 3, brakes 4, and steering system 5, and sends control signals to the display 36 and speaker 37 to output desired information. The controller 10 is configured as a computer having one or more processors 10a (typically CPUs), memory 10b (ROM, RAM, etc.) that stores various programs and data, and input / output devices.

[0049] The off-vehicle camera 21 captures the surroundings of the vehicle 1 and outputs image data. The controller 10 identifies objects (e.g., preceding vehicles, parked vehicles, pedestrians, the road, dividing lines (lane boundaries, white lines, yellow lines), traffic signals, traffic signs, stop lines, intersections, obstacles, etc.) based on the image data received from the off-vehicle camera 21. Furthermore, the controller 10 can determine the curvature of the road on which the vehicle 1 is traveling and the external illumination of the vehicle 1 based on the image data received from the off-vehicle camera 21. Furthermore, the off-vehicle camera 21 is an example of a "driving environment information acquisition device" in the present invention.

[0050] The radar 22 measures the position and speed of an object (especially a preceding vehicle, a parked vehicle, a pedestrian, a fallen object on the road, etc.). As the radar 22, for example, a millimeter wave radar can be used. The radar 22 transmits radio waves in the direction of travel of the vehicle 1, and receives reflected waves generated when the transmitted waves are reflected by the object. Then, the radar 22 measures the distance between the vehicle 1 and the object (for example, the distance between the vehicles), and the relative speed of the object relative to the vehicle 1 based on the transmitted waves and the received waves. In addition, in this embodiment, a laser radar, an ultrasonic sensor, etc. can be used instead of the radar 22 to measure the distance and relative speed to the object. In addition, a plurality of sensor devices can also be used to form a position and speed measuring device. In addition, the radar 22 is equivalent to an example of a "driving environment information acquisition device" in the present invention.

[0051] The navigation system 23 internally stores map information and can provide the map information to the controller 10. Based on the map information and the current vehicle position information, the controller 10 identifies roads, intersections, traffic signals, buildings, and the like surrounding the vehicle 1 (particularly in the direction of travel). Furthermore, the controller 10 can determine the curvature and slope of the road on which the vehicle 1 is traveling based on the map information and the current vehicle position information. The map information can also be stored within the controller 10. The positioning system 24 is a GPS system and / or a gyroscope system that detects the position of the vehicle 1 (current vehicle position information).

[0052] The vehicle speed sensor 25 detects the speed of the vehicle 1 based on, for example, the rotational speed of the wheels and the drive shaft. The acceleration sensor 26 detects the acceleration of the vehicle 1. The acceleration includes the acceleration in the front-rear direction and the lateral acceleration (i.e., lateral acceleration) of the vehicle 1. In addition, the controller 10 can determine the slope of the road on which the vehicle 1 is traveling based on the speed and acceleration of the vehicle 1. In addition, in this specification, the acceleration includes not only the rate of change of speed in the direction in which the speed increases, but also the rate of change of speed in the direction in which the speed decreases (i.e., deceleration). In addition, the vehicle speed sensor 25 and the acceleration sensor 26 are also equivalent to an example of the "driving environment information acquisition device" in the present invention.

[0053] The yaw rate sensor 27 detects the yaw rate of the vehicle 1. The steering angle sensor 28 detects the rotation angle (steering angle) of the steering wheel of the steering system 5. The steering torque sensor 29 detects the torque (steering torque) applied to the steering shaft via the steering wheel. The accelerator sensor 30 detects the amount of depression of the accelerator pedal. The brake sensor 31 detects the amount of depression of the brake pedal.

[0054] The in-vehicle camera 32 captures the driver's image and outputs image data. The controller 10 detects the driver's line of sight based on the image data received from the in-vehicle camera 32. The in-vehicle camera 32 is an example of a "line of sight detection device" in the present invention.

[0055] The PCM 33 controls the driving force source 2 of the vehicle 1 to adjust the driving force of the vehicle 1. For example, the PCM 33 controls the engine's spark plugs, fuel injection valves, throttle valve, variable valve mechanism, transmission 3, and the inverter that supplies power to the electric motor. When the vehicle 1 needs to be accelerated or decelerated, the controller 10 sends a control signal to the PCM 33 to adjust the driving force.

[0056] The DSC 34 controls the driving force source 2 and brakes 4 of the vehicle 1, performing deceleration control and posture control of the vehicle 1. For example, the DSC 34 controls the hydraulic pump and valve unit of the brakes 4, controlling the driving force source 2 via the PCM 33. When deceleration control or posture control of the vehicle 1 is required, the controller 10 sends a control signal to the DSC 34 to adjust the driving force or generate braking force.

[0057] The EPS 35 controls the steering system 5 of the vehicle 1. For example, the EPS 35 controls a motor that applies torque to the steering shaft of the steering system 5. When the traveling direction of the vehicle 1 needs to be changed, the controller 10 sends a control signal to the EPS 35 to change the steering direction.

[0058] The display 36 is provided in front of the driver in the vehicle cabin and displays image information to the driver. For example, a liquid crystal display or a head-up display is used as the display 36. The speaker 37 is provided in the vehicle cabin and outputs various audio information.

[0059] [Driver status estimation process]

[0060] Next, refer to Figure 3 and Figure 4 , the driver state estimation process of the driver state estimation device 100 according to this embodiment will be described. Figure 3 This is a flowchart of a driver state estimation process for estimating whether the driver is in a distracted state or a normal state. Figure 4 This is a conceptual diagram explaining the types of attention objects to which the driver's line of sight is directed.

[0061] The driver state estimation process is started when the vehicle 1 is powered on, and is repeatedly executed by the controller 10 at a predetermined cycle (for example, every 0.05 to 0.2 seconds).

[0062] When the driver state estimation process starts, the controller 10 first acquires driving environment information based on signals received from sensor devices including the exterior camera 21 , the radar 22 , the vehicle speed sensor 25 , and the acceleration sensor 26 (step S1 ).

[0063] Next, the controller 10 detects the driver's line of sight based on the signal received from the in-vehicle camera 32 (step S2 ).

[0064] Next, the controller 10 acquires objects (caution objects) around the vehicle 1 to which the driver's line of sight should be directed based on the driving environment information acquired in step S1 (step S3). In the driver state estimation process of this embodiment, other vehicles are used as caution objects.

[0065] Next, the controller 10 obtains the measured value f of the gaze frequency toward each attention object i within the most recent observation time (e.g., 10 seconds) based on the driver's line of sight detected in step S2 and the attention object acquired in step S3. (i) and the measured value of fixation time g (i) (Step S4). Specifically, based on the direction of the driver's line of sight determined from the image acquired from the in-vehicle camera 32 and the position of the attention object acquired by the out-vehicle camera 21 and the radar 22, if the direction of the line of sight and the direction of the attention object overlap for a predetermined time (e.g., 0.1 seconds) or longer based on the position of the driver's head, the controller 10 determines that the driver is looking at the attention object. In addition, the measured value f of the gaze frequency is (i) The number of times the driver changes from not looking at the attention object i to looking at the attention object i during the most recent observation time (e.g., 10 seconds). (i) The controller 10 obtains the measured value f of the gaze frequency. (i) and the measured value of fixation time g (i) Stored in the memory 10b.

[0066] Next, the controller 10 calculates the observable time T of each cautionary object i based on the driving environment information acquired in step S1. obs (i) (Step S5) Observable time T obs (i) The time during which the attention object i exists at a position that the driver can look at during the most recent observation time (e.g., 10 seconds). For example, the controller 10 calculates the time during which the attention object i exists within a predetermined angular range (e.g., within 80 degrees to the left and right, and within 50 degrees to the top and bottom) from the direction of travel of the vehicle 1 based on the driver's head position as the observable time T. obs (i) .

[0067] Next, the controller 10 calculates the relative risk R of each cautionary object i based on the driving environment information acquired in step S1. (i) (Step S6). Relative risk R (i)is a numerical value between 0 and 1 that indicates the risk of collision between the vehicle 1 and the caution object i. For example, the controller 10 calculates the collision margin time (TTC) of the caution object i based on the position and relative speed of the caution object i acquired by the vehicle exterior camera 21 and the radar 22. Then, using TTC as a variable, the controller 10 calculates the relative risk R by setting the value such that the smaller the TTC, the higher the relative risk R. (i) The larger the mathematical formula, the relative risk R (i) .

[0068] Next, the controller 10 calculates the speed v of the vehicle 1 obtained in step S1 and the observable time T calculated in step S5. obs (i) and the relative risk R calculated in step S6 (i) , calculate the predicted value f of the gaze frequency of each attention object i p (i) (Step S7) The predicted value of the fixation frequency f p (i) The controller 10 calculates the calculated gaze frequency prediction value f by the following equation which models the gaze frequency of the driver in a normal state. p (i) Stored in the memory 10b.

[0069]

[0070] Here, v is the speed of vehicle 1, and h1 and h2 are nonlinear functions. Based on data on vehicle speed, attention objects, and line of sight obtained through driving experiments using a driving simulator, the driver's gaze frequency under various driving conditions is measured. Regression analysis using this measured data determines the coefficients and constant terms for h1 and h2. The predetermined coefficients and constant terms for h1 and h2 are stored in memory 10b.

[0071] Generally, the higher the vehicle speed v, the lower the driver's dynamic vision. Therefore, in high-speed areas, there is a tendency for the driver to delay the discovery of objects of attention and to reduce the frequency of attention. In addition, since the object rapidly approaching the vehicle is the relative risk R (i) The time that a high-rise object is in the driver's field of vision is short, so the driver's discovery is delayed and the frequency of attention is likely to decrease. (i) respectively affect the frequency of gaze. Moreover, the observable time T obs (i) The longer the time, the more frequent the attention to the object. Therefore, by combining the vehicle speed v, relative risk R (i) and observable time T obs (i)By modeling the relationship with the gaze frequency in a hierarchical manner, it is possible to predict the gaze frequency of the attention object i with high accuracy.

[0072] Next, the controller 10 calculates the observable time T in step S5. obs (i) , calculate the predicted value g of the fixation time of each attention object i p (i) (Step S8) Gaze time prediction value g p (i) The controller 10 calculates the calculated gaze time prediction value g by the following equation which models the gaze time of the driver in a normal state. p (i) Stored in the memory 10b.

[0073]

[0074] Here, k is a nonlinear function. The coefficient and constant term of k can be determined by measuring the driver's gaze duration under various driving conditions in a normal state based on data on attentional objects and line of sight obtained through driving experiments using a driving simulator. Regression analysis using this measured data is then performed to determine the coefficient and constant term of k. The predetermined coefficient and constant term of k are stored in memory 10b.

[0075] The present inventors experimentally confirmed the following: the observable time T obs (i) The longer the time, the longer the attention time to the object of attention. Therefore, by using the observable time T obs (i) By modeling the fixation time, it is possible to predict the fixation time on the attention object i with high accuracy.

[0076] Next, the controller 10 calculates the actual value f of the gaze frequency of each attention object i based on the actual value f of the attention frequency of each attention object i. (i) and the measured value of fixation time g (i) , predicted value of fixation frequency f p (i) and the predicted fixation time g p (i) Calculate the abnormality degree GF of the driver's state according to the type of attention object f GF s and GF u (Step S9). GF f Indicates that the object of attention is the preceding vehicle, GF s Indicates that the object of attention is a vehicle on the side. u Indicates that the object of attention is an unconfirmed object.

[0077] Here, refer to Figure 4 The types of objects to be paid attention to in this embodiment are described below. Figure 4 In the example, A represents the vehicle itself, B represents the preceding vehicle traveling in the same lane as vehicle A, and C and D represent the adjacent vehicles traveling in the adjacent lane in front of the vehicle. Furthermore, the driver is looking at vehicle B and adjacent vehicle C, but is not looking at adjacent vehicle D (e.g., the driver's gaze is directed toward the adjacent vehicle for less than 0.1 seconds). In this embodiment, the types of these attention objects are classified into three categories: (1) preceding vehicle B, (2) adjacent vehicle C, and (3) unidentified object D.

[0078] The position of the leading vehicle is determined using image data acquired from the exterior camera 21, while the position of the accompanying vehicle is determined using the radar 22. Therefore, different systematic errors are involved in position determination and gaze assessment. Consequently, if the gaze frequency and duration are not evaluated separately for the leading vehicle, accompanying vehicles, and unidentified objects, multiple systematic errors will be included, reducing the accuracy of driver status estimation.

[0079] Therefore, by classifying attention objects into three categories: (1) preceding vehicles, (2) vehicles on the sides, and (3) unidentified objects, and evaluating the difference between the measured and predicted values of the gaze frequency and gaze time, the driver's state can be appropriately estimated.

[0080] Specifically, for each attention object i, the controller 10 obtains the measured value f of the gaze frequency stored in the memory 10b during the most recent predetermined time (for example, 10 seconds). (i) and the measured value of fixation time g (i) , predicted value of fixation frequency f p (i) and the predicted fixation time g p (i) , and calculate the abnormality degree GF for each attention object i by the following formula val (i) The subscript "val" of the abnormality level indicates "f" when the type of the attention object is a preceding vehicle, "s" when it is a side vehicle, and "u" when the object is unconfirmed. In the following equation, "MA" represents the moving average over the most recent specified period of time.

[0081]

[0082] When there are multiple objects of the same type, the abnormality degree GF of each object i is calculated. val (i) The average value of the driver's state abnormality GF for each type of attention object in the latest specified time is calculated. f GFs and GF u .

[0083] Next, the controller 10 calculates the abnormality degree GF in step S9. f GF s and GF u , calculate the comprehensive abnormality degree d (step S10). For example, the controller 10 accumulates the abnormality degree GF f GF s and GF u For each three-dimensional data set as a variable, the Mahalanobis distance between the latest data point of the three-dimensional data and the centroid (average) of the accumulated data set is calculated as the comprehensive abnormality d.

[0084] Next, the controller 10 determines whether the comprehensive abnormality degree d calculated in step S10 is a threshold value d. th The above (step S11).

[0085] As a result, when the comprehensive abnormality d is not the threshold d th In the above case (step S11 : NO), the controller 10 estimates that the driver's state is normal (step S12 ), and ends the driver state estimation process.

[0086] On the other hand, when the comprehensive abnormality d is the threshold d th In the above case (step S11 : YES), the controller 10 estimates that the driver is distracted (step S13 ).

[0087] Next, the controller 10 sends control signals to the display 36 and speaker 37, causing the display 36 and speaker 37 to output an alert notifying the driver that the driver is distracted (step S14). At this time, the display 36 and speaker 37 may also output image information and audio information (sight guidance information) to guide the driver's gaze toward an object that the driver has not visually recognized. After step S14, the controller 10 terminates the driver state estimation process.

[0088] [Modification]

[0089] In the above embodiment, the abnormality degree d and the threshold value d are combined to form a th The driver's state can be estimated by comparing the abnormality d, but it is also possible to use the abnormality GF according to the type of the attention object instead of the comprehensive abnormality d. f GF s and GF u The driver's state is estimated by comparing the respective values with respective predetermined threshold values.

[0090] In the above embodiment, the Mahalanobis distance is used to calculate the comprehensive abnormality d, but the comprehensive abnormality d can also be calculated by other calculation methods. For example, the abnormality GF of each type of attention object can be calculated. f GF s and GF u The normalized and synthesized value is taken as the comprehensive abnormality degree d.

[0091] Furthermore, in the above-described embodiment, description has been given using other vehicles as cautionary objects. However, in addition to other vehicles, pedestrians, obstacles, and the like may also be used.

[0092] Alternatively, the gaze frequency prediction value f can be modified by performing individual learning for each driver. p (i) and the predicted fixation time g p (i) .

[0093] [Function, effect]

[0094] Next, the effects of the driver state estimation device 100 according to the above-described embodiment will be described.

[0095] The controller 10 is based on the relative risk R (i) , observable time T obs (i) And the speed v of vehicle 1, calculate the predicted value f of the gaze frequency p (i) , based on the observable time T obs (i) , calculate the predicted value of fixation time g p (i) , based on the measured value of the gaze frequency f (i) and the predicted value f p (i) The difference between the actual value of the fixation time g (i) and the predicted value g p (i) The driver's abnormality d calculated by the product of the difference is the predetermined threshold value d th In the above situation, the driver is estimated to be distracted. This method can reflect the impact of the driving environment on gaze frequency and duration, accurately predict the gaze frequency and duration of a normal driver, and accurately estimate the driver's state based on the difference between the gaze frequency and duration and the actual measured values. This makes it possible to accurately estimate a driver's distracted state, regardless of the driving environment.

[0096] In addition, the controller 10 calculates the abnormality degree GF according to the type of the attention object. val (i)The comprehensive abnormality degree d is calculated by synthesis, and when the comprehensive abnormality degree is the predetermined threshold value d th In the above case, it is estimated that the driver is distracted. Therefore, even if the predicted values of the gaze frequency and gaze time contain different systematic errors depending on the type of the attention object, the abnormality degree GF calculated according to the type of attention object can be used so that these multiple systematic errors do not overlap. val (i) To estimate the driver's state, it is possible to more accurately estimate that the driver is distracted.

[0097] Furthermore, even when the predicted values of the gaze frequency and gaze time for the preceding vehicle, the side vehicle, and the unidentified object contain different systematic errors, the controller 10 can use the abnormality degree GF calculated separately for the preceding vehicle, the side vehicle, and the unidentified object so that these multiple systematic errors do not overlap. f GF s and GF u To estimate the driver's state, it is possible to more accurately estimate that the driver is distracted.

Claims

1. A driver state estimation device for estimating the state of a driver driving a vehicle, characterized in that: have: A driving environment information acquisition device, the driving environment information acquisition device acquires driving environment information of the vehicle; a sight line detection device, the sight line detection device detecting the sight line of the driver; as well as a controller configured to estimate the driver's state based on the driving environment information and the driver's line of sight, The controller is configured as follows: calculating a relative risk indicating a risk of collision between the vehicle and an object of concern around the vehicle based on the driving environment information; Based on the driving environment information, an observable time is calculated, the observable time indicating the time during which the attention object exists at a position where the driver can look. calculating a predicted value of a gaze frequency indicating the number of times the driver gazes at the attention object during a predetermined observation time based on the relative risk, the observable time, and the speed of the vehicle; calculating a predicted value of the time during which the driver continuously gazes at the attention object during a predetermined observation time based on the observable time, obtaining, based on the driving environment information and the driver's line of sight, actual values of the driver's gaze frequency and gaze duration at the attention object during a predetermined observation time; calculating the abnormality degree of the driver based on the product of the difference between the actual measurement value and the predicted value of the gaze frequency and the difference between the actual measurement value and the predicted value of the gaze time, When the abnormality level is equal to or greater than a predetermined threshold, it is estimated that the driver is in a distracted state.

2. The driver state estimation device according to claim 1, characterized in that: The controller is configured as follows: determining the type of the caution object based on the driving environment information, Calculating the abnormality degree according to the type of the attention object, The abnormality levels calculated for each type of the attention object are combined to calculate a comprehensive abnormality level. When the comprehensive abnormality level is equal to or greater than a predetermined threshold value, it is estimated that the driver is in a distracted state.

3. The driver state estimation device according to claim 2, characterized in that: The types of caution objects include preceding vehicles, vehicles on the sides, and unconfirmed objects.

Images