DRIVER STATE ESTIMATION DEVICE AND DRIVER STATE DETERMINATION METHOD

Abstract

A driver state estimation device (100) includes a controller (10) configured to estimate whether the driver is in the inattentive state. Based on the driving environment information and / or the driver's line of sight, the controller detects feature values x i , which include a frequency and an amplitude of a driver's saccade and a top-down attention score indicating a degree of deviation from a convenient line-of-sight distribution to an attention object around the vehicle, uses the acquired feature values x i , a predetermined weight coefficient a i for each of the characteristic values x iand a predetermined constant a0 to calculate an inattention probability p representing a probability that the driver is in the inattentive state by a sigmoid function and estimates that the driver is in the inattentive state when a state in which the calculated inattention probability p is equal to or higher than a predetermined value continues for a predetermined time or longer.

Description

[Technical field]

[0001] The present invention relates to a driver state estimation device that estimates a state of a driver driving a vehicle. The present invention also relates to a method for determining a state of the driver. [Technical background]

[0002] One of the main causes of traffic accidents is a state in which a driver's concentration on driving is poor, that is, a so-called inattentive state. Conventionally, the following technique and the like have been proposed as a technique for detecting the inattentive state. In the technique (see, for example, Patent Literature 1), the main focus is on a finding that a movement speed and the duration per amplitude of a saccade, that is, a rapid eye movement that occurs when the driver's line of sight moves, differ between a case where the driver deliberately looks at a position other than that on a road and a case where the driver drives normally and detects a forward view. [List of citations][Patent literature]

[0003] [Patent Literature 1] JP2017-224066A [Summary][Technical Problem]

[0004] However, in the conventional technique as described above, even if the driver is not in the inattentive state, the driver can be estimated to be in the inattentive state if a frequency or change amount of the driver's line of sight movement changes due to disease, aging, or the like. That is, in the conventional technique, it is difficult to accurately estimate that the driver is in the inattentive state by distinguishing the inattentive state from other abnormal conditions such as disease.

[0005] The invention was made to solve such a problem, and an object thereof is to provide a driver state estimation device capable of estimating that a driver is in an inattentive state by distinguishing the inattentive state from other abnormal states such as a disease. [Solution to the problem]

[0006] The above-described problem is solved by the invention as defined in claim 1. Specifically, a driver state estimation device that estimates a state of a driver driving a vehicle includes: driving environment information acquisition means that acquires driving environment information of the vehicle; line-of-sight detection means that detects a line of sight of the driver; and a controller configured to estimate, based on the driving environment information and the driver's line of sight, whether the driver is in an inattentive state. The controller is configured to generate a feature value x i(i = 1, ..., n) detect each of a plurality of indicators of search behavior changed according to the driver's state based on the driving environment information and / or the driver's line of sight. The controller is further configured to calculate an inattention probability p, which represents a probability that the driver is in the inattentive state, by the following equation using the detected feature value x i , a predetermined weight coefficient a i for each of the characteristic values x i and a predetermined constant a0. p=11+e−(a0+∑i=1naixi)

[0007] The controller is further configured to estimate that the driver is in the inattentive state when a state in which the calculated inattention probability p is equal to or higher than a predetermined value persists for a predetermined time or longer. The feature values x i include or are: a frequency and an amplitude of a driver's saccade, which are detected based on the driver's line of sight; and a top-down attention score indicating a degree of deviation from a convenient line-of-sight distribution to an attention object around the vehicle, which is detected based on the driving environment information and the driver's line of sight.

[0008] According to the invention, for each of the plurality of indicators of the search behavior changed according to the driver's condition, the controller detects the feature values x i, which include: the frequency and amplitude of the driver's saccade; and the top-down attention score, which indicates the degree of deviation from the appropriate line-of-sight distribution to the attention object around the vehicle, and uses the acquired feature values x i , the pre-determined weight coefficient a i for each of the characteristic values x i and the predetermined constant a ito calculate the inattention probability p through the sigmoidal function. Accordingly, instead of merely focusing on any one indicator of the driver's search behavior, unique changes in the saccade frequency and amplitude and the top-down attention score in the driver's inattentive state are comprehensively detected to quantitatively evaluate the probability that the driver is in the inattentive state. In this way, it is possible to estimate the inattentive state by distinguishing the changes in the feature values therein from those caused by the driver's illness, aging, or the like.

[0009] The controller can be configured to convert each of the captured feature values x i based on driving environment information.

[0010] According to the embodiment as described above, the controller corrects each of the detected feature values X i based on the driving environment information. This makes it possible to calculate the feature values x i in such a way as to cancel out an influence of the vehicle's driving environment in order to calculate the inattention probability p more accurately, and thus it is possible to prevent an erroneous estimation of the driver's state caused by the driving environment.

[0011] The controller may be configured to detect a gradient of a road on which the vehicle is traveling based on the driving environment information and to calculate the feature value x as the gradient increases. i in a direction where it is less likely that the driver will be estimated in the inattentive state. The controller can set the feature value x icorrect so that it is larger with increasing gradient.

[0012] According to the embodiment as described above, the controller corrects the feature value x as the gradient of the road on which the vehicle is traveling increases. i in the direction in which the driver is less likely to be estimated in the inattentive state. Accordingly, if the driver is likely to be estimated in the inattentive state due to the steep gradient of the road and a tendency for the driver's line of sight to be concentrated in a narrow area, it is possible to estimate the feature value x i in such a way that the influence of the gradient of the road is eliminated and thus the inattention probability p is calculated even more accurately.

[0013] The controller may be configured to detect a curvature of a road on which the vehicle is traveling based on the driving environment information and, as the curvature increases, to increase the feature value x i in a direction where it is less likely that the driver will be estimated in the inattentive state. The controller can set the feature value x i correct it so that it is larger with increasing curvature.

[0014] According to the embodiment as described above, the controller corrects the feature value x as the curvature of the road on which the vehicle is traveling increases. iin a direction where the driver is less likely to be estimated in the inattentive state. Accordingly, if the driver is likely to be estimated in the inattentive state due to the large curvature of the road and the tendency for the driver's line of sight to be concentrated in the narrow area, it is possible to estimate the feature value x i in such a way that the influence of the curvature of the road is eliminated and thus the inattention probability p is calculated even more accurately.

[0015] The controller may be configured to detect an illuminance outside the vehicle based on the driving environment information and, as the illuminance decreases, to adjust the feature value x i in a direction where it is less likely that the driver will be estimated in the inattentive state. The controller can set the feature value x icorrect so that it is larger as the illuminance decreases.

[0016] According to the embodiment configured as described above, as the illuminance outside the vehicle decreases, the controller corrects the feature value x i in the direction in which the driver is less likely to be estimated in the inattentive state. Accordingly, if the driver is likely to be estimated in the inattentive state due to the low illuminance outside the vehicle and the tendency for the driver's line of sight to be concentrated in the narrow area, it is possible to estimate the feature value x i in such a way that the influence of illuminance is eliminated and thus the inattention probability p is calculated even more accurately.

[0017] The controller may be configured to detect a speed of the vehicle based on the driving environment information and, as the speed increases, to calculate the feature value x i in a direction where it is less likely that the driver will be estimated in the inattentive state. The controller can set the feature value x i correct so that it is larger with increasing speed.

[0018] According to the embodiment as described above, the controller corrects the feature value x as the speed of the vehicle increases iin the direction in which the driver is less likely to be estimated in the inattentive state. Accordingly, if the driver is likely to be estimated in the inattentive state due to the high vehicle speed and the tendency for the driver's line of sight to be concentrated in the narrow area, it is possible to estimate the feature value x i in such a way that the influence of the vehicle speed is eliminated and thus the inattention probability p is calculated even more accurately. [Effects of the invention]

[0019] According to the driver state estimation device of the invention, it is possible to estimate that the driver is in the inattentive state by distinguishing the inattentive state from other abnormal states such as a disease. [Brief description of the drawings] Fig. 1 is an explanatory view of a vehicle on which a driver condition estimation device according to an embodiment of the invention is mounted. Fig. 2 is a block diagram of the driver state estimation device according to the embodiment of the invention. Fig. 3 is a flowchart of individual learning processing according to the embodiment of the invention. Fig. 4 is a flowchart of driver state estimation processing according to the embodiment of the invention. Fig. 5 includes diagrams each exemplifying a correction coefficient map according to the embodiment of the invention. Fig. 6 includes time charts exemplifying temporal changes of feature values and inattention probabilities of search behavior indicators of a driver according to the embodiment of the invention. [Description of embodiments]

[0020] Hereinafter, a driver condition estimation apparatus according to an embodiment of the invention will be described with reference to the accompanying drawings. [System configuration]

[0021] First, a configuration of the driver state estimation device according to the present embodiment will be described with reference to Fig. 1 and Fig. 2 described. Fig. 1 is an explanatory view of a vehicle on which the driver condition estimation device is mounted, and Fig. 2 is a block diagram of the driver state estimation device.

[0022] A vehicle 1 according to the present embodiment may include: a driving power source 2 such as an internal combustion engine or an electric motor that outputs a driving force; a transmission 3 that transmits the driving force output from the driving power source 2 to drive wheels; a brake 4 that applies a braking force to the vehicle 1; and a steering device 5 for steering the vehicle 1.

[0023] A driver state estimation device 100 is configured to estimate a state of a driver of the vehicle 1 and may be configured to execute control of the vehicle 1 and driver assistance control as needed. As shown in Fig. As illustrated in Figure 2, the driver state estimation device 100 includes a controller 10, one or a plurality of sensors, one or a plurality of control systems, and one or a plurality of information output devices.

[0024] In particular, the plurality of sensors may include an exterior camera 21 and / or a radar 22 for acquiring driving environment information of the vehicle 1, and a navigation system 23 and a positioning system 24 for detecting a position of the vehicle 1. Alternatively or additionally, the plurality of sensors may also include at least one of a vehicle speed sensor 25, an acceleration sensor 26, a yaw rate sensor 27, a steering angle sensor 28, a steering torque sensor 29, an accelerator pedal sensor 30, and a brake sensor 31 for detecting a behavior of the vehicle 1 and a driving action by the driver. Alternatively or additionally, the plurality of sensors may include a vehicle interior camera 32 for detecting a driver's line of sight.

[0025] The plurality of control systems may include at least one of a powertrain control module (PCM) 33 that controls the power source 2 and the transmission 3, a dynamic stability control (DSC) system 34 that controls the power source 2 and the brake 4, and a power steering system (EPS) 35 that controls the steering device 5. The plurality of information output devices may include a display 36 that outputs image information and / or a speaker 37 that outputs audio information.

[0026] For example, other sensors may include: a peripheral sonar that measures a distance to and position of a structure around the vehicle 1; corner radars that each measure an approach of the peripheral structure at a respective one of four corners of the vehicle 1; and various sensors that each detect the state of the driver (e.g., a heart rate sensor, an electrocardiogram sensor, a steering wheel grip force sensor, and the like).

[0027] For example, the controller 10 performs various calculations based on signals received from the plurality of sensors, transmits control signals to the PCM 33, the DSC 34, the EPS 35 for appropriately operating the driving power source 2, the transmission 3, the brake 4, and the steering device 5, and transmits control signals for outputting desired information to the display 36 and the speaker 37. The controller 10 may be configured by a computer including one or more processors 10a (usually CPUs), memories 10b (such as ROM and RAM) for storing various programs and data, an input / output device, and the like.

[0028] The external camera 21 captures an image of the surroundings of the vehicle 1 and outputs image data. The controller 10 detects an object (e.g., a preceding vehicle, a parked vehicle, a pedestrian, a roadway, a dividing line (a lane line, a white line, and a yellow line), a traffic signal, a traffic sign, a stop line, an intersection, an obstacle, and the like) based on the image data received from the external camera 21. Specifically, the controller 10 can identify a curvature of a road on which the vehicle 1 is traveling and an illuminance outside the vehicle 1 based on the image data received from the external camera 21. The external camera 21 corresponds to an example of the "driving environment information acquisition device" in the invention.

[0029] The radar 22 measures a position and a speed of the object (specifically, the preceding vehicle, the parked vehicle, the pedestrian, a dropped object on the roadway, and the like). A millimeter-wave radar, for example, can be used as the radar 22. The radar 22 transmits a radio wave in a traveling direction of the vehicle 1 and receives a reflected wave generated when the transmitted wave is reflected by the object. Then, the radar 22 measures a distance (for example, a distance between vehicles) between the vehicle 1 and the object and a relative speed of the object to the vehicle 1 based on the transmitted wave and the received wave.

[0030] In the present embodiment, instead of radar 22, a laser radar, an ultrasonic sensor, or the like may be used to measure the distance to the object and its relative speed. Alternatively, a plurality of sensors may be used to form a position and speed measuring device. Radar 22 corresponds to one example of the "driving environment information acquisition device" in the invention.

[0031] The navigation system 23 stores map information therein and can provide the map information to the controller 10. The controller 10 can identify the road, intersection, traffic signal, building, and the like that exist around the vehicle 1 (particularly in the traveling direction of the vehicle 1) based on the map information and current vehicle position information. The controller 10 can also identify the curvature and gradient of the road on which the vehicle 1 is traveling based on the map information and the current vehicle position information. The map information can be stored in the controller 10.

[0032] The positioning system 24 is a GPS system and / or a gyroscopic system and detects the position of the vehicle 1 (the current vehicle position information). The navigation system 23 and the positioning system 24 also correspond to examples of the "driving environment information acquisition device" in the invention.

[0033] The vehicle speed sensor 25 detects a speed of the vehicle 1 based, for example, on a rotational speed of the wheel or a drive shaft. The acceleration sensor 26 detects an acceleration of the vehicle 1. This acceleration includes an acceleration in a longitudinal direction of the vehicle 1 and an acceleration in a lateral direction (i.e., a lateral acceleration) thereof. In particular, the controller 10 can identify the gradient of the road on which the vehicle 1 is traveling based on the speed and acceleration of the vehicle 1.

[0034] Acceleration may include not only a rate of change of speed in a speed-increasing direction, but also a rate of change of speed in a speed-decreasing direction (i.e., deceleration). The vehicle speed sensor 25 and the acceleration sensor 26 also correspond to examples of the "driving environment information detecting device" in the invention.

[0035] The yaw rate sensor 27 detects a yaw rate of the vehicle 1. The steering angle sensor 28 detects a rotation angle (a steering angle) of a steering wheel of the steering device 5. The steering torque sensor 29 detects a torque (steering torque) applied to a steering shaft via the steering wheel. The accelerator pedal sensor 30 detects a depression amount of an accelerator pedal. The brake sensor 31 detects a depression amount of a brake pedal. Here, the yaw rate sensor 27, the steering angle sensor 28, the steering torque sensor 29, the accelerator pedal sensor 30, and the brake sensor 31 also correspond to examples of the "driving environment information acquisition device" in the invention.

[0036] The in-vehicle camera 32 captures an image of the driver and outputs image data. The controller 10 detects the driver's line of sight based on the image data received from the in-vehicle camera 32. The in-vehicle camera 32 corresponds to an example of the "line of sight detection device" in the invention.

[0037] The PCM 33 controls the driving power source 2 of the vehicle 1 to adjust the driving power of the vehicle 1. For example, the PCM 33 controls a spark plug, a fuel injection valve, a throttle valve and a variable valve mechanism of the internal combustion engine, the transmission 3, an inverter that supplies electrical power to the electric motor, and the like. When the vehicle 1 needs to be accelerated or decelerated, the controller 10 transmits a control signal to the PCM 33 to adjust the driving power.

[0038] The DSC 34 controls the driving power source 2 and the brake 4 of the vehicle 1, and performs deceleration control and attitude control of the vehicle 1. For example, the DSC 34 controls a hydraulic pump, a valve unit, and the like of the brake 4, and controls the driving power source 2 via the PCM 33. When the deceleration control or attitude control of the vehicle 1 needs to be executed, the controller 10 transmits a control signal to the DSC 34 for adjusting the driving force or generating the braking force.

[0039] The EPS 35 controls the steering device 5 of the vehicle 1. For example, the EPS 35 controls an electric motor that applies torque to the steering shaft of the steering device 5, and the like. When the direction of travel of the vehicle 1 needs to be changed, the controller 10 transmits a control signal to the EPS 35 to change a steering direction.

[0040] The display 36 is provided in front of the driver in a cabin and displays image information for the driver. A liquid crystal display or a head-up display, for example, is used as the display 36. The speaker 37 is installed in the cabin and outputs various types of audio information. [Driver condition estimation]

[0041] Next, a driver state estimation by the driver state estimation device 100 of the present embodiment will be described with reference to Fig. 3 to Fig. 5 described. Fig. Figure 3 is a flowchart of individual learning processing in which individual learning is performed to standardize feature values of search behavior indicators. Fig. 4 is a flowchart of driver state estimation processing for estimating whether the driver is in an inattentive state or normal. Fig. 5 includes diagrams each illustrating a correction coefficient map for correcting the feature value of the search behavior indicator.

[0042] First, an overview of the driver state estimation in the present embodiment will be described. The present inventors conducted driving tests for 100 or more subjects using a driving simulator to investigate how a driver's behavior for visually checking the surroundings of the vehicle (particularly in the vehicle's traveling direction) (hereinafter referred to as the "search behavior") changes between a case where the driver was normal and a case where the driver was in the inattentive state.Specifically, for each of a case where the inattentive state was simulated by having the driver perform mental calculation so that he was unable to concentrate on driving, and a case where the normal state was simulated by having the driver drive normally without performing mental calculation, the subjects were made to drive in various types of driving environments (an urban area, a highway, a mountain road, daytime, nighttime, and the like), and thereby the movement of the driver's line of sight while driving was measured.

[0043] As a result, it was found that the feature values (e.g., including a frequency and an amplitude of a saccade, i.e., a rapid eye movement that occurs when the driver's line of sight moves) of a plurality of indicators related to the driver's search behavior were each changed according to a tendency characteristic of each of the indicators between the case where the driver was normal and the case where the driver was in the inattentive state.Accordingly, the inventors of the invention assumed that it was possible to calculate a probability that the driver was in the inattentive state from each of the feature values during the actual driving by setting whether the estimated behavior of the driver corresponded to the case where the driver's inattentive state was simulated and the case where the driver's normal state was simulated as response variables of two values from data on the movement of the line of sight acquired by the above-mentioned driving tests and data on the driving environment simulated by the driving simulator, by setting a value acquired by standardizing each of the feature values of the majority of the indicators of the search behavior as an explanatory variable, and by conducting a logistic regression analysis and detecting a regression coefficient in advance.

[0044] Specifically, the driver state estimation device 100 acquires each of the feature values of the plurality of indicators of the driver's search behavior based on the driving environment information of the vehicle 1 and / or the driver's line of sight. For example, the driver state estimation device 100 acquires, as the feature values of the plurality of indicators of the search behavior: the amplitude and frequency of the driver's line of sight saccade; a top-down attention score indicating a degree of deviation from a proper line of sight distribution to an attention object around the vehicle 1; and a bottom-up attention score indicating a degree of directing the line of sight to a high-saliency position.

[0045] Each of the acquired feature values can be corrected according to a driving scene, such as the gradient or curvature of the road. Furthermore, each of the corrected feature values can be standardized using an average value (or a median) and a variance of each of the feature values acquired in advance by performing individual learning processing for each driver, at the time when the driver is normal. The probability that the driver is in the inattentive state can be calculated by substituting each of the feature values after standardization into a sigmoid function including a regression coefficient acquired by performing logistic regression analysis based on the driving test described above.

[0046] When a state in which the calculated probability is equal to or higher than a predetermined threshold persists for a predetermined time or longer, the driver state estimation device 100 estimates that the driver is in an inattentive state. As described, rather than focusing solely on one of the indicators of the driver's search behavior, unique changes in the feature values of the plurality of indicators in the driver's inattentive state are comprehensively detected to quantitatively evaluate the probability that the driver is in the inattentive state. In this way, it is possible to estimate the inattentive state by distinguishing the changes in the feature values in the inattentive state from those caused by the driver's illness, aging, or the like. [Individual learning processing]

[0047] Next, the individual learning processing is discussed with reference to Fig. 3. In the individual learning processing, the driver state estimation device 100 can calculate, for each driver, the average value and variance of each of the feature values used when standardizing each of the feature values of the plurality of search behavior indicators in the driver state estimation processing. That is, the individual learning of the average value and variance of each of the feature values at the time when the driver is in the normal state is performed. The individual learning processing is started, for example, when the first trip is started on the day of vehicle 1.

[0048] When the individual learning processing is started, the controller 10 first recognizes the current driver based on, for example, the information received from the vehicle interior camera 32 (step S1).

[0049] Next, the controller 10 acquires the driving environment information, for example, based on the signals received from the sensors including at least one of the outside camera 21, the radar 22, the navigation system 23, the positioning system 24, the vehicle speed sensor 25, the acceleration sensor 26, the yaw rate sensor 27, the steering angle sensor 28, the steering torque sensor 29, the accelerator pedal sensor 30, and the brake sensor 31 (step S2).

[0050] Next, based on the driving environment information acquired in step S1, the controller 10 may determine whether a condition (a learning condition) for performing individual learning is met (step S3). Individual learning must be performed when the influence of the driving environment on the driver's search behavior is relatively small and when the driver is in the normal state.

[0051] For example, it is assumed that the influence of the driving environment on the driver's search behavior is relatively small when the following conditions are met: that the vehicle 1 is currently located in the urban area; that the vehicle speed is within a predetermined range (for example, 20 km / h or higher and lower than 60 km / h); that the road on which the vehicle is traveling is flat (for example, the gradient is less than 3%); that the road on which the vehicle is traveling is straight (for example, the curvature radius is 2000 m or larger); and that the current time is daytime. Furthermore, if there is no sudden driving or impact, it is assumed that a hazard avoidance operation or collision caused by the driver's inattentive state has not occurred, that is, the driver is in the normal state.

[0052] Thus, the controller 10 can determine that the condition for performing the individual learning is satisfied when the following are satisfied: that the current position of the vehicle 1 is the urban area, that the vehicle speed is within the predetermined range (for example, 20 km / h or higher and lower than 60 km / h), that the road on which the vehicle is traveling is flat (for example, the gradient is less than 3%), that the road on which the vehicle is traveling is straight (for example, the radius of curvature is 2000 m or larger), that the current time is the time of day and / or that the sudden driving operation or impact does not occur.

[0053] Consequently, if the condition for performing the individual learning is not satisfied (step S3: NO), the processing may return to step S2, and the processing in steps S2 and S3 is repeated until the condition for performing the individual learning is satisfied.

[0054] On the other hand, when the condition for performing the individual learning is satisfied (step S3: YES), the controller 10 detects the driver's line of sight based on the signal received from the vehicle interior camera 32 (step S4).

[0055] Next, the controller 10 can calculate the frequency x1 and amplitude x2 of the eye movement, specifically the saccade, based on the detected driver's line of sight (step S5). The saccade is one of the indicators related to the driver's search behavior. The saccade is a sudden eye movement to detect a visual target in the fovea of the retina and refers to an eye movement to move the line of sight from a gaze point where the line of sight stagnates or remains for a predetermined time to the next gaze point.

[0056] In the present embodiment, the amplitude and frequency of the saccade are used as the feature values of the saccade. The amplitude of the saccade refers to the amount of movement when the driver's line of sight moves from the viewpoint to the next viewpoint, and the frequency of the saccade refers to the number of times the line of sight moves from the viewpoint to the next viewpoint within a predetermined time.

[0057] For example, the controller 10 calculates the number of saccades per unit time as the saccade frequency x1 based on the number of saccades within the predetermined time (for example, 30 seconds). Furthermore, the controller 10 calculates the average value of the saccade amplitudes in the last predetermined time (for example, 30 seconds) as the saccade amplitude x2.

[0058] Next, based on the driving environment information acquired in step S1, the controller 10 can detect the object (attention object) to which the driver should pay attention in front of the vehicle 1 in the traveling direction (step S6). Examples of the attention object include another vehicle, an obstacle, a pedestrian, a traffic light, and a traffic sign.

[0059] Next, the controller 10 may calculate a top-down attention score x3 based on the driver's line of sight detected in step S4 and the attention object detected in step S6 (step S7). Top-down attention is one of the indicators related to the driver's search behavior and refers to an attention mechanism for actively moving the line of sight to a position intended by a person.

[0060] For example, if the driver recognizes in advance that the other vehicle is the attention object, the driver can actively direct their line of sight toward the other vehicle with priority over the other position. In the present embodiment, the top-down attention score is used as the feature value of top-down attention. The top-down attention score refers to a numerical value indicating the degree of deviation from the appropriate line of sight distribution to the attention object around vehicle 1.

[0061] For example, based on a top-down attention model created in advance and the driving environment information, the controller 10 calculates the appropriate number of glances and a glance time when the driver looks at each of the attention objects present in front of the vehicle 1 for a predetermined time (for example, 10 seconds). The top-down attention model is a mathematical expression in which a coefficient is set such that the appropriate number of glances and the glance time for each of the attention objects are calculated by substituting the vehicle speed, a time to collision (TTC) of the attention object, and a time at which the attention object exists within a visible range in front of the vehicle 1.The top-down attention model can be created in advance by conducting driving tests for a majority of subjects in the normal state using the driving simulator and learning the results of the driving tests. The top-down attention model is stored, for example, in the memory 10b.

[0062] For example, the controller 10 acquires the number of glances and the glance time when the driver looks at an object in front of the vehicle 1 from the driving environment information and the driver's line of sight. The controller 10 also calculates a difference between a predetermined number of glances and a predetermined glance time. Then, the controller 10 calculates the top-down attention score by multiplying the difference between the number of glances and the difference in glance time.

[0063] Specifically, the controller 10 can acquire, from the driving environment information and the driver's line of sight, the number of times of gaze and the gaze time when the driver gazes at each of the attention objects present in front of the vehicle 1 in the last predetermined time (for example, 10 seconds), and calculates, for each of the attention objects, differences from the appropriate number of gaze times and the gaze time calculated using the top-down attention model. Then, the controller 10 calculates, as a top-down attention score x3, a value obtained by multiplying an average value of the differences in the number of gaze times and an average value of the differences in the gaze times for each of the calculated attention objects.

[0064] Next, the controller 10 may acquire the conspicuity distribution for the last predetermined time (for example, 30 seconds) ahead of the vehicle 1 in the traveling direction based on the driving environment information acquired in step S1 (step S8). Conspicuity is a characteristic of attracting a person's gaze. That is, a highly conspicuous area in the driver's field of view is an area that easily attracts the driver's gaze due to, for example, a large color difference or a large luminance difference or a large movement with respect to the surrounding area. The controller 10 may acquire the conspicuity distribution by processing a temporal and spatial arrangement of colors, brightness, contrast, movement, and the like in the image acquired by the external camera 21 by a known image processing method.

[0065] Next, the controller 10 may calculate a bottom-up attention score x4 based on the driver's line of sight detected in step S4 and the saliency distribution acquired in step S8 (step S9). Bottom-up attention is one of the indicators related to the driver's search behavior and refers to an attention mechanism for passively moving the line of sight to a position of high saliency.

[0066] In the present embodiment, the bottom-up attention score is used as the feature value of bottom-up attention. The bottom-up attention score refers to a numerical value indicating the degree of deviation from the appropriate line-of-sight distribution to the attention object around the vehicle 1.

[0067] For example, controller 10 generates a ROC (Receiver Operating Characteristic) curve in which a probability that the saliency at a random point in front of vehicle 1 exceeds a predetermined threshold and a probability that the saliency in a direction in which the driver turns their line of sight exceeds a predetermined threshold are plotted in the last predetermined time (for example, 30 seconds) acquired based on the driving environment information and the driver's line of sight, while changing the predetermined thresholds. Then, controller 10 may multiply an area under the curve (AUC) of the ROC curve by a predetermined coefficient to calculate the bottom-up attention score x4.In this case, as the driver's tendency to direct his line of sight toward the high saliency object increases, the AUC approaches 1 as the maximum value and the bottom-up attention score is increased x4.

[0068] Next, the controller 10 stores the feature values x calculated in steps S5, S7 and S9 i (i=1, 2, 3, 4) in a learning database (step S10). The learning database is stored in the memory 10b.

[0069] Next, the controller 10 may determine whether a total time in which each of the feature values x i accumulated in the learning database after the start of the individual learning processing, that is, a time in which the learning condition is satisfied has reached a predetermined time (for example, 20 minutes) after the start of the individual learning processing (step S11). Consequently, when the total time in which each of the feature values x iis accumulated has not reached the predetermined time (step S11: NO), the processing returns to step S2 and the processing in steps S2 to S11 is repeated until the total time in which each of the feature values x i accumulated, reaches the predetermined time.

[0070] On the other hand, if the total time in which each of the characteristic values x i accumulated has reached the predetermined time (step S11: YES), the controller 10 generates an average value µ i and a variance σ i each of the feature values x i calculate (step S12).

[0071] Next, the controller 10 can calculate the mean value µ i and the variance σ i , calculated in step S12, in the memory 10b in association with the drivers detected in step S1 (step S13). Thereafter, the controller 10 terminates the individual learning processing. [Driver state estimation processing]

[0072] Next, the driver state estimation processing will be described with reference to Fig. 4. The driver state estimation processing may be started when the vehicle 1 is turned on and / or may be executed repeatedly by the controller 10 in a predetermined cycle (e.g., every 0.05 to 0.2 seconds). The driver state estimation processing may also be executed in parallel with the individual learning processing described with reference to Fig. 3 was described.

[0073] When the driver state estimation processing is started, the controller 10 first acquires the driving environment information based on the signals received from the sensors including the outside camera 21, the radar 22, the navigation system 23, the positioning system 24, the vehicle speed sensor 25, the acceleration sensor 26, the yaw rate sensor 27, the steering angle sensor 28, the steering torque sensor 29, the accelerator pedal sensor 30, and the brake sensor 31 (step S21).

[0074] Next, based on the driving environment information acquired in step S21, the controller 10 may determine whether an inattention determination condition for determining the driver's inattentive state, that is, a condition for estimating the driver's state, is satisfied (step S22). There is a driving scene in which the driver's line of sight is erroneously estimated as the inattentive state due to the driver's line of sight being concentrated in a narrow range. Examples of such a scene include a case where the vehicle 1 is traveling in a tunnel or a highway interchange, and a case where the vehicle 1 is changing lanes.

[0075] Accordingly, the driving scene likely to be erroneously estimated as the inattentive state and a driving scene in which the driver is unlikely to be in the inattentive state at all are defined in advance as driving scenes that are not subject to driver state estimation, respectively. Then, in the case where the driving scene identified from the driving environment information acquired in step S21 does not correspond to the driving scene that is not subject to driver state estimation, the controller 10 determines that the inattentiveness determination condition is met.

[0076] Consequently, when the inattention determination condition is not satisfied (step S22: NO), the controller 10 ends the driver state estimation processing.

[0077] On the other hand, when the inattention determination condition is satisfied (step S22: YES), the controller 10 detects the driver's line of sight based on the signal received from the vehicle interior camera 32 (step S23).

[0078] Next, the controller 10 specifically calculates the frequency x1 and the amplitude x2 of the saccade based on the detected driver's line of sight (step S24). The methods for calculating the frequency x1 and the amplitude x2 of the saccade are the same as those in step S5 for the individual learning processing.

[0079] Next, based on the driving environment information acquired in step S21, the controller 10 may detect the object (the attention object) to which the driver should pay attention in front of the vehicle 1 in the traveling direction (step S25).

[0080] Next, the controller 10 may calculate the top-down attention score x3 based on the driver's line of sight detected in step S23 and the attention object detected in step S25 (step S26). A method for calculating the top-down attention score x3 is the same as that in step S7 for the individual learning processing.

[0081] Next, the controller 10 may acquire the conspicuity distribution for the last predetermined time (for example, 30 seconds) ahead of the vehicle 1 in the traveling direction based on the traveling environment information acquired in step S21 (step S27).

[0082] Next, the controller 10 may calculate the bottom-up attention score x4 based on the driver's line of sight detected in step S23 and the saliency distribution acquired in step S27 (step S28). A method for calculating the bottom-up attention score x4 is the same as that in step S9 for the individual learning processing.

[0083] Next, the controller 10 may calculate each of the feature values x calculated in steps S24, S26 and S28 i (i=1, 2, 3, 4) based on the driving scene (step S29). Specifically, for each of the road gradient, the road curvature, the illuminance, and the vehicle speed, a correction coefficient map is created, in which a correction coefficient of the respective feature value x i is determined, stored in the memory 10b.

[0084] The controller 10 can detect the gradient and curvature of the road on which the vehicle 1 is traveling, the illuminance outside the vehicle 1, and the vehicle speed based on the driving environment information acquired in step S21, and acquires the correction coefficient corresponding to each of the detected road gradient, road curvature, illuminance, and vehicle speed by referring to the respective correction coefficient map stored in the memory 10b. Then, the controller 10 makes a correction by multiplying each of the feature values x i with the recorded correction coefficient.

[0085] Fig. 5A is an exemplary figure defining the correction coefficient of the saccade frequency x1 according to the road gradient. As the road gradient increases uphill or downhill, the driver tries to clearly check a road condition in the traveling direction, and thus the driver's line of sight tends to be concentrated in a narrow range. As a result, the saccade frequency x1 can be reduced and become a value similar to the saccade frequency in the inattentive state. Accordingly, the correction coefficient is set such that as the road gradient increases uphill or downhill, the correction coefficient of the saccade frequency x1 becomes greater than 1, that is, in a direction where the driver is less likely to be estimated in the inattentive state. In this way, the saccade frequency x1 can be corrected in such a way that the influence of the road gradient is canceled out.

[0086] Fig. 5B is an exemplary figure defining the correction coefficient of the saccade frequency x1 according to the road curvature. As the road curvature increases (i.e., the road becomes curved or banked), the driver tries to properly check the road condition in the traveling direction, and thus the driver's line of sight tends to be concentrated in the narrow area. As a result, the saccade frequency x1 can be reduced and become the similar value to the saccade frequency in the inattentive state. Accordingly, the correction coefficient is set such that as the road curvature increases, the correction coefficient of the saccade frequency x1 becomes greater than 1, that is, it is set in the direction in which the driver is less likely to be estimated in the inattentive state. In this way, the saccade frequency x1 can be corrected in such a way that the influence of the road curvature is canceled.

[0087] Fig. 5C is an exemplary figure defining the correction coefficient of the saccade frequency x1 according to the illuminance outside the vehicle. When the illuminance becomes low (i.e., the exterior of the vehicle becomes dark), the driver tries to check the road condition in the traveling direction well, and thus the driver's line of sight tends to be concentrated in the narrow range. As a result, the saccade frequency x1 can be reduced and become the similar value to the saccade frequency in the inattentive state. Accordingly, the correction coefficient is set such that as the illuminance decreases, the correction coefficient of the saccade frequency x1 becomes greater than 1, that is, set in the direction where the driver is less likely to be estimated in the inattentive state.In this way, the saccade frequency x1 can be corrected in such a way that the influence of the illuminance is eliminated.

[0088] Fig. 5D is an exemplary diagram defining a correction coefficient of the saccade frequency x1 according to the vehicle speed. As the vehicle speed increases, the driver's field of vision becomes narrower, and thus the driver's line of sight tends to be concentrated in the narrow area. As a result, the saccade frequency x1 can be reduced and become the similar value to the saccade frequency in the inattentive state. Accordingly, the correction coefficient is set such that as the vehicle speed increases, the correction coefficient of the saccade frequency x1 becomes greater than 1, that is, set in the direction in which it is less likely to be estimated that the driver is in the inattentive state. In this way, the saccade frequency x1 can be corrected in such a way that the influence of the vehicle speed is canceled.

[0089] Fig. 5 exemplifies the maps each defining the correction coefficient of the saccade frequency x1, and the correction coefficient maps can be analogously set for the saccade amplitude x2, the top-down attention score x3, and the bottom-up attention score x4, and can be stored in the memory 10b.

[0090] Next, the controller 10 may correct each of the feature values x i using the mean µ i and the variance σ i of the respective characteristic value x i standardize the values stored in the memory 10b during the individual learning processing (step S30). In this way, it is possible to standardize each of the feature values x i which reflects the current driver condition, where the feature value x i = 0 of the driver in the normal state is a reference value.

[0091] Next, based on the driving environment information acquired in step S21, the controller 10 can identify whether the road on which the vehicle 1 is traveling corresponds to an ordinary road or the expressway, and acquires a weight coefficient a i , which is calculated in advance for each of the feature values x corresponding to the identified road i is set (step S31). As described above, whether the estimated behavior of the driver corresponds to the case where the driver's inattentive state is simulated and the case where the driver's normal state is simulated is set as the response variables of the two values from the data on the movement of the line of sight acquired by the driving test using the driving simulator and the data on the driving environment simulated by the driving simulator, the value obtained by standardizing each of the feature values x iwas recorded is set as the explanatory variable, and the logistic regression analysis was performed to calculate the regression coefficient in advance. Such a regression coefficient is called the weight coefficient a i each of the feature values x i stored in the memory 10b.

[0092] Here, the driver's search behavior differs between the ordinary road, where the vehicle speed is low but there are a large number of attention objects such as pedestrians and intersections, and the expressway, where the vehicle speed is high but there are few attention objects such as no pedestrians, no intersection, or the like. Accordingly, in the present embodiment, a driving test simulating the ordinary road and a driving test simulating the expressway are conducted using the driving simulator, and the above-described logistic regression analysis is performed on each of the test results. In this way, the weight factor a i for each of the case where the vehicle is traveling on the ordinary road and the case where the vehicle is traveling on the expressway, and stored in the memory 10b.

[0093] Next, the controller 10 uses each of the feature values x standardized in step S30 i and the weight coefficient a detected in step S31 i to calculate the inattention probability p, which represents the probability that the driver is in the inattentive state, using the following sigmoid function and stores the calculated inattention probability p in the memory 10b (step S32). p=11+e−(a0+∑i=1naixi)

[0094] Next, the controller 10 detects the inattention probability p stored in the memory 10b and determines whether a state in which the inattention probability p is equal to or higher than a threshold p th (for example, 80%), continues for a predetermined time (for example, 16 seconds) or longer until a present time (step S33).

[0095] Consequently, if the state in which the inattention probability p is equal to or higher than the threshold p th does not continue for the predetermined time or longer until the present time (step S33: NO), the controller 10 judges that the driver's condition is normal (step S34), and ends the driver condition estimation processing.

[0096] On the other hand, if the state in which the inattention probability p is equal to or higher than the threshold p th is, continues for the predetermined time or longer until the present time (step S33: YES), the controller 10 that the driver is in the inattentive state.

[0097] Next, the controller 10 may transmit the control signal to the display 36 and the speaker 37 and cause the display 36 and the speaker 37 to output an alarm to notify the driver that the driver is in the inattentive state (step S36). At this time, the display 36 and the speaker 37 may be caused to output the image information and audio information (line-of-sight guidance information) for guiding the driver's line of sight to the attention object that the driver has not visually recognized. After step S36, the controller 10 terminates the driver state estimation processing.

[0098] Fig. 6 includes time diagrams showing temporal changes in the characteristic value x iand the inattention probability p of each of the search behavior indicators when a driving test simulating the urban area is conducted using the driving simulator. In Fig. 6A, B and C, a horizontal axis represents time. In addition, a vertical axis of Fig. 6A the value of each of the corrected and standardized characteristic values x i , a vertical axis of Fig. 6B indicates a i x i , which is calculated by multiplying each of the corrected and standardized characteristic values x i with the weight coefficient a i and a vertical axis from Fig. 6C denotes the inattention probability p.

[0099] In Fig. 6A and B, a dashed line indicates the saccade frequency x1, a one-dot dashed line indicates the saccade amplitude x2, a two-dot dashed line indicates the top-down attention score x3, and a dotted line indicates the bottom-up attention score x4. A solid line in Fig. 6B denotes a sum of a i x i and a solid line in Fig. 6C denotes the inattention probability p.

[0100] By correcting and standardizing the individual characteristic values x i in driver state estimation processing, as in Fig. As illustrated in Figure 6A, the influence of the driving scene on the search behavior can be eliminated and each of the feature values x i can be evaluated using 0 as a common reference value. In the example from Fig. Figure 6A shows that the saccade frequency x1 (the dashed line), the saccade amplitude x2 (the one-dotted line), and the bottom-up attention score x4 (the dotted line) are relatively far from 0.

[0101] Furthermore, by multiplying each of the feature values x i with the weight coefficient a i , which is captured by the logistic regression analysis, the evaluation taking into account a size of the influence of the respective characteristic value x i to estimate whether the driver is in the inattentive state. In the example from Fig. 6A, the saccade frequency x1 (the dashed line), the saccade amplitude x2 (the one-dot dashed line) and the bottom-up attention score x4 (the dotted line) are relatively far from 0. In contrast, according to Fig. 6B, the saccade frequency x1 (the dashed line) indicates a value that is relatively larger than 0 compared to the saccade amplitude x2 (the one-dot dashed line) and the bottom-up attention score x4 (the dotted line) (especially between time t1 and time t2).

[0102] If the inattention probability p is calculated using the Fig. 6B illustrated product a i x i from each of the characteristic values x i and the respective weight coefficient a i is calculated as in Fig. As illustrated in Figure 6C, the inattention probability p is equal to or higher than the threshold p th between time t1 and time t2. Thus, in the case where a time from time t1 to time t2 is equal to or longer than a predetermined time (for example, 18 seconds), it is estimated that the driver is in the inattentive state. [Modified examples]

[0103] In the above-described embodiment, the description was made that the saccade frequency x1 and amplitude x2, the top-down attention score x3, and the bottom-up attention score x4 are used as the feature values of the majority of the indicators of the driver's search behavior. However, some of these may be used in combination, or a feature value of another indicator may be combined.

[0104] Furthermore, in the above-described embodiment, the description has been made that the controller 10 performs the correction by multiplying each of the feature values x i with the correction coefficient. However, the correction can be made by adding or subtracting the correction coefficient to or from each of the feature values x i be made. [Functionality / Effects]

[0105] Next, the operation and effects of the driver state estimation device 100 in the present embodiment described above will be described.

[0106] For each of the plurality of indicators of the search behavior changed according to the driver's condition, the controller 10 detects the feature values x i , which include: the frequency and amplitude of the driver's saccade; and the top-down attention score, which indicates the degree of deviation from the appropriate line-of-sight distribution to the attention object around the vehicle 1, and uses the acquired feature values x i , the pre-determined weight coefficient a i for each of the characteristic values x iand the pre-specified constant a0 to calculate the inattention probability p through the sigmoidal function. Accordingly, instead of merely focusing on any one indicator of the driver's search behavior, unique changes in the saccade frequency and amplitude and the top-down attention score in the driver's inattentive state are comprehensively detected to quantitatively evaluate the probability that the driver is in the inattentive state. In this way, it is possible to estimate the inattentive state by distinguishing the changes in the feature values therein from those caused by the driver's illness, aging, or the like.

[0107] In particular, since the controller 10 each of the detected feature values x i corrected based on the driving environment information, possible to adjust the feature values x iin such a way as to eliminate the influence of the driving environment of vehicle 1, thereby more accurately calculating the inattention probability p. Thus, it is possible to prevent an erroneous estimation of the driver's state caused by the driving environment.

[0108] Furthermore, in particular, the controller 10 corrects the feature value x with increasing gradient of the road on which the vehicle 1 is traveling. i in the direction in which the driver is less likely to be estimated in the inattentive state. Accordingly, if the driver is likely to be estimated in the inattentive state due to the steep gradient of the road and the tendency for the driver's line of sight to be concentrated in the narrow area, it is possible to estimate the feature value x iin such a way that the influence of the gradient of the road is eliminated and thus the inattention probability p is calculated even more accurately.

[0109] Furthermore, in particular, the controller 10 corrects the feature value x with increasing curvature of the road on which the vehicle 1 is traveling. i in the direction in which the driver is less likely to be estimated in the inattentive state. Accordingly, if the driver is likely to be estimated in the inattentive state due to the large curvature of the road and the tendency for the driver's line of sight to be concentrated in the narrow area, it is possible to estimate the feature value x i in such a way that the influence of the curvature of the road is eliminated and thus the inattention probability p is calculated even more accurately.

[0110] Furthermore, in particular, the controller 10 corrects the feature value x with decreasing illuminance outside the vehicle 1 i in the direction in which the driver is less likely to be estimated in the inattentive state. Accordingly, if the driver is likely to be estimated in the inattentive state due to the low illuminance outside the vehicle and the tendency for the driver's line of sight to be concentrated in the narrow area, it is possible to estimate the feature value x i in such a way that the influence of the illuminance is eliminated and thus the inattention probability p is calculated even more accurately.

[0111] Furthermore, in particular, the controller 10 corrects the feature value x with increasing speed of the vehicle 1 iin the direction in which the driver is less likely to be estimated in the inattentive state. Accordingly, if the driver is likely to be estimated in the inattentive state due to the high vehicle speed and the tendency for the driver's line of sight to be concentrated in the narrow area, it is possible to estimate the feature value x i in such a way that the influence of the vehicle speed is eliminated and thus the inattention probability p is calculated even more accurately. List of reference symbols 1 vehicle 10 controllers 100 Driver condition estimation device 21 outdoor camera 22 radars 23 Navigation system 24 Positioning system 25 Vehicle speed sensor 26 Accelerometer 27 Yaw rate sensor 28 Steering angle sensor 29 Steering torque sensor 30 Accelerator pedal sensor 31 Brake sensor 32 vehicle interior camera 36 ad 37 speakers QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] JP 2017-224066A

[0003]

Claims

[1] A driver state estimation device (100) that estimates a state of a driver driving a vehicle (1), the driver state estimation device (100) comprising: a driving environment information acquisition device (21 to 31) configured to acquire driving environment information of the vehicle (1); a line-of-sight detection device (32) configured to detect a line of sight of the driver; and a controller (10) configured to estimate whether the driver is in an inattentive state based on the driving environment information and the driver's line of sight, wherein the controller (10) is configured, a characteristic value x i (i = 1, ..., n) detect each of a plurality of indicators of search behavior changed according to the driver's condition based on the driving environment information and / or the driver's line of sight, an inattention probability p, which represents a probability that the driver is in the inattentive state, by the following equation using the detected feature value x i , a predetermined weight coefficient a i for each of the characteristic values x i and a predetermined constant a0, p=11+e−(a0+∑i=1naixi), and to estimate that the driver is in the inattentive state when a state in which the calculated inattention probability p is equal to or higher than a predetermined value persists for a predetermined time or longer, and the feature values x iinclude: a frequency and an amplitude of a driver's saccade, which are detected based on the driver's line of sight; and a top-down attention score indicating a degree of deviation from a convenient line-of-sight distribution to the attention object around the vehicle (1), which is detected based on the driving environment information and the driver's line of sight. [2] The driver state estimation device (100) according to claim 1, wherein the controller (10) is configured to calculate each of the detected feature values x i , based on driving environment information. [3] Driver condition estimation device (100) according to one of the preceding claims, wherein the controller (10) is configured to detect an incline of a road on which the vehicle (1) is traveling based on the driving environment information, and the controller (10) is configured to increase the feature value x with increasing gradient i in a direction where it is less likely that the driver will be estimated in the inattentive state, or the feature value x i to correct it so that it is larger with increasing gradient. [4] Driver state estimation device (100) according to one of the preceding claims, wherein the controller (10) is configured to detect a curvature of a road on which the vehicle (1) is traveling based on the driving environment information, and the controller (10) is configured to increase the feature value x with increasing curvature i in a direction where it is less likely that the driver will be estimated in the inattentive state, or the feature value x i to correct it so that it is larger with increasing curvature. [5] Driver state estimation device (100) according to one of the preceding claims, wherein the controller (10) is configured to detect an illuminance outside the vehicle (1) based on the driving environment information, and the controller (10) is configured to decrease the characteristic value x with decreasing illuminance i in a direction where it is less likely that the driver will be estimated in the inattentive state, or the feature value x i to correct it so that it is larger with decreasing illuminance. [6] Driver state estimation device (100) according to one of the preceding claims, wherein the controller (10) is configured to detect a speed of the vehicle (1) based on the driving environment information, and the controller (10) is configured to increase the characteristic value x with increasing speed iin a direction where it is less likely that the driver will be estimated in the inattentive state, or the feature value x i to correct it so that it is larger with increasing speed. [7] Driver state estimation device (100) according to one of the preceding claims, wherein the controller (10) is configured to record a number of glances and a glance time from the driving environment information and the driver's line of sight when the driver looks at an object in front of the vehicle (1), to calculate a difference between a predetermined number of times of looking and a predetermined looking time, and than calculating the top-down attention score by multiplying the difference in the number of times of looking and the difference in looking time. [8] The driver state estimation device (100) according to any one of the preceding claims, further comprising an output device (36, 37), wherein the output device (36, 37) is configured to output an alarm when the driver is estimated to be in the inattentive state, or when the state in which the calculated inattentiveness probability p is equal to or higher than the predetermined value continues for the predetermined time or longer. [9] Vehicle (1) comprising the driver state estimation device (100) according to any one of the preceding claims. [10] A method for determining a state of a driver driving a vehicle (1), the method comprising: Acquiring driving environment information of the vehicle (1); Detecting a driver’s line of sight; Recording a characteristic value x i(i = 1, ..., n) each of a plurality of indicators of search behavior changed according to the driver's condition based on the driving environment information and / or the driver's line of sight; and Calculate a probability p by the following equation using the detected feature value x i , a predetermined weight coefficient a i for each of the characteristic values x i and a predetermined constant a0, p=11+e−(a0+∑i=1naixi); and Determining whether a state in which the calculated probability p is equal to or higher than a predetermined value persists for a predetermined time or longer, wherein the feature values x iinclude: a frequency and an amplitude of a driver's saccade, which are detected based on the driver's line of sight; and a top-down attention score indicating a degree of deviation from a convenient line-of-sight distribution to the attention object around the vehicle (1), which is detected based on the driving environment information and the driver's line of sight.

Images