DRIVER STATE ESTIMATION DEVICE AND DRIVER STATE DETERMINATION METHOD

Abstract

A driver state estimation device (100) includes: an in-vehicle camera (32) that detects a driver's line of sight; and a controller (10) configured to estimate a driver's state based on the driver's line of sight. The controller sets a predetermined visual range including a line-of-sight direction when the driver directs their line of sight in a traveling direction, detects a distribution of driver's viewpoints within a predetermined time based on the driver's line of sight, and estimates that the driver is in an abnormal state when a proportion of viewpoints included in the predetermined visual range among all viewpoints included in the detected distribution of viewpoints is equal to or greater than a predetermined proportion.

Description

[Technical field]

[0001] The present invention relates to a driver state estimation device that estimates a state of a driver driving a vehicle. The present invention also relates to a method for determining a state of the driver. [Technical background]

[0002] Conventionally, a driver condition detection device for detecting an abnormality of a driver of a vehicle has been proposed (see, for example, Patent Literature 1). The device described in Patent Literature 1 detects the amplitude and frequency of a saccade of the vehicle driver (a jumping eye movement of the driver to intentionally move a line of sight), and also detects a degree of caution that increases with the increase in the number of warning points in an external vehicle environment that the driver needs to check during driving. Thus, the driver's abnormality is detected based on the degree of caution and the amplitude and frequency of the driver's saccade. [List of citations][Patent literature]

[0003] [Patent Literature 1] JP2021-077136A [Summary][Technical Problem]

[0004] In a driving environment that requires a rapid visual search over a wide range, such as a situation where the driver is attempting to make a right turn at an intersection and needs to quickly move the line of sight over a wide range to simultaneously check a large number of warning objects such as a vehicle in an oncoming lane and a pedestrian about to walk on a crossing, the amplitude of the saccade increases when the driver's condition is normal, whereas the amplitude of the saccade does not increase and tends to remain at a low value when the driver's condition is abnormal due to the onset of a brain disorder or the like. Thus, it is possible to estimate whether the driver's condition is normal based on the amplitude of the saccade.

[0005] However, in a driving environment that does not require rapid visual search in a wide area, such as a situation where there are a few other vehicles nearby and the number of warning objects is small while driving on an expressway, the amplitude of the saccade does not increase even when the driver's condition is normal. Accordingly, there is no clear difference in the amplitude of the saccade between the case where the driver's condition is normal and the case where the driver's condition is abnormal. In this case, it is difficult to accurately estimate the driver's condition using the conventional technique described above. That is, it may be difficult to accurately estimate the driver's condition depending on the driving environment.

[0006] The invention was made to solve such a problem and therefore has the task of accurately estimating a driver's condition regardless of a driving environment. [Solution to the problem]

[0007] The above-described problem is solved by the invention defined in independent claims. Specifically, a driver state estimation device that estimates a state of a driver driving a vehicle includes: a line-of-sight detector that detects a line of sight of the driver; and a controller configured to estimate the driver's state based on the driver's line of sight. The controller may be configured to set a predetermined visual range including a line-of-sight direction when the driver directs their line of sight in a traveling direction of the vehicle. The controller is configured to detect a distribution of the driver's viewpoints within a predetermined time based on the driver's line of sight.The controller is further configured to estimate that the driver is in an abnormal state when a proportion of the viewpoints included in the predetermined view range among all the viewpoints included in the acquired distribution of view points is equal to or greater than a predetermined proportion, or to determine whether a proportion of the viewpoints included in the predetermined view range among all the viewpoints included in the acquired distribution of view points is equal to or greater than a predetermined proportion.

[0008] According to the present invention, the controller estimates that the driver is in the abnormal state when the proportion of viewpoints included in the predetermined visual range, which includes the line-of-sight direction of the case where the driver directs their line of sight toward the vehicle's advancing direction, among all the viewpoints included in the driver's viewpoint distribution within the predetermined time is equal to or greater than the predetermined threshold. Thus, based on the proportion of viewpoints included in the predetermined visual range, it is possible to identify that a tendency of the driver's line of sight to concentrate near the advancing direction is increasing due to some disturbance, and thus it is possible to accurately estimate that the driver is in the abnormal state regardless of the driving environment.

[0009] The driver state estimation device may further include a driving environment information acquisition device that acquires driving environment information of the vehicle, and the controller may be configured to set a size of the predetermined view area based on the driving environment information.

[0010] According to the embodiment as described above, since the controller sets the size of the predetermined visual range based on the driving environment information, by setting the predetermined visual range to the appropriate size according to the driving environment information, it is possible to more accurately estimate whether the driver's condition is abnormal according to the driving environment.

[0011] The controller may be configured to: detect the number of objects based on the driving environment information; and set the predetermined visual range larger than that in the case where the number of objects is less than a predetermined threshold when the number of warning objects is equal to or greater than the predetermined threshold. Specifically, the controller may be configured to: detect the number of objects or warning objects near the vehicle based on the driving environment information; and set the predetermined visual range larger than that in the case where the number of objects or warning objects is less than a predetermined threshold when the number of objects or warning objects is equal to or greater than the predetermined threshold.Furthermore, in particular, the controller can be configured to set the predetermined field of view larger as the number of objects or warning objects increases.

[0012] According to the embodiment described above, when the number of objects or warning objects is equal to or greater than the predetermined threshold, the controller sets the predetermined visual range larger than that in the case where the number of warning objects is less than the predetermined threshold. Accordingly, based on the characteristic that the tendency of the driver's line of sight in the abnormal state to concentrate on the vicinity of the vehicle's traveling direction increases with the relatively small number of warning objects, it is possible to set the predetermined visual range to such a size that the difference from the driver in a normal state becomes noticeable, and thus it is possible to more accurately estimate whether the driver's state is abnormal according to the driving environment.

[0013] The controller may be configured to set, as the predetermined visual range, a circular area centered on an average direction of the distribution of the line-of-sight directions within a predetermined time based on the driver's line of sight.

[0014] According to the embodiment as described above, since the controller sets the circular area centered on the average direction of the distribution of the line-of-sight directions within the predetermined time as the predetermined view area, it is possible to set the predetermined view area serving as a reference for estimating the driver's state based on the line-of-sight directions of the actual driver, and thus it is possible to more accurately estimate whether the driver's state is abnormal. [Effects of the invention]

[0015] According to the invention, it is possible to accurately estimate the driver's condition regardless of the driving environment. [Brief description of the drawings] [ Fig. 1] Fig. 1 is a view illustrating a vehicle on which a driver condition estimation device according to an embodiment of the invention is mounted. [ Fig. 2] Fig. 2 is a block diagram of the driver state estimation device according to the embodiment of the invention. [ Fig. 3] Fig. 3 is a view illustrating an example of a distribution of viewpoints of a driver who visually recognizes a traveling direction of the vehicle. [ Fig. 4] Fig. 4 is a diagram illustrating a relationship between a size of a center area and a center check proportion p. [ Fig. 5] Fig. 5 is a diagram illustrating a difference Δp between a center check percentage p u of a subject with a brain disorder and a center check percentage p n of a subject without disturbance. [ Fig. 6] Fig. 6 is a flowchart of driver state estimation processing according to the embodiment of the invention. [Description of embodiments]

[0016] Hereinafter, a driver condition estimation apparatus according to an embodiment of the invention will be described with reference to the accompanying drawings. [System configuration]

[0017] First, a configuration of the driver state estimation device according to the present embodiment will be described with reference to Fig. 1 and Fig. 2 described.

[0018] Fig. 1 is a view illustrating a vehicle on which the driver condition estimation device is mounted, and Fig. 2 is a block diagram of the driver state estimation device.

[0019] A vehicle 1 according to the present embodiment may include: a driving power source 2 such as an internal combustion engine or an electric motor that outputs a driving force; a transmission 3 that transmits the driving force output from the driving power source 2 to drive wheels; a brake 4 that applies a braking force to the vehicle 1; and a steering device 5 for steering the vehicle 1.

[0020] A driver state estimation device 100 is configured to estimate a state of a driver of the vehicle 1 and can execute control of the vehicle 1 and driver assistance control as needed. As shown in Fig. As illustrated in Figure 2, the driver state estimation device 100 includes a controller 10, one or a plurality of sensors, one or a plurality of control systems, and one or a plurality of information output devices.

[0021] For example, the plurality of sensors may include an exterior camera 21 and / or a radar 22 for acquiring driving environment information of the vehicle 1, and a navigation system 23 and a positioning system 24 for detecting a position of the vehicle 1. Alternatively or additionally, the plurality of sensors may include at least one of a vehicle speed sensor 25, an acceleration sensor 26, a yaw rate sensor 27, a steering angle sensor 28, a steering torque sensor 29, an accelerator pedal sensor 30, and a brake sensor 31 for detecting a behavior of the vehicle 1 and a driving operation by the driver. Alternatively or additionally, the plurality of sensors may include a vehicle interior camera 32 for detecting a driver's line of sight.

[0022] The plurality of control systems may include at least one of a powertrain control module (PCM) 33 that controls the power source 2 and the transmission 3, a dynamic stability control (DSC) system 34 that controls the power source 2 and the brake 4, and a power steering system (EPS) 35 that controls the steering device 5. The plurality of information output devices includes a display 36 that outputs image information and a speaker 37 that outputs audio information.

[0023] For example, other sensors may include: a peripheral sonar that measures a distance to and position of a structure around the vehicle 1; corner radars that each measure an approach of the peripheral structure at a respective one of four corners of the vehicle 1; and various sensors that each estimate the driver's condition (e.g., a heart rate sensor, an electrocardiogram sensor, a steering wheel grip force sensor, and the like).

[0024] For example, the controller 10 performs various calculations based on signals received from the plurality of sensors, transmits control signals to the PCM 33, the DSC 34, and the EPS 35 for appropriately operating the drive power source 2, the transmission 3, the brake 4, and the steering device 5, and transmits control signals for causing the display 36 and the speaker 37 to output desired information. The controller 10 may be configured by a computer including one or more processors 10a (typically, CPUs), a memory 10b (ROM, RAM, and the like) for storing various programs and data, an input / output device, and the like.

[0025] The external camera 21 captures an image of the surroundings of the vehicle 1 and outputs image data. The controller 10 detects an object (e.g., a preceding vehicle, a parked vehicle, a pedestrian, a roadway, dividing lines (a lane line, a white line, and a yellow line), a traffic signal, a traffic sign, a stop line, an intersection, an obstacle, and the like) based on the image data received from the external camera 21. The external camera 21 corresponds to an example of the "driving environment information acquisition device."

[0026] The radar 22 measures a position and a speed of the object (specifically, the preceding vehicle, the parked vehicle, the pedestrian, a dropped object on the roadway, or the like). A millimeter-wave radar, for example, can be used as the radar 22. The radar 22 transmits a radio wave in a traveling direction of the vehicle 1 and receives a reflected wave generated when the transmitted wave is reflected by the object. Then, the radar 22 measures a distance (for example, a distance between vehicles) between the vehicle 1 and the object and a relative speed of the object to the vehicle 1 based on the transmitted wave and the received wave.

[0027] In the present embodiment, instead of radar 22, a laser radar, an ultrasonic sensor, or the like may be used to measure the distance to the object and its relative speed. Alternatively, a plurality of sensors may be used to form a position and speed measuring device. Radar 22 corresponds to an example of the "driving environment information acquisition device."

[0028] The navigation system 23 stores map information therein and can provide the map information to the controller 10. The controller 10 can identify a road, intersection, traffic signal, building, and the like that exist around the vehicle 1 (particularly in the traveling direction of the vehicle) based on the map information and current vehicle position information. The map information can be stored in the controller 10. The positioning system 24 is a GPS system and / or a gyroscopic system and detects the position of the vehicle 1 (the current vehicle position information). The navigation system 23 and the positioning system 24 also correspond to examples of the "driving environment information acquisition device."

[0029] The vehicle speed sensor 25 detects a speed of the vehicle 1 based, for example, on a rotational speed of the wheel or drive shaft. The acceleration sensor 26 detects an acceleration of the vehicle 1. This acceleration includes an acceleration in a longitudinal direction of the vehicle 1 and an acceleration in a lateral direction (i.e., a lateral acceleration) thereof. The acceleration may include not only a rate of change of the speed in a speed-increasing direction, but also a rate of change of the speed in a speed-decreasing direction (i.e., deceleration).

[0030] The yaw rate sensor 27 detects a yaw rate of the vehicle 1. The steering angle sensor 28 detects a rotation angle (steering angle) of a steering wheel of the steering device 5. The steering torque sensor 29 detects a torque (steering torque) applied to a steering shaft via the steering wheel. The accelerator pedal sensor 30 detects the amount of depression of an accelerator pedal. The brake sensor 31 detects the amount of depression of a brake pedal.

[0031] The in-vehicle camera 32 captures an image of the driver and outputs image data. The controller 10 detects the driver's line of sight based on the image data received from the in-vehicle camera 32. The in-vehicle camera 32 corresponds to an example of the "line of sight detection device" in the invention.

[0032] The PCM 33 controls the driving power source 2 of the vehicle 1 to adjust the driving power of the vehicle 1. For example, the PCM 33 controls a spark plug, a fuel injection valve, a throttle valve and a variable valve mechanism of the internal combustion engine, the transmission 3, an inverter that supplies electrical power to the electric motor, and the like. When the vehicle 1 needs to be accelerated or decelerated, the controller 10 transmits a control signal to the PCM 33 to adjust the driving power.

[0033] The DSC 34 controls the driving power source 2 and the brake 4 of the vehicle 1, and performs deceleration control and attitude control of the vehicle 1. For example, the DSC 34 controls a hydraulic pump, a valve unit, and the like of the brake 4, and controls the driving power source 2 via the PCM 33. When the deceleration control or attitude control of the vehicle 1 needs to be executed, the controller 10 transmits a control signal to the DSC 34 for adjusting the driving force or generating the braking force.

[0034] The EPS 35 controls the steering device 5 of the vehicle 1. For example, the EPS 35 controls an electric motor that applies torque to the steering shaft of the steering device 5, and the like. When the direction of travel of the vehicle 1 needs to be changed, the controller 10 transmits a control signal to the EPS 35 to change a steering direction.

[0035] The display 36 is provided in front of the driver in a cabin and displays image information for the driver. A liquid crystal display or a head-up display, for example, is used as the display 36. The speaker 37 is installed in the cabin and outputs various types of audio information. [Overview of Driver State Estimation]

[0036] Next, an overview of the driver state estimation by the driver state estimation device 100 of the present embodiment will be described with reference to Fig. 3 to Fig. 5 described. Fig. 3 is a view illustrating an example of a distribution of viewpoints of the driver who visually recognizes the traveling direction of the vehicle. Fig. 4 is a diagram illustrating a relationship between a size of a center region and a center check ratio p representing a ratio of viewpoints included in the center region to all viewpoints. Fig. 5 is a diagram illustrating a difference Δp between a center check percentage p u in an anomalous state and a center check proportion p n in a normal state.

[0037] The present inventors conducted driving tests with a plurality of subjects with a brain disorder and a plurality of healthy subjects without a disorder using a driving simulator to investigate how a driver's behavior of visually checking the surroundings of the vehicle (particularly in the vehicle's traveling direction) (hereinafter referred to as "search behavior") changes between a case where the driver's condition was normal and a case where the driver's condition was abnormal. Specifically, each of the drivers was made to drive in different driving environments (an ordinary road without an intersection, an ordinary road with an intersection, an expressway, and the like) in which the number of objects (warning objects) to which the driver should pay attention was different, and a movement of the line of sight of each of the subjects during driving was measured.

[0038] Consequently, it was found that the healthy subjects without the disturbance (corresponding to the drivers in the normal state) repeatedly showed such a searching behavior that each of the subjects basically directed his line of sight to the vicinity of the traveling direction of the vehicle, temporarily moved his line of sight in a direction separate from the traveling direction toward the warning object (for example, another vehicle, a rearview mirror, a side mirror, or the like), and returned his line of sight to the vicinity of the traveling direction, thereby checking the surroundings of the vehicle.

[0039] In contrast, in the subjects with the brain disorder (which correspond to the drivers in the abnormal state), the result was such that a large proportion of the subjects directed their line of sight toward the vicinity of the vehicle's direction of travel and were less likely to move their line of sight away from the direction of travel compared to the subjects without the disorder.

[0040] Accordingly, to quantitatively evaluate the proportion of directing the line of sight toward the vicinity of the vehicle's traveling direction (e.g., the forward direction), the present inventors extracted the viewpoints where the line of sight stagnated or remained for a predetermined time (e.g., 0.3 seconds or approximately 0.3 seconds) for each of the subjects with the brain disorder and the subjects without the disorder, and acquired a distribution of the viewpoints. Then, the present inventors identified, as in Fig. 3 shows the number of viewpoints as an example (e.g. in Fig. 3 points marked by black circles) located in a predetermined field of view (hereinafter referred to as the “central area”, which is an area of ​​the Fig. 3 by an imaginary line) which includes the line of sight direction when the driver directed his line of sight in the direction of travel of the vehicle, and the number of viewpoints outside the central area (e.g. in Fig. 3 points marked by white circles) and calculated the proportion or ratio of viewpoints contained in the central area to all viewpoints (hereinafter referred to as the “center review proportion”).

[0041] Fig. Figure 4 illustrates a relationship between the size of the center area and the center check percentage p when the driving test was conducted using the driving simulator on the ordinary road without the intersection. Fig. 4, a horizontal axis represents the size of the center area (a value obtained by plotting a radius of the center area through a visual angle with a driver’s head position as a reference) and a vertical axis represents the center check ratio p. In addition, a dashed line in Fig. 4 a center check portion p u of subjects with the brain disorder and a solid line indicates the center check proportion p n of the subjects without the disturbance.

[0042] As in Fig. 4 illustrates the center verification proportion p regardless of the size of the center area. uof subjects with the brain disorder greater than the center-check proportion p n of subjects without the disorder. This indicates that the percentage of subjects who direct their line of sight toward the vicinity of the vehicle's direction of travel is higher in subjects with the brain disorder than in subjects without the disorder. Furthermore, in a range of 2 degrees to 15 degrees, there is a difference between the center verification percentage p u of subjects with brain disorder and center check percentage p n of subjects without the disturbance is particularly large. The size of the central region can be defined by a diameter instead of an angle.

[0043] This illustrates Fig. 5 the difference Δp between the center check portion p u of subjects with brain disorder and center check percentage p n of the subjects without the disturbance. In Fig. 5 a horizontal axis represents the size of the central area and a vertical axis represents Δp = p u - p n In addition, a solid line in Fig. 5 Δp at the time of conducting the driving test on the ordinary road without the intersection (i.e., a driving environment in which the number of warning objects is average), a one-dot chain line indicates Δp at the time of conducting the driving test on the ordinary road with the intersection (i.e., a driving environment in which the number of warning objects is relatively large), and a dashed line indicates Δp at the time of conducting the driving test on the expressway (i.e., a driving environment in which the number of warning objects is relatively small).

[0044] As in Fig. 5 illustrates that regardless of the driving environment, Δp has a peak value in a range where the size of the center area is 2 degrees to 15 degrees. That is, there is a general tendency that the center check proportion p u of the subject with the brain disorder is greater than the center check proportion p n of the subject without the disturbance. Furthermore, the size of the central area where Δp reaches the peak is maximum (6 degrees) in the case of the driving environment where the number of warning objects is relatively large (the one-dotted line in Fig. 5), becomes minimal in the case of the driving environment (3 degrees), where the number of warning objects is relatively small (the dashed line in Fig. 5), and has an intermediate value (4 degrees) in the case of the driving environment where the number of warning objects is average (the solid line in Fig. 5). This indicates that since the tendency for the line of sight of the subject with the brain disorder to concentrate near the vehicle's traveling direction is stronger as the number of warning objects decreases than that of the subject without the disorder, the difference in the center check ratio p becomes larger, that is, a difference in the distribution of viewpoints becomes apparent as the center area decreases. For this reason, by calculating the center check ratio p from the driver's viewpoint distribution, it is possible to accurately estimate whether the driver's condition is abnormal regardless of the driving environment. It is possible to more accurately estimate whether the driver's condition is abnormal by adjusting the center area to an appropriate size according to the number of warning objects according to the driving environment. [Driver state estimation processing]

[0045] Next, a flow of driver state estimation processing executed by the driver state estimation device 100 according to the present embodiment will be described with reference to Fig. 6 described. Fig. 6 is a flowchart of driver state estimation processing for estimating the driver's state.

[0046] The driver state estimation processing in Fig. 6 is started, for example, when the vehicle 1 is turned on, and can be executed by the controller 10 repeatedly in a predetermined cycle (for example, every 0.05 to 0.2 seconds or about 0.05 to about 0.2 seconds).

[0047] When the driver state estimation processing is started, the controller 10 may acquire the driving environment information based on the signals received from the sensors including the outside camera 21, the radar 22, the navigation system 23, the positioning system 24, the vehicle speed sensor 25, the acceleration sensor 26, the yaw rate sensor 27, the steering angle sensor 28, the steering torque sensor 29, the accelerator pedal sensor 30, and / or the brake sensor 31 (step S1).

[0048] Next, the controller 10 detects the driver's line of sight, for example, based on the signal received from the vehicle interior camera 32 (step S2).

[0049] Next, based on the driving environment information acquired in step S1, the controller 10 can detect the object (the warning object) to which the driver should pay attention in front of the vehicle 1 in the traveling direction (step S3). Examples of the warning object include another vehicle, the obstacle, the pedestrian, a traffic light, and / or a traffic sign.

[0050] Next, based on the driver's line of sight detected in step S2, the controller 10 detects the driver's viewpoints, specifically, the distribution of the driver's viewpoints, for a first predetermined time, that is, the most recent predetermined time (for example, 30 seconds or approximately 30 seconds) (step S4). For example, if it is detected based on the driver's line of sight detected in step S2 that the driver's line of sight has stagnated for a second predetermined time (for example, 0.3 seconds or approximately 0.3 seconds), which may be shorter than the first predetermined time, the controller 10 identifies a position of the viewpoint by expressing a direction of the line of sight, that is, a direction from the driver's head position toward the viewpoint, by a combination of an azimuth and an elevation angle with the traveling direction (or forward direction) of the vehicle as a reference.

[0051] Then, the positions of the identified viewpoints are accumulated in the memory 10b. The viewpoint position may be repeatedly accumulated during the execution of the driver state estimation processing. Then, in step S4, the controller 10 reads the viewpoint positions for the most recent predetermined time accumulated in the memory 10b to detect the viewpoint distribution.

[0052] Next, the controller 10 determines whether the number of warning objects detected in step S3 is equal to or greater than a predetermined threshold N1 (step S5). N1 is set in advance and stored in the memory 10b. The driving environment in which the number of warning objects is less than N1 corresponds to a driving environment in which the number of warning objects is relatively small, such as an expressway.

[0053] Therefore, if the number of warning objects is not equal to or greater than N1 (i.e., less than N1) (step S5: NO), the controller 10 sets a center area A1 with a size corresponding to the driving environment where the number of warning objects is relatively small, and calculates the proportion of viewpoints included in the center area A1 (the center check proportion p) (step S6). The center area A1 can be set as a circular area centered on an average direction of the distribution of the line-of-sight directions for the most recent predetermined time (for example, 30 seconds). Furthermore, the size of the center area A1 (the value obtained by representing the radius of the center area by the visual angle with the driver's head position as the reference) is set to, for example, 3 degrees based on the result of the driving test described above.

[0054] On the other hand, if the number of warning objects is equal to or greater than N1 (step S5: YES), the controller 10 may determine whether the number of warning objects detected in step S3 is equal to or greater than a predetermined threshold value N2 (step S7). N2 is a value greater than N1, is set in advance, and is stored in the memory 10b.

[0055] The driving environment at a time when the number of warning objects is equal to or greater than N1 and less than N2 corresponds to a driving environment where the number of warning objects is average, such as an ordinary road without an intersection. The driving environment at a time when the number of warning objects is equal to or greater than N2 corresponds to a driving environment where the number of warning objects is relatively large, such as an ordinary road with an intersection.

[0056] Therefore, if the number of warning objects is not equal to or greater than the predetermined threshold N2 (i.e., equal to or greater than N1 and less than N2) (step S7: NO), the controller 10 may set a center area A2 with a size corresponding to the driving environment in which the number of warning objects is average, and calculates the proportion of viewpoints included in the center area A2 (the center check proportion p) (step S8). The center area A2 is set as a circular area centered on the average direction of the distribution of the line-of-sight directions for the most recent predetermined time (for example, 30 seconds). In addition, the size of the center area A2 is set to, for example, 4 degrees based on the result of the driving test described above.

[0057] On the other hand, if the number of warning objects is equal to or greater than N2 (step S7: YES), the controller 10 may set a center area A3 with a size corresponding to the driving environment in which the number of warning objects is relatively large, and calculates the proportion of the viewpoints included in the center area A3 (the center check proportion p) (step S9). The center area A3 is set as a circular area centered on the average direction of the distribution of the line-of-sight directions for the most recent predetermined time (for example, 30 seconds). The size of the center area A3 is set, for example, to 6 degrees based on the result of the driving test described above.

[0058] After the processing in steps S6, S8 or S9, the controller 10 determines whether the calculated center check percentage p is equal to or greater than a threshold value p th (step S10). The threshold value p this set in advance and stored in the memory 10b. The threshold value p th is set to 0.5, for example, based on the result of the driving test described above.

[0059] If the center check proportion p is equal to or greater than the threshold p th (step S10: YES), it is assumed that the driver's line of sight is concentrated in the central area, and the line of sight is less likely to move out of the central area. Thus, the controller 10 estimates that the driver is in the abnormal state (step S11).

[0060] If the center check proportion p is equal to or greater than the threshold p this (step S10: YES), the controller 10 may transmit the control signal to the display 36 and / or the speaker 37 and cause the display 36 and / or the speaker 37 to output an alarm to notify the driver that the driver is in the abnormal state (step S12). At this time, the display 36 and / or the speaker 37 may be caused to output the image information and the audio information (line-of-sight guidance information) for guiding the driver's line of sight to the warning object that the driver has not visually recognized.

[0061] Alternatively and / or additionally, if the center check proportion p is equal to or greater than the threshold p thThat is, when the controller 10 estimates that the driver is in the abnormal state (or the driver has the brain disorder), the controller 10 controls the vehicle (1) to stop. For example, the controller 10 controls the driving power source 2, the transmission 3, the brake 4, and / or the steering device 5 so that the vehicle 1 stops on the side of the road.

[0062] On the other hand, if the center check proportion p is not equal to or greater than the threshold p th is (step S10: YES), it is assumed that the driver's line of sight frequently moves not only within the center area but also outside the center area. Thus, the controller 10 can estimate that the driver's condition is normal (step S13).

[0063] After step S12 or S13, the controller 10 may terminate the driver state estimation processing. [Modified examples]

[0064] In the above-described embodiment, it was described that the central area is set as the circular area centered on the average direction of the distribution of the line-of-sight directions for the most recent predetermined time (for example, 30 seconds). However, the central area may be an area centered on the traveling direction of the vehicle or may have a shape other than a circle. [Functionality / Effects]

[0065] Next, an operation and effects of the driver state estimation device 100 in the present embodiment described above will be described.

[0066] The controller 10 estimates in the case where the proportion (center check proportion p) of the viewpoints included in the center region including the line of sight direction of the case where the driver directs his line of sight in the advancing direction of the vehicle 1 among all the viewpoints included in the distribution of the driver's viewpoints within the predetermined time is equal to or greater than the predetermined threshold p th is that the driver is in the abnormal state. Thus, based on the center check proportion p, it is possible to identify that the tendency of the driver's line of sight to concentrate near the traveling direction increases due to some disturbance, and thus it is possible to accurately estimate that the driver is in the abnormal state regardless of the driving environment.

[0067] In particular, since the controller 10 sets the size of the center area based on the driving environment information, by setting the center area to the appropriate size according to the driving environment information, it is possible to more accurately estimate whether the driver's condition is abnormal according to the driving environment.

[0068] Further, specifically, when the number of warning objects is equal to or greater than the predetermined threshold, the controller 10 sets the center area larger than that in the case where the number of warning objects is smaller than the predetermined threshold. Further, specifically, the controller 10 sets the center area larger as the number of warning objects increases. Accordingly, based on the characteristic that the tendency of the driver's line of sight in the abnormal state to concentrate near the advancing direction of the vehicle 1 increases with the relatively smaller number of warning objects, it is possible to set the center area to such a size that the difference from the driver in the normal state becomes noticeable, and thus it is possible to more accurately estimate whether the driver's state is abnormal according to the driving environment.

[0069] Further, in particular, since the controller 10 sets the circular area centered on the average direction of the distribution of the line-of-sight directions within the predetermined time as the center area, it is possible to set the center area serving as the reference for estimating the driver's state based on the line-of-sight direction of the actual driver, and thus it is possible to estimate more accurately whether the driver's state is abnormal. List of reference symbols 1 vehicle 10 controllers 100 Driver condition estimation device 21 outdoor camera 22 radars 23 Navigation system 24 Positioning system 25 Vehicle speed sensor 26 Accelerometer 27 Yaw rate sensor 28 Steering angle sensor 29 Steering torque sensor 30 Accelerator pedal sensor 31 Brake sensor 32 vehicle interior camera 36 ad 37 speakers QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] JP 2021-077136A

[0003]

Claims

[1] A driver state estimation device (100) that estimates a state of a driver driving a vehicle (1), the driver state estimation device (100) comprising: a line-of-sight detection device (32) configured to detect a line of sight of the driver; and a controller (10) configured to estimate a state of the driver based on the driver's line of sight, wherein the controller (10) is configured: detect a distribution of driver viewpoints within a predetermined time based on the driver's line of sight; and in a case where a proportion of the viewpoints included in a predetermined field of view (A) among all the viewpoints included in the detected distribution of viewpoints is equal to or greater than a predetermined proportion, to estimate that the driver is in an abnormal state, and wherein the predetermined viewing area (A) includes a line of sight direction in a case where the driver directs his line of sight in a direction of travel of the vehicle (1). [2] The driver condition estimation device (100) according to claim 1, wherein the controller (10) is configured to set the predetermined viewing range (A). [3] Driver condition estimation device (100) according to claim 1 or 2, further comprising: a driving environment information acquisition device (21 to 24) configured to acquire driving environment information of the vehicle (1), wherein the controller (10) is configured to set a size of the predetermined field of view (A) based on the driving environment information. [4] The driver condition estimation device (100) according to claim 3, wherein the controller (10) is configured: detect a number of objects or warning objects in a vicinity of the vehicle (1) based on the driving environment information; and setting the predetermined viewing area (A) larger than that in a case where the number of the object or the warning objects is less than a predetermined threshold when the number of the objects or the warning objects is equal to or greater than the predetermined threshold. [5] The driver state estimation device (100) according to any one of the preceding claims, wherein the controller (10) is configured to set, as the predetermined visual range (A), a circular area centered on an average direction of a distribution of the visual line directions within a predetermined time on a basis of the driver's visual line. [6] The driver state estimation device (100) according to any one of the preceding claims, wherein the direction of travel of the vehicle (1) is a forward direction of the vehicle (1). [7] The driver state estimation device (100) according to any one of the preceding claims, wherein the controller (100) is configured to determine points at which the line of sight stagnates or remains for a predetermined time as the viewpoints. [8] Driver state estimation device (100) according to one of the preceding claims, wherein the line-of-sight detection means (32) is a camera to be provided in the vehicle (1). [9] Vehicle (1) comprising the driver condition estimation device according to any one of the preceding claims. [10] A method for determining a state of a driver driving a vehicle (1), the method comprising: Detecting a driver’s line of sight; detecting a distribution of viewpoints of the driver within a predetermined time based on the driver's line of sight; and Determining whether a proportion of the viewpoints contained in the predetermined field of view (A) among all viewpoints contained in the detected distribution of viewpoints is equal to or greater than a predetermined proportion, wherein the predetermined viewing area (A) includes a line of sight direction in a case where the driver directs his line of sight in a direction of travel of the vehicle (1).

Images