Driver status determination device

Abstract

To determine the driver's status more accurately without increasing the cost of a vehicle.SOLUTION: A driver status determination device 1 includes a sunlight sensor 32 that detects light outside a vehicle, an in-vehicle camera 4 that captures a driver's face image, and an ECU 10 connected to the sunlight sensor 32 and the in-vehicle camera 4 via a communication path 5. The ECU 10 determines the driver's closed eyes based on the image capture results obtained by the in-vehicle camera 4, executes a first process to issue a warning to a cabin or to command the automatic stopping of a vehicle 100 based on the determination result, determines a first condition to be satisfied when an output value of the sunlight sensor 32 exceeds a first threshold value, and a second condition to be satisfied when the value obtained by subtracting the most recent moving average of the output value from the output value at the current time exceeds a second threshold value, respectively, and delays or inhibits the execution of the first process when both the first and second conditions are satisfied.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present disclosure relates to a driver state determination device. [Background technology]

[0002] Patent Document 1 discloses a drowsy driving warning device. This device includes an eye opening detection means, an open / closed eye judgment reference value correction means, an open / closed state judgment means, an alertness judgment means, and an alarm means. The eye opening detection means detects the eye opening degree based on a facial image. The open / closed eye judgment reference value correction means estimates the state in which the driver feels glare and corrects the reference value used for the open / closed eye judgment. The open / closed state judgment means judges the open / closed state of the eyes based on the eye opening degree and the corrected reference value. The alertness judgment means judges the alertness degree from a change in the open / closed state of the eyes based on the judgment result by the open / closed state judgment means. The alarm means issues an alarm based on the judgment result by the alertness judgment means.

[0003] According to the Patent Document 1, the open / closed eye judgment reference value correction means estimates the state in which the driver feels glare based on density information of the driver's eye position. The accuracy of the warning is improved by correcting the reference value based on the estimation result.

[0004] Patent Document 2 discloses a vehicle monitoring device. This vehicle monitoring device includes a driving state measurement unit, an alarm determination unit, and a sunlight direction acquisition unit. The driving state determination unit measures a number of driving states including the degree of eye opening of the driver based on an image of the driver. The alarm determination unit determines whether or not to activate an alarm device based on the degree of eye opening. The sunlight direction determination unit acquires the sunlight direction by referring to a sunlight direction database that stores date, time, location information, and sunlight direction in association with each other.

[0005] According to Patent Document 2, when the vehicle travel direction and the sunshine direction satisfy a predetermined relationship, the warning determination unit causes the driving state determination unit to execute the eye opening degree detection process again. By executing the detection process again according to the sunshine direction, it is possible to take into account cases where the driver's face is exposed to direct sunlight, and to suppress erroneous determinations of drowsy driving, inattentive driving, etc.

[0006] Patent Document 3 discloses that an illuminance sensor detects the illuminance on the face of an occupant. This illuminance sensor measures and outputs the environmental illuminance, and is also used in an automatic light switching device for a vehicle. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2000-198369 A [Patent Document 2] JP 2000-47086 A [Patent Document 3] Japanese Patent Application Publication No. 9-53917 Summary of the Invention [Problem to be solved by the invention]

[0008] The inventors of the present application have devised a mechanism that determines whether the driver is normal (for example, whether the driver is dozing off) by determining whether the driver is closed, and issues an alarm or automatically stops the vehicle if the driver is not normal.

[0009] However, when sunlight, such as the setting sun, shines in, the driver may unintentionally close his or her eyes, even if the driver is normal. In order to more appropriately issue a warning or automatically stop the vehicle, it is necessary to determine whether the driver is in such a situation.

[0010] Possible methods for determining unintentional eye closure are described in, for example, Patent Documents 1 and 2. However, these known methods require the installation of new hardware for precisely analyzing the eye position and the introduction of a program specialized in determining the direction of sunlight, which is inconvenient for reducing vehicle costs.

[0011] Therefore, it is conceivable to use an illuminance sensor for automatically turning on vehicle lights, as described in, for example, Patent Document 3. However, an illuminance sensor used for automatic lighting does not have as good a resolution on the bright side as it does on the dark side.

[0012] Therefore, according to the results of experiments conducted by the inventors of the present application, it has been newly discovered that when using an illuminance sensor such as that described in Patent Document 3, it is not possible to clearly distinguish between a cloudy sky where the driver does not feel glare and a situation where the driver closes his / her eyes due to the glare. This is inconvenient for accurately judging the driver's state.

[0013] The present disclosure has been made in consideration of the above points, and an object of the present disclosure is to more accurately determine the state of a driver without increasing the cost of the vehicle. [Means for solving the problem]

[0014] A first aspect of the present disclosure relates to a driver state determination device for determining the state of a driver operating a vehicle. The driver state determination device includes an on-board sensor that detects light outside the vehicle and outputs a signal having a value set to increase as the light becomes brighter, an on-board camera that captures a face image of the driver, and a controller connected to the on-board sensor and the on-board camera via a communication path, and the controller determines whether the driver has closed eyes based on the image capture result of the on-board camera, executes a first process for issuing a warning in the vehicle cabin or commanding automatic stopping of the vehicle based on the determination result of the closed eyes, determines a first condition that is satisfied when an output value of the on-board sensor exceeds a first threshold value or is equal to or greater than the first threshold value, and a second condition that is satisfied when a value obtained by subtracting a most recent moving average of the output value from the current output value exceeds a second threshold value or is equal to or greater than the second threshold value, and delays or suppresses execution of the first process when both the first condition and the second condition are satisfied.

[0015] According to the inventors of the present application, when the brightness entering the vehicle interior increases significantly in a short period of time, such as when the setting sun shines in immediately after exiting a tunnel, the eyes may not be able to adapt to light quickly enough, and it has been newly discovered that this may lead to the driver unintentionally closing their eyes.

[0016] In order to determine light adaptation, it is necessary to detect the change in the amount of light on the bright side in detail. However, a general vehicle-mounted sensor used for so-called automatic light control does not have a higher resolution on the bright side than on the dark side. Therefore, based on conventional knowledge alone, it is inconvenient to use such a vehicle-mounted sensor for determining light adaptation.

[0017] In response to these problems, the inventors of the present application have come up with a new idea of ​​using the most recent moving average of the output value of the vehicle-mounted sensor in addition to the output value itself in order to precisely determine the increase in the amount of light.

[0018] In reality, when the output value of an on-board sensor increases significantly in a short period of time, the most recent moving average increases more slowly than the output value. Although the moving average changes toward the same value as the output value, the timing at which the change is completed lags behind the output value.

[0019] That is, when the output value of the vehicle-mounted sensor increases in a short period of time, the value obtained by subtracting the moving average from that output value becomes a positive value and temporarily increases.

[0020] Therefore, the inventors of the present application considered that eye closure due to light adaptation may occur when the following two conditions (A) and (B) are both satisfied.

[0021] Output value > 1st threshold value …(A) Output value - moving average > second threshold value … (B) These conditions (A) and (B) are nothing but the first and second conditions, respectively. First, when the first condition is satisfied, it can be determined that the driver is in an environment where he or she may feel glare. Furthermore, when the second condition is satisfied, it can be determined that the brightness entering the vehicle has increased so much in a short period of time that the driver may feel glare.

[0022] Therefore, when both the first and second conditions are satisfied, it is possible to determine that the driver is in a situation where the driver's eyes cannot adapt to light and may close their eyes unintentionally. If the driver unintentionally closes their eyes due to glare rather than drowsiness or other factors, it is possible to determine that the driver is normal.

[0023] According to the first aspect, when both the first and second conditions are satisfied, that is, when the driver is determined to be normal, the controller delays or suppresses execution of the first process, thereby making it possible to delay or suppress execution of the first process caused by erroneous detection of the driver's state.

[0024] The first aspect can be realized by using a relatively common on-board sensor. Also, the first and second conditions can be determined without using a sophisticated sunshine program. Therefore, the driver's condition can be determined more accurately without increasing the cost of the vehicle.

[0025] In addition, according to a second aspect of the present disclosure, the on-board sensor may be an illuminance sensor that detects the illuminance of light entering from outside the vehicle, and the controller may automatically turn on the headlamps of the vehicle when the illuminance is less than a predetermined value.

[0026] According to the second aspect, the in-vehicle sensor is an illuminance sensor that can be used for automatically turning on headlamps, i.e., a so-called auto light sensor. By using the illuminance sensor as a sensor for determining the first and second conditions, it is advantageous in suppressing an increase in the cost of the vehicle.

[0027] In addition, according to a third aspect of the present disclosure, the on-board sensor may be an illuminance sensor that detects the illuminance of light entering from outside the vehicle, and the controller may determine whether it is raining or not based on the detection signal of the illuminance sensor, and if it is determined that it is raining, automatically operate the windshield wipers of the vehicle.

[0028] According to the third aspect, the on-board sensor is an illuminance sensor that can be used for automatic wiper operation, i.e., a rain sensor. By using the illuminance sensor as a sensor for determining the first and second conditions, it is advantageous in suppressing an increase in the cost of the vehicle.

[0029] Furthermore, according to a fourth aspect of the present disclosure, the controller may count the duration of the closed eyes, and execute the first process when the duration is equal to or longer than a predetermined period, and reset the count of the duration when both the first condition and the second condition are satisfied.

[0030] According to the fourth aspect, when both the first condition and the second condition are satisfied, the controller resets the count of the duration. By resetting the count of the duration, the first process can be prevented from being executed. This is advantageous in delaying or suppressing the execution of the first process caused by an erroneous detection of the driver's state.

[0031] According to a fifth aspect of the present disclosure, the first threshold value may be greater than the second threshold value.

[0032] According to the fifth aspect, by setting the first threshold value relatively large, it is possible to more reliably determine whether the driver is in an environment where the driver may feel dazzled, which contributes to more accurate determination of the driver's state. Effect of the Invention

[0033] As described above, according to the present disclosure, the driver's state can be determined more accurately without increasing the cost of the vehicle. [Brief description of the drawings]

[0034] [Figure 1] FIG. 1 is a schematic diagram illustrating a vehicle equipped with a driver condition determination device. [Diagram 2] FIG. 2 is a block diagram illustrating the configuration of the driver condition determination device. [Diagram 3] FIG. 3 is a perspective view illustrating the layout of a sunlight sensor. [Figure 4] FIG. 4 is a graph illustrating the resolution of a sunlight sensor. [Figure 5A] FIG. 5A is a diagram for explaining a determination of the first exception condition. [Figure 5B] FIG. 5B is a diagram for explaining the determination of the second exception condition. [Figure 5C] FIG. 5C is a diagram corresponding to FIG. 5B and illustrating a case where the second exceptional condition is not satisfied. [Figure 6]FIG. 6 is a flowchart illustrating a process performed by the driver condition determination device. [Figure 7] FIG. 7 is a flowchart illustrating the processing relating to the first and second exception conditions. [Figure 8] FIG. 8 is a diagram illustrating an example of the output of the sunlight sensor and the actual measured value of the moving average. [Figure 9A] FIG. 9A illustrates an example of the output value of the sunlight sensor and the actual measured value of the moving average. [Figure 9B] FIG. 9B illustrates an example of the output value of the sunlight sensor and the actual measured value of the moving average. [Figure 9C] FIG. 9C illustrates an example of the output value of the sunlight sensor and the actual measured value of the moving average. [Figure 9D] FIG. 9D illustrates an example of the output value of the sunlight sensor and the actual measured value of the moving average. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0035] Hereinafter, embodiments of the present disclosure will be described in detail. Note that the following description is merely an example.

[0036] Fig. 1 is a schematic diagram illustrating a vehicle 100 equipped with a driver condition determination device 1 according to this embodiment. Fig. 2 is a block diagram illustrating the configuration of the driver condition determination device 1. Fig. 3 is a perspective view illustrating the layout of the sunlight sensor 32, and Fig. 4 is a graph illustrating the resolution of the sunlight sensor 32.

[0037] In the following description, the front, rear, left, top, and bottom as viewed from the driver of the vehicle 100 are simply referred to as the front, rear, left, top, and bottom, respectively. These definitions of the front-rear direction, the left-right direction, and the up-down direction are merely for the sake of convenience in order to simplify the description.

[0038] <Overall configuration of vehicle 100> The vehicle 100 shown in Fig. 1 is a four-wheeled automobile. The vehicle 100 is equipped with a power source 100a such as an engine or a motor. The vehicle 100 runs when the power source 100a drives the wheels. The vehicle 100 may be a gasoline vehicle, a hybrid vehicle, an electric vehicle, or a fuel cell vehicle. There is no particular limitation on the type of power source 100a.

[0039] A driver state determination device 1 is mounted on the vehicle 100. This driver state determination device 1 is a device for determining the state of the driver operating the vehicle 100. As shown in Fig. 2, the driver state determination device 1 according to this embodiment includes vehicle equipment 2, an on-vehicle sensor 3, an on-vehicle camera 4, a communication path 5, and an ECU (Electrical Control Unit) 10. The driver state determination device 1 can also operate the vehicle equipment 2 and an actuator 6 based on the determined state of the driver.

[0040] The vehicle devices 2 are composed of headlamps 21, windshield wipers 22, emergency flashers 23, a display 24, and a horn 25. These vehicle devices 2 are connected to the ECU 10 via a communication path 5. The communication path 5 transmits and receives signals between each vehicle device 2 and the ECU 10. For example, a controller area network (CAN) may be adopted as the communication protocol in the communication path 5.

[0041] The on-board sensors 3 are composed of a steering wheel sensor 31 and a sunlight sensor 32. These on-board sensors 3 are connected to the ECU 10 via a communication path 5. The communication path 5 allows signals to be transmitted and received between each on-board sensor 3 and the ECU 10.

[0042] The steering wheel sensor 31 is built into the steering wheel of the vehicle 100. The steering wheel sensor 31 is, for example, a capacitance type touch sensor. The steering wheel sensor 31 detects whether or not the steering wheel is being gripped by the driver. A detection signal of the steering wheel sensor 31 is input to the ECU 10 via the communication path 5.

[0043] The sunlight sensor 32 detects light outside the vehicle 100. The sunlight sensor 32 outputs a signal having a value that is set to increase as the detected light becomes brighter. In other words, the output value of the sunlight sensor 32 increases as the light outside the vehicle becomes brighter. The detection signal of the sunlight sensor 32 is input to the ECU 10 via the communication path 5.

[0044] Specifically, the sunlight sensor 32 according to this embodiment is an illuminance sensor that detects the illuminance of light entering from outside the vehicle. The sunlight sensor 32 as an illuminance sensor is disposed inside the vehicle behind the windshield 100b as shown in FIG. 3. The sunlight sensor 32 detects the illuminance of light entering the vehicle from the front or side of the vehicle 100. A detection signal from the sunlight sensor 32 is input to the ECU 10 at predetermined intervals. The sunlight sensor 32 according to this embodiment inputs its detection signal to the ECU 10 at intervals of, for example, five times per minute.

[0045] More specifically, the detection signal of the sunlight sensor 32 can vary as a 10-bit value, that is, a value from 0 to 1024. The value of this detection signal has, for example, a positive correlation with the brightness of the detected light. If an illuminance sensor is used as the sunlight sensor 32, the value of the detection signal increases as the detected illuminance increases. Hereinafter, the value of the detection signal of the sunlight sensor 32 will be simply referred to as the "signal value of the sunlight sensor 32" or the "output value of the sunlight sensor 32."

[0046] More specifically, the resolution of the sunlight sensor 32 differs between the bright side (high illuminance side) and the dark side (low illuminance side). Note that the resolution here refers to the amount of change in illuminance relative to the amount of change in the signal value of the sunlight sensor 32 (for example, the amount of change in illuminance per 1 LSB). The resolution can also be called the light receiving sensitivity.

[0047] The sunlight sensor 32 according to this embodiment has a smaller resolution on the dark side than on the bright side. That is, the sunlight sensor 32 can capture relatively fine changes in illuminance when driving at night, on cloudy days, in rainy weather, etc. On the other hand, the sunlight sensor 32 captures relatively coarse changes in illuminance when driving during the day on sunny days, etc.

[0048] Fig. 4 illustrates the resolution of the sunlight sensor 32. The horizontal axis of Fig. 4 indicates the output value [LSB] of the sunlight sensor 32, and the vertical axis of the same figure indicates the illuminance [Lx] corresponding to the output value.

[0049] As shown in FIG. 4, the sunlight sensor 32 according to this embodiment has three types of resolution [Lx / LSB]. The resolutions are set to gradually increase in order from the smallest output value. In other words, the resolution on the side with the smallest output value, i.e., the dark side, is set to be relatively smaller than the resolution on the side with the largeest output value, i.e., the bright side. Specifically, the three types of resolutions increase in the order of the first region R1, the second region R2, and the third region R3. The first region R1 includes the lower limit of the output value. The third region R3 includes the upper limit of the output value. The second region R2 is located between the first region R1 and the third region R3.

[0050] By making the resolution on the dark side relatively small, the sunlight sensor 32 becomes suitable for a so-called auto light function, which is a control for automatically turning on the headlamps 21. The sunlight sensor 32 can also be called an auto light sensor.

[0051] The sunlight sensor 32 may also function as a rain sensor that measures reflected light from raindrops on the windshield 100b. When used as a rain sensor, the sunlight sensor 32 may also be used for a function that automatically operates the wipers 22, a so-called auto-wiper function. The auto-wiper function will be described later.

[0052] It should be noted that the sunlight sensor 32 is not limited to an illuminance sensor. The sunlight sensor 32 may be a luminance sensor that detects the luminance of light. The sunlight sensor 32 may be a sensor that detects a physical quantity related to the amount of light, such as the luminous intensity or luminous flux of light, or may be a sensor that detects the amount of light itself. For example, when a luminance sensor is used as the sunlight sensor 32, the output value increases as the detected luminance increases. Similarly, when a light intensity sensor is used as the sunlight sensor 32, the output value increases as the detected luminance increases.

[0053] The vehicle-mounted camera 4 captures a facial image of the driver. The vehicle-mounted camera 4 is connected to the ECU 10 via a communication path 5. An electrical signal corresponding to the facial image captured by the vehicle-mounted camera 4 is input to the ECU 10 via the communication path 5.

[0054] More specifically, the vehicle-mounted camera 4 is disposed inside the vehicle. For example, the vehicle-mounted camera 4 is built into a display 24 located near the dashboard of the vehicle 100. The vehicle-mounted camera 4 can also be called a driver monitoring camera.

[0055] More specifically, the vehicle-mounted camera 4 has a camera sensor with enhanced sensitivity in the near-infrared region and an LED that emits near-infrared light to illuminate the driver. The vehicle-mounted camera 4 captures the situation inside the vehicle, including the driver, using the camera sensor. The situation inside the vehicle captured by the vehicle-mounted camera 4 includes, for example, the driver's driving posture in addition to the face image of the driver.

[0056] The ECU 10 is a so-called electronic control unit. The ECU 10 constitutes a controller in this embodiment. The ECU 10 as a controller is connected to the vehicle device 2, the on-board sensor 3, and the on-board camera 4 via a communication path 5.

[0057] Specifically, the ECU 10 according to this embodiment is a microcomputer having a processor, a memory, and an input / output bus. Further, the ECU 10 may be constituted by a plurality of electronic control units. In that case, each of the plurality of electronic control units will have a processor, a memory, an input / output bus, and the like.

[0058] The ECU 10 is also connected to the actuator 6 of the vehicle 100 via the communication path 5. The actuator 6 of the vehicle 100 may include, for example, one or more combinations of the power steering system of the vehicle 100, the parking brake, the power source 100a, and the auxiliary machine of the power source 100a.

[0059] <Details of ECU 10> As shown in FIG. 2, the ECU 10 includes an auto light unit 11, an auto wiper unit 12, an image analysis unit 13, an abnormality determination unit 14, a first processing unit 15, and a second processing unit 16.

[0060] (Auto Light Unit 11) The auto light unit 11 executes auto light control based on the detection signal of the sunlight sensor 32. Auto light control refers to control that automatically turns on the headlamp 21 and other lamps according to the ambient brightness. Other lamps refer to at least one of the side marker lamp, the tail lamp, and the license plate lamp. The function realized by this auto light control corresponds to the aforementioned auto light function.

[0061] The auto light unit 11 according to this embodiment reads the detection signal of the sunlight sensor 32 and automatically turns on the headlamp 21 and other lights of the vehicle 100 when the illuminance corresponding to the detection signal is less than a predetermined value. Further, the auto light unit 11 reads the detection signal of the sunlight sensor 32 and turns off the headlamp 21 and other lights of the vehicle 100 when the illuminance corresponding to the detection signal is greater than or equal to a predetermined value.

[0062] The automatic light control can be appropriately switched on and off (whether or not the automatic light control is performed) based on the driver's operation. Also, the automatic light unit 11 is not essential in the first place.

[0063] (Auto wiper section 12) The auto-wiper unit 12 executes auto-wiper control based on a detection signal from the sunlight sensor 32. The auto-wiper control refers to control for automatically operating the wipers 22 when it is raining. The function realized by this auto-wiper control corresponds to the auto-wiper function described above.

[0064] The auto-wiper unit 12 according to this embodiment determines whether it is raining or not based on the detection signal of the sunlight sensor 32. Specifically, the auto-wiper unit 12 reads the detection signal of the sunlight sensor 32, and determines whether it is raining (i.e. whether it is raining or not) and the amount of rain based on the detection signal. If it is determined that it is raining, the auto-wiper unit 12 automatically operates the wipers 22 of the vehicle 100. The operating speed of the wipers 22 is set to a speed according to the determined amount of rain, for example.

[0065] The auto-wiper control can be switched on and off (whether or not the auto-wiper control is performed) appropriately based on the driver's operation. Also, the auto-wiper unit 12 is not essential in the first place.

[0066] (Image Analysis Unit 13) The image analysis unit 13 judges whether the driver's eyes are closed based on the imaging result of the in-vehicle camera 4. Specifically, the image analysis unit 13 judges whether the driver's eyes are open or closed based on the facial image captured by the in-vehicle camera 4 and the change in the facial image over time. The eye open or closed state judged by the image analysis unit 13 includes both information indicating whether the driver has his / her eyes closed (whether the eyes are closed or open) and information indicating the time the driver's eyes are closed (the duration of the eyes being closed). The eye open or closed state may include the time the driver's eyes are open (the duration of the eyes being open).

[0067] For example, the image analysis unit 13 detects the opening degree of the driver's left and right eyelids and the driver's facial direction based on the face image. Based on the detection result, the image analysis unit 13 determines whether the driver's left and right eyes are in an open eye state, a closed eye state, or an unspecified open eye state, for each of the driver's left and right eyes. When both the driver's left and right eyes are in an eye-closed state, the image analysis unit 13 determines that the driver has his or her eyes closed. When at least one of the driver's left and right eyes is in an eye-open state, the image analysis unit 13 determines that the driver has his or her eyes open. When the image analysis unit 13 determines that the driver has his or her eyes closed, it calculates the time during which the determination continues and sets the calculated time as the driver's eye-closed time.

[0068] The image analysis unit 13 also detects the driver's gaze and eyelid movements based on the face image captured by the in-vehicle camera 4 and the time change of the face image. The image analysis unit 13 detects the driver's visual behavior and blink activity based on these movements. Information that can be detected as the visual behavior and blink activity includes, for example, the speed of gaze movement, the frequency of gaze movement, and the blink cycle. In addition, the image analysis unit 13 may detect the driver's mouth movement, head shaking, and posture change.

[0069] The result of the determination by the image analysis unit 13 is used in a first process by a first processing unit 15 via an abnormality determination unit 14.

[0070] (Abnormality determination unit 14) The abnormality determination unit 14 determines whether or not the driver is in a state where he or she can drive normally based on the determination result by the image analysis unit 13. Specifically, the abnormality determination unit 14 determines whether or not the driver can drive normally based on the open / closed state of the driver's eyes and the state of gripping the steering wheel. Note that the abnormality determination unit 14 may make a determination using the driving posture of the driver in addition to the open / closed state of the driver's eyes and the state of gripping the steering wheel.

[0071] In this embodiment, a state in which the driver can drive normally refers to a state in which the driver's drowsiness is not increased (for example, the driver is not dozing). Similarly, a state in which the driver cannot drive normally refers to a state in which the driver's drowsiness is increased (for example, the driver is dozing). In other words, in this embodiment, the determination of whether the driver is in a state in which the driver can drive normally is equivalent to the determination of whether the driver's drowsiness is increased.

[0072] Here, the grip state of the steering wheel is a flag indicating whether or not the steering wheel is being gripped. This flag is managed by the ECU 10 based on a detection signal from the handle sensor 31. Note that it is not essential for the abnormality determination unit 14 to use the grip state of the steering wheel for the determination.

[0073] Specifically, the abnormality determination unit 14 determines that the driver is not in a state where he or she can drive normally (the driver's drowsiness is increasing) when any one of the first, second, and third augmentation conditions described below is satisfied. These augmentation conditions can be determined to be satisfied, for example, by counting the duration of closed eyes and when the duration is equal to or longer than a predetermined period.

[0074] Specifically, the first drowsiness condition is satisfied when the driver's eyes are closed for a first period or more. The first period is, in detail, 3 to 7 seconds, more specifically, 4 to 6 seconds. The first drowsiness condition is, for example, a condition that does not depend on the state of gripping the steering wheel. Even if the driver is gripping the steering wheel, if the driver's eyes are closed for the first period or more, the abnormality determination unit 14 determines that the driver's drowsiness is increasing.

[0075] The second enhancement condition is satisfied when the driver's eyes remain open for a second period or more and the driver is not gripping the steering wheel. The length of the second period is at least shorter than the first period. The second period is more specifically a period of 2 seconds or more and 4 seconds or less.

[0076] In the third enhancement condition, first, the presence or absence of drowsiness is judged from a viewpoint different from the state of eye closure. If the presence of drowsiness is detected by the judgment, the time of eye closure within a third period from the detection is judged. Then, if the time of eye closure within the third period continues for the second period or more, the third enhancement condition is satisfied. The third period is at least the second period or more. The third period is specifically a period of 100 seconds or more and 140 seconds or less, and more specifically a period of 110 seconds or more and 130 seconds or less.

[0077] Here, determining whether or not the driver is drowsy from a perspective other than the state in which the eyes are closed refers to a determination based on, for example, the speed of eye movement, the frequency of eye movement, the frequency of blinking, and changes in the facial image other than the eyes.

[0078] For example, if the driver's eye movement is fast and frequent and the blinking cycle is stable, the abnormality determination unit 14 determines that the driver is not sleepy at all (sleepiness level = 1). If the driver's eye movement is slow and the lips are open, the abnormality determination unit 14 determines that the driver is somewhat sleepy (sleepiness level = 2). If the driver's blinking is slow and frequent and the driver's mouth is moving, the abnormality determination unit 14 determines that the driver is sleepy (sleepiness level = 3). If the driver blinks conscious, the eye movement and blinking are slow, the driver shakes his head, or the driver yawns frequently, the abnormality determination unit 14 determines that the driver is very sleepy (sleepiness level = 4). If the driver closes his eyelids or tilts his head back and forth, the abnormality determination unit 14 determines that the driver is very sleepy (sleepiness level = 55).

[0079] In determining whether the third aggravation condition is met, the abnormality determination unit 14 according to the present embodiment determines that "the driver is drowsy" when the drowsiness level is between 3 and 5. Based on the determination result and the eye-closure time within the third period, the driver's increased drowsiness is determined as described above.

[0080] (First processing unit 15) The first processing unit 15 executes a first process based on the result of the eye closure determination by the image analysis unit 13 and the abnormality determination unit 14. The first process refers to a process of issuing a warning in the vehicle cabin or issuing a command to automatically stop the vehicle 100.

[0081] Specifically, the first processing unit 15 according to this embodiment executes the first process when it is determined that the driver is in a state in which he or she cannot drive normally. Specifically, when the first process is started, the first processing unit 15 first starts flashing the hazard warning lights 23 of the vehicle 100 and issues a warning to the driver or passengers by displaying on the display 24. The contents of the display 24 may be configured, for example, by starting an automatic stop of the vehicle 100 and a cancellation procedure of the first process configured by a specific user operation.

[0082] Thereafter, if the driver does not return to a state in which he or she can drive normally and the first process is not canceled, the first processing unit 15 starts flashing the brake lights and sounding the horn 25 in addition to turning on the hazard flasher 23. At the same time, the first processing unit 15 controls the power source 100a and the like to decelerate and stop the vehicle 100.

[0083] (Second processing section 16) FIG. 5A is a diagram for explaining the determination of a first exception condition. FIG. 5B is a diagram for explaining the determination of a second exception condition. The horizontal axis of FIG. 5A and FIG. 5B is time [sec]. The vertical axis of FIG. 5A is the output value V1 of the sunlight sensor 32 or the most recent moving average A1 of the output value V1. The dashed line of FIG. 5A indicates the first threshold value T1. The vertical axis of FIG. 5B indicates a value D1 (=V1-A1) obtained by subtracting the most recent moving average A1 of the output value V1 from the output value V1 of the sunlight sensor 32. The dashed line of FIG. 5B indicates the second threshold value T2.

[0084] The second processing unit 16 determines whether the first exception condition and the second exception condition exist, and when both the first and second exception conditions are satisfied, commands the execution of a second process. The second process is a process for delaying or suppressing the execution of the first process by the first processing unit 15.

[0085] Here, the first exceptional condition refers to a condition that is satisfied when the output value of the sunlight sensor 32 as an on-vehicle sensor exceeds the first threshold value T1 or is equal to or greater than the first threshold value T1. In this embodiment, the former case (when the output value exceeds the first threshold value T1) is adopted as an example. The second exceptional condition refers to a condition that is satisfied when the value obtained by subtracting the most recent moving average of the output value from the current output value exceeds the second threshold value T2 or is equal to the second threshold value T2. In this embodiment, the former case (when the subtracted value exceeds the second threshold value T2) is adopted as an example. The first and second exceptional conditions are examples of the "first condition" and "second condition" in this embodiment, respectively.

[0086] Note that the "current time" referred to here may be the timing at which the first exception condition is determined, for example, the timing at which the first exception condition is satisfied, or may be a timing slightly delayed from the timing at which the first exception condition is satisfied.

[0087] First, as conceptually illustrated in Fig. 5A, the second processing unit 16 determines whether the current output value V1 of the sunlight sensor 32 exceeds a first threshold value T1 (whether the first exceptional condition is satisfied). The first threshold value T1 is a value belonging to the second region R2 or the third region R3 in Fig. 4. Specifically, the first threshold value T1 corresponds to an illuminance of 5000 Lx or more and 9000 Lx or less, more specifically, 6000 Lx or more and 8000 Lx or less.

[0088] 5B, the second processing unit 16 calculates a moving average of the output value of the sunlight sensor 32 in parallel with the determination of the first exception condition. The moving average calculated by the second processing unit 16 is a direct moving average A1. In other words, this moving average A1 is a simple moving average from the output value at the current time to the output value going back a predetermined period of time.

[0089] For example, assuming that the detection signal of the sunlight sensor 32 is input to the ECU 10 at 5 cycles per minute, if the predetermined period = 4 minutes, in this embodiment, the moving average based on the most recent 20 data will be calculated.

[0090] Next, the second processing unit 16 subtracts the value of the calculated moving average A1 from the output value V1 of the sunlight sensor 32 at the current time. Then, the second processing unit 16 determines whether the value D1 obtained by this subtraction exceeds the second threshold value T2 (whether the second exception condition is satisfied). As can be seen from the comparison between FIGS. 5A and 5B, the second threshold value T2 is a threshold value less than the first threshold value T1 (T2 < T1). In other words, the first threshold value T1 is greater than the second threshold value T2.

[0091] That is, assuming that n is a positive integer and t n is the determination time of the first and second exception conditions, the second processing unit 16 determines whether the following conditional expressions (1) and (2) hold simultaneously at time t n .

[0092] s(t n ) > T1 …(1) s(t n ) - a(t n ) > T2 …(2) In the above conditional expressions (1) and (2), s(t n ) is the output value V1 of the sunlight sensor 32 at time t n , and a(t n ) is the most recent moving average A1 of the output value of the sunlight sensor 32 at time t n . Assuming that m is a positive integer, a(t n ) can be expressed as in the following formula (3). In the case of the above-described configuration example, m = 20 - 1 = 19.

[0093] a(t n ) = s(t n ) + s(t n-1 ) + … + s(t n-m ) / (m - 1) …(3) Note that in formula (3), s(t n) does not have to start from s(t n-1 ) or s(t n-2 ) etc., s(t n ) may start calculating the moving average from the value obtained by going back a predetermined time from

[0094] The left side of the above formula (2) corresponds to the subtraction value D1 described above. For example, in FIG. 5A, after time t1, it can be considered that the first exception condition is satisfied. On the other hand, in the example of FIG. 5B, it can be considered that the second exception condition is satisfied only within the period from time t2 to time t3.

[0095] Here, assuming that the relationship t2 < t1 < t3 holds in FIGS. 5A and 5B, within the range from time t1 to time t3, it can be considered that both the first and second exception conditions are satisfied.

[0096] Thus, when both the first and second exception conditions are satisfied, the second processing unit 16 resets the count of the closed-eye time (the duration of closing the eyes). In this case, since the determination by the abnormality determination unit 14, particularly the determination that "the driver is not in a state where he can drive normally" is postponed, thereby, the execution of the first processing by the first processing unit 15 is delayed or suppressed.

[0097] On the other hand, FIG. 5C is a diagram corresponding to FIG. 5B for explaining a case where the second exception condition is not satisfied. The horizontal axis of FIG. 5C is time [sec]. The vertical axis of FIG. 5C shows a value D2 (= V2 - A2) obtained by subtracting the most recent moving average A2 (≠ A1) of the output value V2 of the sunlight sensor 32 from the output value V2 (≠ V1). The broken line in FIG. 5C shows the second threshold value T2 in the same manner as in FIG. 5B.

[0098] For example, as shown in Fig. 5C, when the subtraction value D2 does not exceed the second threshold value T2 and the second exceptional condition is not satisfied, the second processing unit 16 does not reset the count of the eye-closed time (the duration of eye-closure). In this case, the determination by the abnormality determination unit 14, particularly the determination that "the driver is not in a state to drive normally", is not delayed, so that the execution of the first process by the first processing unit 15 is started without delay. The same applies to the case where the first exceptional condition is not satisfied.

[0099] <Specific examples of controlling increased sleepiness> A specific example of control related to increased drowsiness will be described below with reference to Fig. 6 and Fig. 7. Fig. 6 is a flowchart illustrating the processing performed by the driver condition determination device 1. Fig. 7 is a flowchart illustrating the processing related to the first and second exceptional conditions.

[0100] First, in step S1 of Fig. 6, the ECU 10 resets the count of the timer 10a. Next, in step S2, the ECU 10 reads various signals. The signals read by the ECU 10 include detection signals from the handle sensor 31 and the sunlight sensor 32, an image signal acquired by the in-vehicle camera 4, and a count time by the timer 10a. Information output from the image analysis unit 13 may be used for the count time by the timer 10a.

[0101] In the following step S3, the image analysis unit 13 analyzes the image captured by the vehicle-mounted camera 4. In the following steps S4 to S8, the abnormality determination unit 14 determines whether or not any one of the first enhancement condition, the second enhancement condition, and the third enhancement condition is satisfied.

[0102] Specifically, in step S4, the abnormality determination unit 14 determines whether or not the driver has his / her eyes closed based on the analysis result by the image analysis unit 13. If the determination in step S4 is NO, the ECU 10 skips the subsequent processing and returns. In this case, the first processing is not performed.

[0103] On the other hand, if the determination in step S5 is YES, the ECU 10 advances the control process to step S5. In this step S5, the abnormality determination unit 14 determines whether the driver's eye-closure time period has reached the second period. This determination is performed using the count time of the timer 10a.

[0104] If the determination in step S5 is YES, the ECU 10 advances the control process to step S6. In step S6, the ECU 10 issues a warning to the driver through display control of the display 24 and sounding of the horn 25.

[0105] In step S7 following step S6, the abnormality determination unit 14 determines whether the driver is gripping the steering wheel (handle gripped?). This determination can be made using the detection signal of the handle sensor 31.

[0106] If the determination in step S7 is NO, the second acceleration condition is satisfied. In this case, the ECU 10 advances the control process to step S11, and commands the first processing unit 15 to execute the first processing. In this step S11, the first processing is executed.

[0107] On the other hand, if the determination in step S7 is YES, the ECU 10 advances the control process to step S8. In this step S8, the abnormality determination unit 14 determines whether the driver's eyes are closed for a first period or not. This determination is performed using the count time of the timer 10a.

[0108] If the determination in step S8 is YES, the first acceleration condition is satisfied. In this case, the ECU 10 advances the control process to step S11, and commands the first processing unit 15 to execute the first processing. In step S11, the first processing is executed.

[0109] On the other hand, if the determination in step S5 or step S8 is NO, the ECU 10 advances the control process to step S9. In step S9, the ECU 10 counts up the timer 10a, thereby adding to the count time.

[0110] Then, in step S10 following step S9, the ECU 10 executes the second process shown in Fig. 7. In this second process, first, step S21 in Fig. 7 is executed.

[0111] In step S21, the ECU 10 reads various signals. The signals read by the ECU 10 include the detection signal of the sunlight sensor 32 and the count time of the timer 10a.

[0112] In the next step S22, the second processing unit 16 calculates a moving average, particularly the most recent moving average, of the output values ​​of the sunlight sensor 32. Note that step S22 may be performed in parallel with the determination in step S23 described below, or may be performed between step S23 and step S24 also described below.

[0113] In step S23, the second processing unit 16 judges whether or not the current output value of the sunlight sensor 32 exceeds the first threshold value T1 illustrated in Fig. 5A. If the judgment is NO, the ECU 10 ends the control process illustrated in Fig. 7 and returns to step S2 in Fig. 6. In this case, the count time by the timer 10a is continuously held. Step S23 is an example of judging whether or not the first exceptional condition is satisfied.

[0114] On the other hand, if the determination in step S23 is YES, the ECU 10 advances the control process to step S24. Note that the process of step S24 may be performed in parallel with the determination in step S23, or may be performed before the determination in step S23.

[0115] In step S24, the second processing unit 16 judges whether or not the value obtained by subtracting the most recent moving average from the current output value of the sunlight sensor 32 exceeds the second threshold value T2 illustrated in Fig. 5B and Fig. 5C. If the judgment is NO, the ECU 10 ends the control process illustrated in Fig. 7 and returns to step S2 in Fig. 6. In this case, the count time by the timer 10a is continuously held. Step S24 is an example of judging whether or not the second exceptional condition is satisfied.

[0116] On the other hand, if the determination in step S24 is YES, the ECU 10 advances the control process to step S25. In step S25, the second processing unit 16 resets the count time of the timer 10a (returns the count time to zero). After that, the ECU 10 returns the control process to step S2 in Fig. 6. In this case, the count time of the timer 10a is newly counted from zero.

[0117] In addition, when processing steps S4 to S8, the abnormality determination unit 14 may determine whether or not the third enhancement condition is established. Specifically, the abnormality determination unit 14 determines the presence or absence of drowsiness prior to step S4. Details of "drowsiness" here are as described above. When it is determined that drowsiness is present (for example, when the drowsiness level is 3-5), the abnormality determination unit 14 counts the elapsed time from the determination timing as a starting point. The abnormality determination unit 14 determines whether or not the driver's eye-closure time exceeds the second period before the elapsed time reaches the above-mentioned third period. Then, when the eye-closure time exceeds the second period, the abnormality determination unit 14 advances the control process to step S11, whereas when the eye-closure time does not exceed the second period, the abnormality determination unit 14 advances the control process to step S9 and step S11 and counts up the eye-closure time.

[0118] <Situations in which drivers experience glare> Fig. 8 is a diagram illustrating an example of the output of the sunlight sensor and the actual measured value of its moving average. The horizontal axis of Fig. 8 is time [sec]. The vertical axis of Fig. 8 is the output value L1 of the sunlight sensor 32, or the most recent moving average L2 of the output value L1. The lower limit of the output value L1 indicates a value of 0. The upper limit of the output value indicates a value of 1024.

[0119] The present inventors have studied a situation in which a driver may unintentionally close his / her eyes even if the driver is normal, for example, when the setting sun shines in immediately after exiting a tunnel. In this study, the present inventors have focused on the light adaptation of the driver's eyes.

[0120] That is, according to the inventors of the present application, when the brightness and amount of light of the sunlight increases significantly in a short period of time, the light adaptation of the eyes may not be able to keep up. In such a case, it has been newly discovered that the driver may unintentionally close his / her eyes.

[0121] In order to determine light adaptation, it is necessary to detect the change in the amount of light on the bright side in detail. However, as described with reference to FIG. 4, the sunlight sensor 32 used in so-called automatic light control does not have a higher resolution on the bright side than on the dark side. In fact, as shown by the output value (sensor output value) L1 in FIG. 8, it has been known that in a relatively bright environment, the output value L1 of the sunlight sensor 32 changes slightly around its upper limit.

[0122] Therefore, the sunlight sensor 32 cannot precisely reflect the increase or decrease in the amount of light on the bright side in its output value. Therefore, based on conventional knowledge alone, the sunlight sensor 32 is inconvenient to use for determining unintentional eye closure.

[0123] To address these issues, the inventors of the present application came up with a new idea of ​​using the most recent moving average of the output value of the sunlight sensor 32 in addition to the output value itself in order to determine the increase in the amount of light in detail. The change in this moving average over time can be regarded as a sunlight adaptation curve, which indicates the progress of the driver's habituation to glare.

[0124] For example, as shown in the boxed area C1 in Fig. 9, when the output value L1 of the sunlight sensor 32 increases from 0 to V1 in a short period of time, the most recent moving average L2 increases more slowly than the output value L1. Although the moving average L2 changes from 0 to V1 in the same way as the output value L1, the timing at which the change is completed is delayed compared to the output value L1.

[0125] That is, when the output value L1 of the sunlight sensor 32 increases in a short period of time, the value obtained by subtracting the moving average L2 from the output value L1 becomes a positive value and temporarily increases.

[0126] According to the inventors of the present application, in a situation where the moving average L2 does not catch up with the output value L1 (a situation where L1>L2), it can be determined that the driver is not accustomed to the glare. On the other hand, in a situation where the moving average L2 catches up with the output value L1 (a situation where L1≈L2), it can be determined that the driver has become accustomed to the glare. In the former situation, the value obtained by subtracting the moving average L2 from the output value L1 temporarily increases (see the double-headed arrow Ar1 in FIG. 8). After that, when the latter situation is reached, the magnitude of the subtracted value starts to decrease (see the double-headed arrow Ar2 in FIG. 8). By using the value obtained by subtracting the moving average L2 from the output value L1, it is possible to determine whether the driver has become accustomed to the glare.

[0127] The second processing unit 16 according to this embodiment has a configuration that reflects this knowledge.

[0128] That is, the inventors of the present application considered that eye closure due to light adaptation may occur when the following two conditions (A) and (B) are both satisfied.

[0129] Output value L1>first threshold T1 …(A) Output value L1-moving average L2>second threshold T2...(B) These conditions (A) and (B) are nothing but the first and second exceptional conditions shown in the above formulas (1) and (2), respectively. If the first exceptional condition is satisfied, it can be determined that the driver is in an environment where he or she may feel glare. Furthermore, if the second exceptional condition is satisfied, it can be determined that the brightness entering the vehicle has increased so much in a short period of time that the driver may feel glare.

[0130] Therefore, when both the first and second exceptional conditions are satisfied, it is possible to determine that the driver is in a situation where the eyes cannot adapt to light and may close their eyes unintentionally. If the driver unintentionally closes their eyes due to glare, rather than due to drowsiness or other factors, it is possible to determine that the driver is normal.

[0131] 7, when both the first and second exception conditions are satisfied, that is, when the driver is determined to be normal, the second processing unit 16 delays or suppresses the execution of the first process by the first processing unit 15. This makes it possible to delay or suppress the execution of the first process caused by an erroneous detection of the driver's state.

[0132] The configuration according to this embodiment can be realized using a relatively common sunlight sensor 32, as shown in Fig. 2. The first and second exceptional conditions can be determined without using an advanced sunlight program. Therefore, the driver's condition can be determined more accurately without increasing the cost of the vehicle 100.

[0133] 2, the sunlight sensor 32 according to this embodiment is an illuminance sensor, a so-called auto light sensor, that can be used to automatically turn on the headlamps 21. Having the auto light sensor also function as a sensor for determining the first and second exceptional conditions is advantageous in suppressing an increase in the cost of the vehicle 100.

[0134] 2, the sunlight sensor 32 according to this embodiment is an illuminance sensor, a so-called rain sensor, that can be used for the automatic operation of the wipers 22. Having the rain sensor also function as a sensor for determining the first and second exceptional conditions is advantageous in suppressing an increase in the cost of the vehicle 100.

[0135] As illustrated in step S25 of Fig. 7, the second processing unit 16 resets the count of the eye-closed time (the duration of the eyes being closed) when both the first and second exceptional conditions are satisfied. By resetting the count of the eye-closed time, it is possible to prevent the first process from being executed. This is advantageous in delaying or suppressing the execution of the first process caused by an erroneous detection of the driver's state.

[0136] 5A and 5B, by setting the first threshold T1 to be greater than the second threshold T2, it becomes possible to more reliably determine whether the driver is in an environment where he or she may feel glare, which contributes to more accurate determination of the driver's condition.

[0137] <Example using actual sensor measurements> Figures 9A, 9B, 9C, and 9D show examples of output values ​​and moving averages of the sunlight sensor 32. The values ​​shown in Figures 9A, 9B, 9C, and 9D are obtained by using the same sunlight sensor 32, but are different in terms of the measurement location, measurement time, weather at the time of measurement, etc.

[0138] Here, the horizontal axis of each diagram is time [sec]. The section from the lower limit to the upper limit on the horizontal axis indicates a time interval of about 5 to 15 minutes. The vertical axis of each graph indicates the output value or moving average of the sunlight sensor 32. In each graph, the thick line indicates the output value, and the thin line indicates the moving average. For the output value, the section from the lower limit to the upper limit on the vertical axis indicates a transition of values ​​from 0 to 1024.

[0139] 9A illustrates, for example, daytime measurements. In the case of FIG. 9A, the output value of the sunlight sensor 32 fluctuates near its upper limit except for times Ta1, Ta2, and Ta3. This reflects the fact that the resolution on the bright side is relatively large, as described above.

[0140] 9A, it can be seen that the output value of the sunlight sensor 32 sharply decreases to a value significantly larger than the lower limit value, and then sharply increases after a certain time. This behavior suggests that the vehicle 100 has passed under a structure that does not sufficiently block sunlight but blocks the sunlight for a relatively long period of time, such as under a bridge that extends along the traveling direction of the vehicle 100.

[0141] In such a situation, it is considered that the first exceptional condition (output value>first threshold T1) is satisfied, but the second exceptional condition (moving average>second threshold T2) is not satisfied. In this case, it is considered that the light adaptation of the eyes has caught up because the sunlight is not sufficiently blocked. The second processing unit 16 does not delay or suppress the execution of the first process. Depending on the time the driver's eyes are closed, it is determined that the driver is not normal, and the first process is started.

[0142] 9A, the output value of the sunlight sensor 32 drops sharply to near its lower limit, and then rises sharply to near its upper limit. This behavior suggests that the vehicle 100 has passed through a structure that completely blocks sunlight, such as a tunnel.

[0143] In such a situation, it is considered that not only the first exceptional condition but also the second exceptional condition is satisfied. In this case, it is assumed that the eyes have not yet adapted to light and the eyes are closed due to glare. The second processing unit 16 delays or suppresses the execution of the first process. Regardless of the time the driver's eyes are closed, the determination that the driver is not normal is suspended. As a result, the execution of the first process is prevented.

[0144] 9B illustrates the measured value in the evening or when it is raining. In the case of FIG. 9B, the output value of the sunlight sensor 32 is significantly lower than its upper limit. This suggests that although it is evening, the afternoon sun is not shining into the car, or that it is raining.

[0145] 9B, the output value of the sunlight sensor 32 drops sharply from a value significantly lower than the upper limit to the lower limit, and then rises sharply to a value significantly lower than the upper limit. This behavior suggests that the vehicle 100 has passed through a structure that completely blocks sunlight, such as a tunnel.

[0146] In such a situation, it is considered that the second exceptional condition is satisfied, but the first exceptional condition is not satisfied. In this case, it is considered that the driver does not feel glare in the first place. The second processing unit 16 does not delay or suppress the execution of the first processing. Depending on the driver's eye closing time, it is determined that the driver is not normal, and the first processing is started.

[0147] 9C illustrates an example of daytime measurements. In the case of FIG. 9C, the output value of the sunlight sensor 32 fluctuates around its upper limit value over the entire time range. This behavior suggests that the vehicle 100 has passed a structure that does not completely block sunlight, such as under a bridge perpendicular to the traveling direction of the vehicle 100, but blocks sunlight for a relatively short period of time.

[0148] In such a situation, it is considered that the first exceptional condition (output value>first threshold T1) is satisfied, but the second exceptional condition (moving average>second threshold T2) is not satisfied. In this case, it is considered that the light adaptation of the eyes has caught up because the sunlight is not sufficiently blocked. The second processing unit 16 does not delay or suppress the execution of the first process. It is determined that the driver is not normal according to the time the driver's eyes are closed, and the first process is started.

[0149] FIG. 9D illustrates, for example, daytime measurements. In the case of FIG. 9D, the output value of the sunlight sensor 32 increases and decreases sharply, fluctuating between its upper and lower limits. As shown in FIG. 9D, even in a situation where the output value increases and decreases sharply, the moving average changes slowly. By using the moving average, it is possible to stabilize the determination of whether the driver is feeling dazzled. [Explanation of symbols]

[0150] 1 Driver status determination device 2 Vehicle equipment 21 Headlamp 3. Vehicle sensors 32 Sunlight sensor (vehicle sensor) 4. Car Camera 5. Communication Paths 10 ECU (controller) 100 vehicles T1 First threshold T2 Second Threshold

Claims

1. A driver state determination device that determines a state of a driver who operates a vehicle, comprising: an in-vehicle sensor that detects light outside the vehicle and outputs a signal having a value that is set to increase as the light becomes brighter; An in-vehicle camera that captures a facial image of the driver; A controller connected to the on-board sensor and the on-board camera via a communication path, The controller: determining whether the driver has his / her eyes closed based on an image captured by the vehicle-mounted camera; execute a first process of issuing a warning in the vehicle cabin or commanding an automatic stop of the vehicle based on the result of the eye closure determination; determining a first condition that is satisfied when an output value of the on-board sensor exceeds a first threshold value or is equal to or greater than the first threshold value, and a second condition that is satisfied when a value obtained by subtracting a most recent moving average of the output value from the current output value exceeds a second threshold value or is equal to or greater than the second threshold value; When both the first condition and the second condition are satisfied, execution of the first process is delayed or suppressed. A driver condition determination device comprising:

2. 2. The driver condition determination device according to claim 1, the on-board sensor is an illuminance sensor that detects the illuminance of light incident from outside the vehicle; The controller automatically turns on the headlamps of the vehicle when the illuminance is less than a predetermined value. A driver condition determination device comprising:

3. 2. The driver condition determination device according to claim 1, the on-board sensor is an illuminance sensor that detects the illuminance of light incident from outside the vehicle; The controller: determining whether it is raining or not based on a detection signal from the illuminance sensor; When it is determined that it is raining, the wipers of the vehicle are automatically operated. A driver condition determination device comprising:

4. 2. The driver condition determination device according to claim 1, The controller: Counting the duration of the closed eyes, and executing the first process when the duration is equal to or longer than a predetermined period; resetting the count of the duration when both the first condition and the second condition are satisfied A driver condition determination device comprising:

5. 2. The driver condition determination device according to claim 1, The first threshold is greater than the second threshold. A driver condition determination device comprising:

Images