Drowsiness detector

Abstract

To provide a drowsiness detector capable of detecting drowsiness at high accuracy.SOLUTION: A drowsiness detector 1 comprises: an acquisition part 11 for acquiring an eye-opening amount being numerical information corresponding to an amount of raising the eyelids of a driver, on the basis of a face image of the driver of a vehicle; an estimation part 12 for estimating whether or not the driver becomes in a narrow eyes state due to glare, on the basis of the eye-opening amount and the deceleration of the eye-opening amount; and a detection part 13 for, when it is estimated that the driver becomes in the narrow eyes state due to glare, not detecting drowsiness based on the eye-opening amount, and when it is not estimated that the driver becomes in the narrow eyes state due to glare, detecting drowsiness based on the eye-opening amount.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a drowsiness detection device.

Background Art

[0002] Patent Document 1 discloses a technique of weighting the degree of eye opening and closing calculated based on a driver's face image according to the weight corresponding to the luminance information of a predetermined area in the face image, and determining the eye opening and closing state based on the weighted degree of eye opening and closing. According to the patent document, in the case where direct sunlight is incident through the front glass of a vehicle, it is possible to distinguish and detect the driver's squinting state due to the glare of the direct sunlight from the squinting state due to drowsiness.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Non-Patent Documents

[0004]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, the cases where the driver is in a squinted state due to glare are not limited to the cases where direct sunlight is incident. For example, there are cases where the driver is in a squinted state due to glare caused by the light of the oncoming vehicle's headlights or sunlight reflected by the rear glass of the vehicle ahead.

[0006] In some cases, a near-infrared camera may be applied as a camera for imaging the driver's face image. Since the near-infrared camera is insensitive to the light of the headlights, there are cases where it is impossible to distinguish and detect the squinted state due to glare from the squinted state due to drowsiness in the determination relying on the luminance information of the face image. Therefore, when this technology is used for drowsiness detection, there is a possibility of false detection of drowsiness.

[0007] An object of the present invention is to provide a drowsiness detection device capable of detecting drowsiness with high accuracy.

Means for Solving the Problems

[0008] The drowsiness detection device of the present invention includes an acquisition unit that acquires an eye-opening amount, which is numerical information corresponding to the amount by which the driver raises the eyelids, based on the face image of the driver of the vehicle, an estimation unit that estimates whether or not the driver has squinted due to glare based on the eye-opening amount and the decrease rate of the eye-opening amount, and a detection unit that does not perform drowsiness detection based on the eye-opening amount when it is estimated that the driver has squinted due to glare, and performs drowsiness detection based on the eye-opening amount when it is estimated that the driver has not squinted due to glare.

Effects of the Invention

[0009] The drowsiness detection device according to the present invention estimates whether or not the driver has squinted due to glare based on the eye-opening amount and the decrease rate of the eye-opening amount, and does not perform drowsiness detection based on the eye-opening amount when it is estimated that the driver has squinted due to glare. Therefore, according to the drowsiness detection device according to the present invention, there is an effect that drowsiness can be detected with high accuracy.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

[0011] Hereinafter, a drowsiness detection device according to an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited by this embodiment. In addition, the components in the following embodiments include those that can be easily assumed by those skilled in the art or those that are substantially the same.

[0012] [Embodiment] An embodiment will be described with reference to FIGS. 1 to 10. This embodiment relates to a drowsiness detection device. FIG. 1 is a diagram of a vehicle equipped with a drowsiness detection device according to an embodiment.

[0013] As shown in FIG. 1, the drowsiness detection device 1 of the present embodiment is mounted on a vehicle 100. The vehicle 100 is an automobile and is driven by a driver 200. The vehicle 100 has a camera 20 that images the face of the driver 200. The camera 20 is, for example, a near-infrared camera. Note that the camera 20 is not limited to a near-infrared camera. The camera 20 is disposed in front of the driver 200 and is arranged so as to be able to image the eyes 210 of the driver 200. The camera 20 is configured to continuously image the driver 200 at a predetermined frame rate.

[0014] The drowsiness detection device 1 is a device that detects the drowsiness of the driver 200.

[0015] FIG. 2 is a block diagram showing the functional configuration of the drowsiness detection device 1 according to the embodiment. The drowsiness detection device 1 includes an acquisition unit 11, an estimation unit 12, a detection unit 13, and a warning unit 14. The drowsiness detection device 1 is, for example, a computer having an arithmetic circuit, a memory, an input / output interface, and the like. The drowsiness detection device 1 is configured as, for example, an electronic control unit (ECU).

[0016] The acquisition unit 11 acquires an eye opening amount EO, which is numerical information corresponding to the amount by which the driver 200 raises the eyelids, based on the face image of the driver 200 imaged by the camera 20. The acquisition unit 11 detects the eyes of the driver 200 in each of a plurality of face images obtained by imaging at a predetermined frame rate by image recognition. The acquisition unit 11 calculates the eye opening degree EO based on the eye images detected for each frame. Thereby, the temporal transition of the eye opening degree EO is obtained. Hereinafter, the transition means the temporal transition.

[0017] For example, the eye opening degree can be used as the eye opening amount EO. The eye opening degree is numerical information indicated as a percentage with respect to the eye opening distance when the eyes 210 are open in a state where the driver 200 is awake. Note that the eye opening distance is the distance from the upper eyelid to the lower eyelid. The normal eye opening distance when the driver 200 is awake and has the eyes open is denoted as the maximum eye opening distance Dmax. The eye opening degree is set to 0% when the eye opening distance is 0, and is set to 100% when the eye opening distance is equal to the maximum eye opening distance Dmax.

[0018] When the eye opening degree is used as the eye opening amount EO, the acquisition unit 11 specifies the maximum eye opening distance Dmax based on the eye opening distance calculated from each face image. The acquisition unit 11 sets, as the eye opening degree at each timing, the percentage of the eye opening distance calculated from the face image captured at each timing with respect to the maximum eye opening distance Dmax.

[0019] Note that the method for acquiring the maximum eye opening distance Dmax is not limited to a specific method. For example, the acquisition unit 11 may specify, as the maximum eye opening distance Dmax, the eye opening distance calculated from the face image captured at a predetermined timing immediately after the start of the vehicle 100. This is because it is considered that the driver 200 is fully awake immediately after the start of the vehicle 100. Alternatively, the acquisition unit 11 may use, as the maximum eye opening distance Dmax, the maximum value or the average value of the eye opening distances calculated from each face image captured during a predetermined period immediately after the start of the vehicle 100.

[0020] Also, the eye opening distance itself may be used as the eye opening amount EO. When the eye opening distance is used as the eye opening amount EO, the acquisition unit 11 acquires the eye opening distance from each face image and separately acquires the maximum eye opening distance Dmax.

[0021] The estimation unit 12 estimates whether the driver 200 has squinted due to glare based on the eye opening amount EO and the decreasing speed of the eye opening amount EO. The squinted state is a state where the driver 200 squints the eyes 210. The reasons for the driver 200 to be in the squinted state include that the driver 200 feels glare and that the driver 200 feels sleepy. When the driver 200 is in the squinted state, the estimation unit 12 estimates whether the cause of the squinted state is due to glare. The method for estimating whether the driver 200 has squinted due to glare will be described later.

[0022] When it is estimated that the driver 200 has not squinted due to glare, the detection unit 13 detects drowsiness. When it is estimated that the driver 200 has squinted due to glare, the detection unit 13 does not detect drowsiness.

[0023] The detection unit 13 executes drowsiness detection based on the eye opening amount EO. The algorithm for drowsiness detection based on the eye opening amount EO is not limited to a specific algorithm. In one example, the detection unit 13 performs drowsiness detection according to an algorithm called Perclos (Percent of the time eyelids are closed) analysis.

[0024] The detection unit 13 may execute blink analysis during drowsiness detection. In blink analysis, the detection unit 13 can analyze the state of the driver 200's eyes, such as whether it is dry eye, by determining whether the waveform of the eye opening distance during the blink period matches a predetermined pattern.

[0025] The warning unit 14 controls the warning device 40. The warning device 40 is a device that outputs attention or warnings to the driver 200. The warning device 40 is, for example, a warning lamp, a speaker, an image display device, etc. provided in the vehicle 100.

[0026] The drowsiness detection device 1 acquires various information from the vehicle ECU 30. The information to be acquired is, for example, the traveling speed of the vehicle 100, the shift position, the operating state of the brake hold function, the presence or absence of steering hold, and the like.

[0027] When drowsiness is detected while the vehicle 100 is traveling, the warning unit 14 controls the warning device 40 to output a warning about the driver being drowsy. The warning device 40 alerts the driver 200, for example, by voice, warning lamp, image display, etc.

[0028] Subsequently, a method for estimating whether the driver 200 has become in a narrow-eyed state due to glare will be described. In the description of the estimation method, the opening distance will be used as the opening amount EO.

[0029] FIG. 3 is a diagram showing an example of the transition of the opening distance in two cases. Case A is a case where the driver 200 feels drowsy, and case B is a case where the driver feels glare. The transition of the opening distance in each case is plotted on a graph with the elapsed time [min] on the horizontal axis and the opening distance [mm] on the vertical axis.

[0030] In the graph for each case, the opening distance and the opening distance after moving average obtained from the transition of the opening distance are plotted. Also, as an example of the threshold Th0 used for detecting the narrow-eyed state, a dotted line indicating a value obtained by multiplying the maximum opening distance Dmax by 0.5, that is, the opening distance corresponding to an opening degree of 50%, is drawn.

[0031] As is clear from each graph, the noise included in the transition of the opening distance is suppressed by moving averaging. Therefore, the estimation unit 12 uses the opening distance after moving averaging for estimating whether the eyes of the driver 200 have become in a narrow-eyed state due to glare. Hereinafter, unless otherwise specified, the opening distance shall refer to the opening distance after moving averaging.

[0032] In case A, the eye opening distance is smaller than the threshold value Th0 during the period from timing t1 to timing t2. In case B, the eye opening distance is smaller than the threshold value Th0 during the period from timing t3 to timing t4.

[0033] As is clear from the change in the eye opening distance at timings t1 and t3, the rate of decrease in the eye opening distance is faster when the eye opening distance falls below the threshold value Th0 in case B than when it does so in case A. That is, when the driver 200 feels dazzle and enters the narrow-eye state, the eye opening distance decreases faster than when the driver 200 feels drowsiness and enters the narrow-eye state. Note that in this specification, the rate of decrease in the eye opening distance is defined as a value that takes a positive value when the eye opening distance is decreasing.

[0034] Such a tendency that the rate of decrease in the eye opening distance is faster when entering the narrow-eye state due to dazzle than when entering the narrow-eye state due to drowsiness has been confirmed in many people.

[0035] Therefore, in the embodiment, the estimation unit 12 estimates whether the driver 200 has entered the narrow-eye state due to dazzle based on the eye opening distance and the rate of decrease in the eye opening distance. Specifically, when the eye opening distance is smaller than the first threshold value Th1 and the rate of decrease in the eye opening distance is faster than the second threshold value Th2, the estimation unit 12 estimates that the driver 200 has entered the narrow-eye state due to dazzle. When the eye opening distance is larger than the first threshold value Th1 or the rate of decrease in the eye opening distance is slower than the second threshold value Th2, the estimation unit 12 estimates that the driver 200 has not entered the narrow-eye state due to dazzle.

[0036] The first threshold value Th1 is a threshold value for detecting the narrow-eye state. In one example, the estimation unit 12 multiplies the maximum eye opening distance Dmax by a first reference value representing a value of a predetermined eye opening degree in decimal notation, and sets the value obtained by the multiplication as the first threshold value Th1. The above-described threshold value Th0 corresponding to an eye opening degree of 50% may be used as the first threshold value Th1.

[0037] As the second threshold Th2, a value capable of detecting that glare has been felt is set. In one example, the estimation unit 12 multiplies the maximum eye opening distance Dmax by a second reference value, and sets the value obtained by the multiplication as the second threshold Th2.

[0038] The second reference value is set by the designer. An example of setting the second reference value will be described below.

[0039] For example, assume that the designer obtained -4.83 [mm / min] as the change rate of the eye opening distance when the subject was in a squinted state due to glare through an experiment on the subject. When expressed as a decrease rate, -4.83 [mm / min] as the change rate of the eye opening distance becomes 4.83 [mm / min]. It is considered that individual differences occur in the maximum eye opening distance Dmax due to differences among people in features such as eye size, eye color, eyelash length, or eyelash density. The decrease rate of the eye opening distance is affected by this individual difference in the maximum eye opening distance Dmax. Therefore, the designer generalizes the decrease rate of the eye opening distance using the maximum eye opening distance Dmax of the subject. Specifically, the designer assumes that the decrease rate of the eye opening distance can be expressed as the product of the maximum eye opening distance Dmax and a coefficient K, and obtains the coefficient K. For example, in the case of the above-mentioned subject, the designer obtains 0.69 as the coefficient K from 4.83 [mm / min] as the decrease rate of the eye opening distance. The designer also obtains the coefficient K for other subjects. Assuming that the coefficient K obtained from experiments with a plurality of subjects is concentrated around approximately 0.7, the designer subtracts 0.05 as a margin considering individual differences from 0.7, and sets 0.65, which is the value obtained by the subtraction, as the second reference value. The estimation unit 12 acquires the second threshold Th2 by multiplying the maximum eye opening distance Dmax [mm] of the driver 200 by the coefficient K of 0.65 set as the second reference value.

[0040] Note that Non-Patent Document 1 discloses that the measured values of the maximum eye opening distance Dmax of the Chinese group and the Indian group are distributed in the range of 3.3 to 14.3 [mm]. According to this distribution of the measured values and the above setting example, it is reasonable to set the second threshold Th2 in the range of 2.1 to 9.3 [mm / min]. Note that the second threshold Th2 does not necessarily have to be a value within this range.

[0041] Thus, the first threshold Th1 and the second threshold Th2 are set according to the maximum eye opening distance Dmax of the driver 200. Thereby, it is possible to prevent the estimation accuracy from deteriorating due to individual differences in the maximum eye opening distance Dmax.

[0042] Note that the method for determining the first threshold Th1 and the second threshold Th2 is not limited to this. One or both of the first threshold Th1 and the second threshold Th2 may be set in advance in the drowsiness detection device 1 as fixed values that do not depend on the maximum eye opening distance Dmax or the like.

[0043] With reference to FIGS. 4 to 8, a specific example of the estimation by the estimation unit 12 will be described.

[0044] FIG. 4 is a diagram showing the transition of the eye opening distance of the driver 200 acquired by the acquisition unit 11. In this figure, the horizontal axis represents the elapsed time [min], and the vertical axis represents the eye opening distance [mm]. Among the transitions of the eye opening distance shown in FIG. 4, examples of estimation in Parts 1 to 4 will be described. Note that in the description of this specific example, the first threshold Th1 is 5 [mm], and the second threshold Th2 is 4.5 [mm / min].

[0045] FIG. 5 is a diagram for explaining an example of estimation in Part 1. As shown in this figure, the eye opening distance is greater than the first threshold Th1. The rate of decrease of the eye opening distance calculated from the transition of the eye opening distance, here the value of a / b shown in this figure, was 6.759 [mm / min]. Although the rate of decrease of the eye opening distance is faster than the second threshold Th2, since the eye opening distance is greater than the first threshold Th1, the estimation unit 12 estimates that it is not in a squinted state due to glare.

[0046] FIG. 6 is a diagram for explaining an example of estimation in Part 2. As shown in this figure, there is a portion where the eye opening distance is smaller than the first threshold Th1. The rate of decrease of the eye opening distance calculated from the transition of the eye opening distance, here the value of a / b shown in this figure, was 3.977 [mm / min]. Although the eye opening distance is smaller than the first threshold Th1, since the rate of decrease of the eye opening distance is slower than the second threshold Th2, the estimation unit 12 estimates that it is not in a squinted state due to glare.

[0047] FIG. 7 is a diagram for explaining an example of estimation in Part 3. As shown in this figure, there is a portion where the eye opening distance is smaller than the first threshold Th1. The rate of decrease of the eye opening distance calculated from the transition of the eye opening distance, here the value of a / b shown in this figure, was 6.137 [mm / min]. Since the eye opening distance is smaller than the first threshold Th1 and the rate of decrease of the eye opening distance is faster than the second threshold Th2, the estimation unit 12 estimates that it is in a squinted state due to glare.

[0048] FIG. 8 is a diagram for explaining an example of estimation in Part 4. As shown in this figure, there is a portion where the eye opening distance is smaller than the first threshold Th1. The rate of decrease of the eye opening distance calculated from the transition of the eye opening distance, here the value of a / b shown in this figure, was 5.087 [mm / min]. Since the eye opening distance is smaller than the first threshold Th1 and the rate of decrease of the eye opening distance is faster than the second threshold Th2, the estimation unit 12 estimates that it is in a squinted state due to glare.

[0049] In this way, the estimation unit 12 estimates whether the driver 200 has become squinted due to glare based on the eye opening distance and the decreasing speed of the eye opening distance.

[0050] When the driver 200 blinks, the eye opening distance instantaneously and temporarily approaches 0. If such a large change in the eye opening distance due to blinking is included in the transition of the eye opening distance, the estimation accuracy may deteriorate at the timing of blinking or near that timing. Therefore, the estimation unit 12 may perform blink correction before moving averaging. Blink correction is an operation for removing the instantaneous and temporary change in the eye opening distance due to blinking from the transition of the eye opening distance.

[0051] In blink correction, the estimation unit 12 detects the period of blinking from the transition of the eye opening distance. The estimation unit 12 detects the period during which the eye opening distance instantaneously and temporarily approaches 0 as the period of blinking. The estimation unit 12 changes the value of the eye opening distance during the period of blinking based on the value of the eye opening distance before the period of blinking.

[0052] For example, every time a new eye opening distance is acquired, the estimation unit 12 determines that blinking has occurred when the newly acquired eye opening distance has dropped by a predetermined percentage (for example, 20%) or more compared to the eye opening distance acquired a predetermined number of frames (for example, 2 frames) before.

[0053] The estimation unit 12 sets the value of the eye opening distance during the period of blinking thus detected to the same value as the average value of the eye opening distance immediately before the period of blinking. The estimation unit 12 may set the value of the eye opening distance during the period of blinking to the same value as the eye opening distance immediately before the period of blinking.

[0054] FIG. 9 is a diagram for explaining an example of blink correction according to the embodiment. In this figure, the horizontal axis represents the elapsed time, and the vertical axis represents the eye opening distance. The acquisition period of the eye opening distance is displayed in the horizontal axis direction. Each black circle represents the eye opening distance.

[0055] As can be seen from FIG. 9, it can be read that the eye opening distance approaches 0 instantaneously and temporarily during period R1, period R2, and period R3. The estimation unit 12 detects periods R1, R2, and R3 as blink periods. The estimation unit 12 changes all the eye opening distance values in period R1 to be equal to the eye opening distance value of point P1 acquired immediately before period R1. The estimation unit 12 changes all the eye opening distance values in period R2 to be equal to the eye opening distance value of point P2 acquired immediately before period R2. The estimation unit 12 changes all the eye opening distance values in period R3 to be equal to the eye opening distance value of point P3 acquired immediately before period R3. Each dotted circle indicates the updated eye opening distance value.

[0056] In this way, the estimation unit 12 removes the instantaneous and temporary large change in the eye opening distance due to blinking from the transition of the eye opening distance before moving averaging by blink correction. The estimation unit 12 performs blink correction on the transition of the eye opening distance and then performs moving averaging. Then, the estimation unit 12 executes an estimation using the transition of the eye opening distance after moving averaging, the first threshold Th1, and the second threshold Th2. Since the instantaneous and temporary large change in the eye opening distance due to blinking is removed from the transition of the eye opening distance before moving averaging, the deterioration of the estimation accuracy at the timing of blinking or in the vicinity of that timing is suppressed.

[0057] Note that as long as the instantaneous and temporary large change in the eye opening distance due to blinking can be removed from the transition of the eye opening distance before moving averaging, the specific processing in blink correction is not limited to the example described above.

[0058] In the description of the method for estimating whether the driver 200 has entered a squinted state due to glare as described above, the eye opening distance is used as the eye opening amount EO. The eye opening distance can be easily converted into the eye opening degree using the maximum eye opening distance Dmax. Therefore, the eye opening degree may be used as the eye opening amount EO.

[0059] When the eye opening degree is used as the eye opening amount EO, for example, a value of a predetermined eye opening degree corresponding to a first reference value can be used as a first threshold value Th1. Also, for example, a second reference value can be used as a second threshold value Th2. Note that even when the eye opening degree is used as the eye opening amount EO, one or both of the first threshold value Th1 and the second threshold value Th2 may be set in advance in the drowsiness detection device 1 as fixed values that do not depend on, for example, the maximum eye opening distance Dmax.

[0060] When the eye opening degree is used as the eye opening amount EO, if the eye opening degree is smaller than the first threshold value Th1 and the decreasing speed of the eye opening degree is faster than the second threshold value Th2, the estimation unit 12 estimates that the driver 200 has become squinted due to glare. If the eye opening degree is larger than the first threshold value Th1 or the decreasing speed of the eye opening degree is slower than the second threshold value Th2, it is estimated that the driver 200 has not become squinted due to glare. Also, the estimation unit 12 may perform blink correction on the transition of the eye opening amount EO before moving averaging.

[0061] Subsequently, the operation of the drowsiness detection device 1 will be described. FIG. 10 is a flowchart showing an example of the operation of the drowsiness detection device 1 according to the embodiment. Note that a series of operations shown in this flowchart are executed regardless of whether the vehicle 100 is running, for example.

[0062] In step S1, the eye opening amount EO is detected. The acquisition unit 11 acquires the eye opening amount EO of the driver 200 based on the face image acquired from the camera 20. When the eye opening amount EO of the driver 200 is detected, the process proceeds to step S2.

[0063] In step S2, moving averaging calculation is executed. The estimation unit 12 performs moving averaging on the transition of the eye opening amount EO. The estimation unit 12 may perform blink correction before moving averaging. When step S2 is executed, the process proceeds to step S3.

[0064] In step S3, it is determined whether or not the determination criterion that the eye opening amount EO is equal to or less than the first threshold Th1 and the decreasing rate of the eye opening amount EO is equal to or greater than the second threshold Th2 is satisfied. The estimation unit 12 determines whether or not the determination criterion is satisfied based on the transition of the eye opening amount EO after moving averaging. If a negative determination is made in step S3, the process proceeds to step S4, and if an affirmative determination is made, the process proceeds to step S10.

[0065] In step S4, blink analysis is executed. The detection unit 13 executes blink analysis based on the eye opening amount EO. When step S4 is executed, the process proceeds to step S5.

[0066] In step S5, drowsiness is calculated by Perclos analysis. The detection unit 13 performs Perclos analysis on the eye opening amount EO to obtain the degree of drowsiness of the driver 200. The degree of drowsiness calculated by Perclos analysis is a value corresponding to the so-called "PERCLOS", and typically, the larger the value, the higher the drowsiness, and the smaller the value, the lower the drowsiness. Here, "PERCLOS" is a parameter representing the ratio of the closed-eye time per unit time (PERCLOS = closed-eye time / unit time). After step S5 is executed, the process proceeds to step S6.

[0067] In step S6, it is determined whether the vehicle 100 is in motion. The determination in step S6 is made based on, for example, the traveling speed of the vehicle 100. If an affirmative determination is made in step S6, the process proceeds to step S7, and if a negative determination is made, the process proceeds to step S1.

[0068] In step S7, it is determined whether the sleepiness is equal to or greater than a third threshold value Th3. The third threshold value Th3 is a threshold value for detecting sleepiness. When the degree of sleepiness of the driver 200 is equal to or greater than the third threshold value Th3, the detection unit 13 determines that the driver 200 is feeling sleepy. When the degree of sleepiness of the driver 200 is less than the third threshold value Th3, the detection unit 13 determines that the driver 200 is not feeling sleepy. If an affirmative determination is made in step S7, the process proceeds to step S8; if a negative determination is made, the process proceeds to step S1.

[0069] In step S8, the warning device 40 alerts the driver 200 to wake up from sleep, for example, by means of sound, a warning lamp, image display, or the like. When step S8 is executed, the process proceeds to step S9.

[0070] In step S9, it is determined whether there is a warning cancellation signal. If an affirmative determination is made in step S9, the process proceeds to step S1; if a negative determination is made, the process returns to step S9 and the warning output continues.

[0071] In step S10, it is determined that the driver 200 is feeling dazzled. The estimation unit 12 estimates that the driver 200 is in a squinted state due to dazzle. When step S10 is executed, the process proceeds to step S1.

[0072] In the example shown in FIG. 10, when the determination criteria that the eye opening amount EO is less than or equal to a first threshold value Th1 and the decrease rate of the eye opening amount EO is greater than or equal to a second threshold value Th2 are satisfied, the estimation unit 12 estimates that the driver 200 is in a squinted state due to dazzle. When the determination criteria are not satisfied, the estimation unit 12 estimates that the driver 200 is not in a squinted state due to dazzle. The handling when the eye opening amount EO is equal to the first threshold value Th1 and the handling when the decrease rate of the eye opening amount is equal to the second threshold value Th2 are not limited to the example shown in FIG. 10.

[0073] For example, when the eye opening amount EO is equal to the first threshold Th1, the estimation unit 12 may execute the same processing as when the eye opening amount EO is greater than the first threshold Th1. When the decrease rate of the eye opening amount EO is equal to the second threshold Th2, the estimation unit 12 may execute the same processing as when the decrease rate of the eye opening amount EO is slower than the second threshold Th2.

[0074] As described above, according to the embodiment, the drowsiness detection device 1 includes an acquisition unit 11, an estimation unit 12, and a detection unit 13. The acquisition unit 11 acquires an eye opening amount EO, which is numerical information corresponding to the amount by which the driver 200 of the vehicle 100 raises their eyelids, based on a face image of the driver 200. The estimation unit 12 estimates whether or not the driver 200 has squinted due to glare based on the eye opening amount EO and the decrease rate of the eye opening amount EO. When it is estimated that the driver 200 has squinted due to glare, the detection unit 13 does not perform drowsiness detection based on the eye opening amount EO. When it is estimated that the driver 200 has not squinted due to glare, the detection unit 13 performs drowsiness detection based on the eye opening amount EO.

[0075] With this configuration, even when a camera that is insensitive to the light of the headlight is applied as the camera 20, when the driver feels glare from the light of the headlight and squints, it is possible to distinguish and detect the squinting state due to glare from the squinting state due to drowsiness. The same applies to sunlight, and it is possible to distinguish and detect the squinting state due to glare from the squinting state due to drowsiness. Therefore, regardless of the type of light that causes the driver 200 to feel glare, misdetection of the squinting state due to glare as the squinting state due to drowsiness is suppressed. That is, the drowsiness detection device 1 can detect drowsiness with high accuracy.

[0076] Note that as a technology to be compared with the embodiment, it is conceivable to detect the squinting state due to glare from the driver's expression. In order to apply such a technology, it is necessary to provide a camera having a high-pixel imaging device for imaging a face image, which increases the cost of the system.

[0077] In contrast, according to the embodiment, as long as the eye opening amount EO can be obtained, the squinted state due to glare can be detected, so the specifications required for the camera 20 can be low. That is, according to the embodiment, it is possible to improve the drowsiness detection accuracy while suppressing an increase in the cost of the system.

[0078] Also, as another technique to be compared with the embodiment, in order to detect the squinted state due to glare, in addition to the camera for capturing a face image, various sensors such as an illuminance sensor and an outward camera may be provided. However, when such a technique is applied, the cost of the system increases due to the addition of various sensors.

[0079] In contrast, according to the embodiment, as long as the eye opening amount EO can be obtained from the face image by the camera 20, the squinted state due to glare can be detected, so no additional sensors other than the camera 20 are required. That is, according to the embodiment, it is possible to improve the drowsiness detection accuracy while suppressing an increase in the cost of the system.

[0080] According to the embodiment, when the eye opening amount EO is smaller than the first threshold Th1 and the decreasing speed of the eye opening amount EO is faster than the second threshold Th2, the estimation unit 12 estimates that the driver 200 is in a squinted state due to glare. When the eye opening amount EO is larger than the first threshold Th1 or the decreasing speed of the eye opening amount EO is slower than the second threshold Th2, it is estimated that the driver 200 is not in a squinted state due to glare.

[0081] Therefore, it is possible to estimate whether or not the driver 200 is in a squinted state due to glare based on the eye opening amount EO and the decreasing speed of the eye opening amount EO.

[0082] Note that the estimation unit 12 may perform blink correction on the transition of the eye opening amount EO and estimate whether or not the driver 200 is in a squinted state due to glare using the transition of the eye opening amount EO after the blink correction. Blink correction is an operation of detecting the blink period from the transition of the eye opening amount EO and changing the eye opening amount EO during the detected blink period based on the eye opening amount EO before the blink period.

[0083] Therefore, it is possible to suppress deterioration of estimation accuracy at the timing of blinking or in the vicinity of that timing.

[0084] The estimation unit 12 averages the transition of the eye opening amount EO after blink correction, and estimates whether the driver 200 has become in a squinted state due to glare using the transition of the eye opening amount EO after the averaging.

[0085] Therefore, in addition to suppressing deterioration of estimation accuracy at the timing of blinking or in the vicinity of that timing, it is possible to suppress deterioration of estimation accuracy due to noise.

Explanation of Signs

[0086] 1: Drowsiness detection device 11: Acquisition unit, 12: Estimation unit, 13: Detection unit, 14: Warning unit 20: Camera, 30: Vehicle ECU, 40: Warning device 100: Vehicle 200: Driver, 210: Eyes EO: Eye opening amount P1, P2, P3: Points R1, R2, R3: Periods t1, t2, t3, t4: Timings

Claims

1. An acquisition unit that acquires an eye-opening amount, which is numerical information corresponding to the amount by which the driver of the vehicle raises their eyelids, based on a face image of the driver; An estimation unit that estimates whether or not the driver has squinted due to glare based on the eye-opening amount and the rate of decrease of the eye-opening amount; A detection unit that, when it is estimated that the driver has squinted due to glare, does not perform drowsiness detection based on the eye-opening amount, and when it is estimated that the driver has not squinted due to glare, performs drowsiness detection based on the eye-opening amount. A drowsiness detection device comprising the above.

2. The estimation unit estimates that the driver has squinted due to glare when the eye-opening amount is less than a first threshold value and the rate of decrease of the eye-opening amount is faster than a second threshold value, and estimates that the driver has not squinted due to glare when the eye-opening amount is greater than the first threshold value or the rate of decrease of the eye-opening amount is slower than the second threshold value. The drowsiness detection device according to Claim 1.

3. The estimation unit performs blink correction on the transition of the eye-opening amount, and estimates whether or not the driver has squinted due to glare using the transition of the eye-opening amount after the blink correction. The blink correction is an operation of detecting a blink period from the transition of the eye-opening amount and changing the eye-opening amount during the detected period based on the eye-opening amount before the period. The drowsiness detection device according to Claim 1 or Claim 2.

4. The estimation unit performs moving averaging on the transition of the eye-opening amount after the blink correction, and estimates whether or not the driver has squinted due to glare using the transition of the eye-opening amount after the moving averaging. The drowsiness detection device according to Claim 3.

Images