Drowsiness detection device and program for detecting drowsiness

Abstract

To provide a drowsiness detector and a program for the drowsiness detection capable of improving accuracy of drowsiness detection.SOLUTION: A PERCLOS calculation unit 33 calculates a PERCLOS, which is a time ratio that a driver is closing his or her eyes for a fixed time Ta, on the basis of an image captured by a camera 2. A drowsiness detection unit 34 detects drowsiness of the driver when the PERCLOS is a threshold or more. A threshold setting unit 35 sets the threshold lower when the initial PERCLOS calculated first from the start of driving is less than a% than when the initial PERCLOS is a% or more.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a drowsiness detection device and a program for drowsiness detection. [Background technology]

[0002] PERCLOS (PERcent of eye lid CLOSure) is one of the indices for measuring the degree of fatigue of a driver of a vehicle such as an automobile (Patent Document 1). PERCLOS is the percentage of time that the driver has his / her eyes closed per minute. Drowsiness can be detected when this PERCLOS is equal to or greater than a threshold value. However, there is an issue that PERCLOS when a driver feels drowsy varies from person to person, and if the threshold value is set uniformly, drowsiness cannot be detected with high accuracy.

[0003] Therefore, it is conceivable to apply the technology described in Patent Document 2 so that the driver can change the threshold value at will. However, when the driver changes the threshold value by himself, he needs to check the past PERCLOS history and the threshold value, which is troublesome and may result in the threshold value not being changed or not being set to an appropriate value. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2022-42248 [Patent Document 2] Japanese Patent Application Publication No. 10-40481 Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention has been made in consideration of the above-mentioned circumstances, and has an object to provide a drowsiness detection device and a drowsiness detection program that improve the accuracy of drowsiness detection. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, a drowsiness detection device according to the present invention has the following features. An imaging unit that captures an image of a face of a person to be detected; a first calculation unit that calculates a drowsiness index value that is a time during which the subject has his / her eyes closed during a first predetermined time or a value corresponding to the time based on the image captured by the image capture unit; a drowsiness detection unit that detects drowsiness of the detection subject when the drowsiness index value is equal to or greater than a threshold value; 、 a threshold setting unit that sets the threshold based on an initial drowsiness index value that is calculated for the first time after a predetermined trigger; and, A second calculation unit that calculates the number of blinks of the detection target person in a second predetermined time based on the image captured by the image capture unit; Equipped with The threshold setting unit sets the threshold based on the initial drowsiness index value and an initial number of blinks calculated for the first time after the predetermined trigger. It is a drowsiness detection device.

[0007] In order to achieve the above-mentioned object, a drowsiness detection program according to the present invention has the following features. On the computer, a first calculation unit that calculates a drowsiness index value that is a time during which the detection subject has his / her eyes closed during a first predetermined time or a value corresponding to the time, based on an image of the detection subject's face captured by an image capture unit; a drowsiness detection unit that detects drowsiness of the detection subject when the drowsiness index value is equal to or greater than a threshold value; a threshold setting unit that sets the threshold based on an initial drowsiness index value that is calculated for the first time after a predetermined trigger; A second calculation unit that calculates the number of blinks of the detection target person in a second predetermined time based on the image captured by the image capture unit; and make it work 、 The threshold setting unit sets the threshold based on the initial drowsiness index value and an initial number of blinks calculated for the first time after the predetermined trigger. It is a program for detecting drowsiness. Effect of the Invention

[0008] According to the present invention, it is possible to provide a drowsiness detection device and a drowsiness detection program that improve the accuracy of drowsiness detection.

[0009] The present invention has been briefly described above. Furthermore, the details of the present invention will be further clarified by reading the following description of the preferred embodiment of the present invention (hereinafter, referred to as "embodiment") with reference to the accompanying drawings. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing a drowsiness detection device according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a time chart for explaining the function of the eye closure detection unit shown in FIG. [Diagram 3] FIG. 3 is a time chart for explaining the function of the PERCLOS calculation unit shown in FIG. [Figure 4] FIG. 4 is a flowchart showing the operation of the drowsiness detection device shown in FIG. [Diagram 5] FIG. 5 is a time chart showing PERCLOS for a person with a normal blink rate and a person with a low blink rate. [Figure 6] FIG. 6 is a time chart showing the opening and closing of the eyes of a person who is blinking deeply. [Figure 7] FIG. 7 is a time chart showing the opening and closing of the eyes of a person who blinks shallowly. [Figure 8] FIG. 8 is a block diagram showing a drowsiness detection device according to the second embodiment of the present invention. [Figure 9] FIG. 9 is a flowchart showing the operation of the drowsiness detection device shown in FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Specific embodiments of the present invention will be described below with reference to the accompanying drawings.

[0012] (First embodiment) The drowsiness detection device 1 of the present embodiment shown in Fig. 1 is a device that is mounted in a vehicle and detects drowsiness of a driver who is a detection subject seated in the driver's seat. As shown in the figure, the drowsiness detection device 1 includes a camera 2 as an imaging unit, a control unit 3, and an alarm output unit 4.

[0013] The camera 2 is installed in the meter unit in front of the driver's seat or on the column cover so that it can capture an image of the face of the driver sitting in the driver's seat. The camera 2 supplies the captured image of the driver's face to the control unit 3, which will be described later.

[0014] The control unit 3 is configured by a known microcomputer including a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc., and functions as a computer that operates according to a program for drowsiness detection. The control unit 3 has a face detection unit 31, a closed-eye detection unit 32, a PERCLOS calculation unit 33 as a first calculation unit, a drowsiness detection unit 34, and a threshold setting unit 35.

[0015] The face detection unit 31 extracts a face image from the image captured by the camera 2. The eye closure detection unit 32 obtains the degree of eye opening and closing from the face image detected by the face detection unit 31, and detects closed eyes from the degree of eye opening and closing. In this embodiment, as shown in FIG. 2, when the degree of eye opening and closing is 100% fully open and 0% fully closed, closed eyes are detected when the degree of eye opening and closing becomes a certain value (e.g., 20%) or less, and the detection result is supplied to the PERCLOS calculation unit 33.

[0016] As shown in FIG. 3, the PERCLOS calculation unit 33 calculates PERCLOS shown in the following formula (1) as a drowsiness index value based on the result of eye closure detection from the eye closure detection unit 32. PERCLOS=(T1+T2+T3) / Ta×100% …(1)

[0017] PERCLOS is the ratio of the time (T1+T2+T3) that the driver has his / her eyes closed to the given time Ta (=first predetermined time). The PERCLOS calculation unit 33 calculates PERCLOS for each given time Ta.

[0018] The drowsiness detection unit 34 detects the driver's drowsiness when PERCLOS is equal to or greater than a threshold, and generates a warning in the warning output unit 4. The threshold setting unit 35 sets a threshold based on an initial PERCLOS calculated at the first fixed time Ta after the start of driving is detected (after a predetermined trigger).

[0019] The warning output unit 4 is composed of a display device such as a meter display, a navigation monitor, or a head-up display, and notifies the driver by sound, voice, text, icon, or the like that drowsiness has been detected or that a break is recommended.

[0020] Next, the operation of the above-mentioned drowsiness detection device 1 will be described with reference to the flowchart of Fig. 4. First, when the control unit 3 detects the start of driving, it resets the threshold value used for the above-mentioned drowsiness detection (i.e., sets it to the initial state). The control unit 3 detects the start of driving, for example, when the ignition switch is turned on.

[0021] The control unit 3 repeatedly performs the drowsiness detection process shown in Fig. 4 every time one frame of image captured by the camera 2 is supplied. In the drowsiness detection process, the control unit 3 extracts a face image from the image captured by the camera 2 (S1). Next, the control unit 3 obtains the degree of eye opening and closing from the face image detected by the face detection unit 31, and detects eye closure based on the degree of eye opening and closing (S2). Next, when closed eyes are detected in S2, the control unit 3 counts up the time that the driver's eyes are closed, and when a certain time Ta has elapsed, calculates PERCOLS based on the counted up time (S3).

[0022] If the fixed time Ta has not elapsed and PERCLOS has not been calculated (N in S4), the control unit 3 ends the process. If the fixed time Ta has elapsed and PERCLOS has been calculated (Y in S4), the control unit 3 determines whether or not a threshold has been set (S5). If a threshold has been set (Y in S5), the control unit 3 immediately proceeds to S9.

[0023] If the threshold value is not set (N in S5), the control unit 3 determines whether the calculated PERCLOS is equal to or greater than a % (a predetermined value) (S6). When the threshold value is not set, the PERCLOS calculated in S3 is the initial PERCLOS calculated for the first time after the start of operation is detected.

[0024] If the initial PERCLOS is equal to or greater than a% (Y in S6), the control unit 3 sets the threshold to C1% (S8) and then proceeds to S9. If the initial PERCLOS is less than a% (N in S6), the control unit 3 sets the threshold to C2%, which is lower than C1%, and then proceeds to S9.

[0025] In S9, if PERCLOS calculated in S3 is equal to or greater than the threshold (Y in S9), the control unit 3 determines that the subject is drowsy (S11), controls the alarm output unit 4 to generate an alarm, and then ends the process. If PERCLOS calculated in S3 is less than the threshold (N in S9), the control unit 3 determines that the subject is not drowsy (S10), and ends the process.

[0026] As shown in Figure 5, even if a person who blinks infrequently becomes sleepy and blinks more frequently, PERCLOS does not increase as much compared to a person who blinks normally. If the threshold is set uniformly, for example to 12%, as in the conventional technology, it is not possible to detect drowsiness in people who blink infrequently.

[0027] According to the above-described embodiment, when the initial PERCLOS is less than a%, the control unit 3 sets the threshold lower than when the initial PERCLOS is a% or more (C1>C2). That is, as shown in FIG. 6, a person who blinks deeply has a high PERCLOS, and as shown in FIG. 7, a person who blinks shallowly has a low PERCLOS without detecting closed eyes. Therefore, a person who blinks less frequently or shallowly has an initial PERCLOS less than a%, and the threshold can be set lower than the threshold corresponding to a person who blinks normally. This makes it possible to set a threshold appropriate for the driver, thereby improving the accuracy of drowsiness detection.

[0028] According to the above-described embodiment, the detection of the start of driving is set as the predetermined trigger, which allows the threshold value to be determined based on the initial drowsiness index value close to the time of awakening, thereby further improving the accuracy of drowsiness detection.

[0029] Second embodiment Next, a drowsiness detection device 1 of a second embodiment will be described with reference to FIG. 8. In FIG. 8, parts equivalent to those of the drowsiness detection device 1 shown in FIG. 1 already described in the first embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted. A difference between the drowsiness detection device 1B of the second embodiment and the drowsiness detection device 1 of the first embodiment is that the control unit 3 includes a blink count calculation unit 36. The blink count calculation unit 36 ​​counts (calculates) the number of blinks of the driver in a certain time Ta (a second predetermined time) based on the detection result of the closed eye detection unit 32. The threshold setting unit 35 sets a threshold based on the PERCLOS and the blink count calculated in the first certain time Ta after driving starts.

[0030] Next, the operation of the drowsiness detection device 1 of the second embodiment will be described with reference to the flowchart of Fig. 9. In Fig. 9, steps that perform the same operations as those in the flowchart shown in Fig. 4 already described in the first embodiment are omitted, or the same reference numerals are used and detailed descriptions thereof are omitted.

[0031] In the drowsiness detection process, the control unit 3 executes the operations of S1 to S6 in the same manner as in the first embodiment. In S6, the control unit 3 determines whether the initial PERCLOS is equal to or greater than a% (predetermined value) (S6). If the initial PERCLOS is equal to or greater than a% (Y in S6), the control unit 3 sets the threshold to C1% (S8) and then proceeds to S9. If the initial PERCLOS is less than a% (N in S6), the control unit 3 determines whether the number of blinks calculated in a certain period of time Ta is equal to or greater than b (predetermined number) (S12).

[0032] If the blink count is b or more (Y in S12), the control unit 3 sets the threshold to C2%, which is lower than C1%, (S7) and then proceeds to S9. If the blink count is less than b (N in S12), the control unit 3 sets the threshold to C3%, which is lower than C2%, (S13) and then proceeds to S9. Since the process from S9 onwards is the same as in the first embodiment, the description will be omitted.

[0033] For people whose blink counts are measured to a certain extent, PERCLOS increases slightly when they become more sleepy, but for people whose blink counts are barely measured, PERCLOS does not increase much even when they become more sleepy.

[0034] According to the above-described embodiment, when the initial PERCLOS is less than a% and the initial blink count is less than b, the control unit 3 sets the threshold lower than when the initial PERCLOS is less than a% and the initial blink count is b or more (C2>C3). As a result, a person who blinks less often will have a blink count less than b, and the threshold can be set lower than a person who blinks more often. This can further improve the accuracy of drowsiness detection. C2 and C3 may be calculated using the following formulas with PERCLOS (α) and the blink count (β) as parameters. Threshold value = n / (α × Ta / β) … (2)

[0035] Next, in order to confirm the effects of the first and second embodiments, the inventors measured the initial PERCLOS of subjects A to E (detection subjects) twice each and had them self-report their drowsiness levels when they became more drowsy. The threshold values ​​and detection results of the conventional, first, and second embodiment drowsiness detection devices 1 and 1B are summarized in Table 1 below.

[0036] [Table 1]

[0037] In the example shown in Table 1, a=2.5%, b=15 times, C1=12%, C2=5%, and C3=3%. As is clear from Table 1, in the first and second times of subject C, whose initial PERCLOS was less than 2.5%, and in the first and second times of subject D, drowsiness could not be detected even if the subject felt drowsy using the conventional technology in which the threshold value was uniformly set to 12%. In contrast, it was found that the drowsiness detection device 1 of the first embodiment was able to detect drowsiness in the second time of subject C, improving the accuracy of drowsiness detection. It was found that the drowsiness detection device 1B of the second embodiment was able to detect drowsiness in the first and second times of subject C, and in the second time of subject D, further improving the accuracy of drowsiness detection. When using formula (2) with n = 0.3, the detection results for the first and second times for subject C and the first and second times for subject D were as follows: first time threshold for subject C: 0.3 ÷ (0.2% × 180 seconds ÷ 4 times) = 3.3, second time for subject C: 4.6, first time for D: 1.8. Since the second time for subject D cannot be calculated, if the lower limit of the threshold (for example, 3) is set, the detection rate will be 90 to 100, and it is found that the accuracy is improved.

[0038] The present invention is not limited to the above-described embodiment, and can be appropriately modified, improved, etc. In addition, the material, shape, size, number, arrangement location, etc. of each component in the above-described embodiment are arbitrary as long as the present invention can be achieved, and are not limited.

[0039] According to the above-described embodiment, the drowsiness index value is calculated as the percentage of time that the driver has his / her eyes closed during the fixed time Ta, but the present invention is not limited to this. The drowsiness index value may be calculated as the integral of the time that the driver has his / her eyes closed during the fixed time Ta.

[0040] According to the above-mentioned embodiment, the control unit 3 detects the start of driving when the ignition switch is turned on and resets the drowsiness detection threshold, but this is not limited to the above. For example, since the driver may be changed with the engine running, the drowsiness detection threshold may be reset when the driver performs a reset operation (predetermined trigger), or the threshold may be remeasured at regular intervals after the start of driving and updated if there is a change.

[0041] According to the embodiment described above, the certain time Ta for calculating PERCLOS and the certain time Ta for calculating the number of blinks are the same, but this is not limited thereto and they may be different times.

[0042] Here, features of the embodiments of the drowsiness detection device and the drowsiness detection program according to the present invention described above will be briefly summarized and listed in the following [1] to [6]. [1] An imaging unit (2) that captures an image of the face of a person to be detected; a first calculation unit (33) that calculates a drowsiness index value that is a time during which the subject has his / her eyes closed during a first predetermined time or a value corresponding to the time, based on the image captured by the imaging unit (2); a drowsiness detection unit (34) that detects drowsiness of the detection subject when the drowsiness index value is equal to or greater than a threshold value; In a drowsiness detection device (1), The device further includes a threshold setting unit (35) that sets the threshold based on an initial drowsiness index value that is calculated for the first time after a predetermined trigger. Drowsiness detection device (1).

[0043] According to the above configuration [1], a threshold suitable for the driver can be set, and the accuracy of drowsiness detection can be improved.

[0044] [2] In the drowsiness detection device (1) according to [1], The threshold setting unit (35) sets the threshold lower when the initial drowsiness index value is less than a predetermined value than when the initial drowsiness index value is equal to or greater than the predetermined value. Drowsiness detection device (1).

[0045] According to the configuration of [2] above, for a person who blinks infrequently or shallowly, the initial drowsiness index value is less than the predetermined value, and the threshold can be set lower than the threshold corresponding to a person who blinks normally. This makes it possible to set a threshold more suitable for the driver, thereby improving the accuracy of drowsiness detection.

[0046] [3] In the drowsiness detection device (1) according to [1], A second calculation unit (36) that calculates the number of blinks of the detection subject in a second predetermined time based on the image captured by the image capture unit (2), The threshold setting unit (35) sets the threshold based on the initial drowsiness index value and an initial number of blinks calculated for the first time after the predetermined trigger. Drowsiness detection device (1).

[0047] According to the above configuration [3], a threshold value that is more suitable for the driver can be set, and the accuracy of drowsiness detection can be improved.

[0048] [4] In the drowsiness detection device (1) according to [3], The threshold setting unit (35) When the initial drowsiness index value is less than a predetermined value, the threshold value is set lower than when the initial drowsiness index value is equal to or greater than the predetermined value. When the initial drowsiness index value is less than the predetermined value and the initial number of blinks calculated for the first time after the predetermined trigger is less than the predetermined number, the threshold value is set lower than when the initial drowsiness index value is less than the predetermined value and the initial number of blinks is equal to or greater than the predetermined number. Drowsiness detection device (1).

[0049] According to the configuration of [4] above, the number of blinks of a person who blinks infrequently is less than the predetermined number, and the threshold value can be set lower than that of a person who blinks frequently. This makes it possible to further improve the accuracy of drowsiness detection.

[0050] [5] In the drowsiness detection device (1) according to any one of [1] to [4], Detection of the start of operation is set as the predetermined trigger. Drowsiness detection device (1).

[0051] According to the above configuration [5], the threshold value can be determined based on the initial drowsiness index value close to the time of awakening, thereby further improving the accuracy of drowsiness detection.

[0052] [6] A computer (3) a first calculation unit (33) that calculates a drowsiness index value, which is a time during which the detection subject has his / her eyes closed during a first predetermined time or a value corresponding to the time, based on an image of the detection subject's face captured by the capture unit (2); a drowsiness detection unit (34) that detects drowsiness of the detection subject when the drowsiness index value is equal to or greater than a threshold value; a threshold setting unit (35) that sets the threshold based on an initial drowsiness index value that is calculated for the first time after a predetermined trigger; A program for detecting drowsiness.

[0053] According to the configuration [6] above, a threshold suitable for the driver can be set, thereby improving the accuracy of drowsiness detection. [Explanation of symbols]

[0054] 1. Drowsiness detection device 2 Camera (photography section) 33 PERCLOS calculation unit (first calculation unit) 34 Drowsiness detection unit 35 Threshold setting unit 36 Blink count calculation unit (second calculation unit)

Claims

1. An imaging unit that captures an image of a face of a person to be detected; a first calculation unit that calculates a drowsiness index value that is a time during which the subject has his / her eyes closed during a first predetermined time or a value corresponding to the time, based on the image captured by the image capture unit; a drowsiness detection unit that detects drowsiness of the detection subject when the drowsiness index value is equal to or greater than a threshold value; a threshold setting unit that sets the threshold based on an initial drowsiness index value that is calculated for the first time after a predetermined trigger; A second calculation unit that calculates the number of blinks of the detection target person in a second predetermined time based on the image captured by the image capture unit; Equipped with The threshold setting unit sets the threshold based on the initial drowsiness index value and an initial number of blinks calculated for the first time after the predetermined trigger. Drowsiness detection device.

2. 2. The drowsiness detection device according to claim 1, the threshold setting unit sets the threshold lower when the initial drowsiness index value is less than a predetermined value than when the initial drowsiness index value is equal to or greater than the predetermined value. Drowsiness detection device.

3. An imaging unit that captures an image of a face of a person to be detected; a first calculation unit that calculates a drowsiness index value that is a time during which the subject has his / her eyes closed during a first predetermined time or a value corresponding to the time, based on the image captured by the image capture unit; a drowsiness detection unit that detects drowsiness of the detection subject when the drowsiness index value is equal to or greater than a threshold value; a threshold setting unit that sets the threshold based on an initial drowsiness index value that is calculated for the first time after a predetermined trigger; Equipped with the first calculation unit obtains an open / closed degree representing a degree of opening of the eyes from the image, detects closed eyes when the open / closed degree is equal to or less than a certain value, calculates a total time during which the closed eyes are detected within the first predetermined time, and calculates a ratio of the total time to the first predetermined time as the drowsiness index value; the threshold setting unit sets the threshold lower when the initial drowsiness index value is less than a predetermined value than when the initial drowsiness index value is equal to or greater than the predetermined value. Drowsiness detection device.

4. An imaging unit that captures an image of a face of a person to be detected; a first calculation unit that calculates a drowsiness index value that is a time during which the subject has his / her eyes closed during a first predetermined time or a value corresponding to the time, based on the image captured by the image capture unit; a drowsiness detection unit that detects drowsiness of the detection subject when the drowsiness index value is equal to or greater than a threshold value; a threshold setting unit that sets the threshold based on an initial drowsiness index value that is calculated for the first time after a predetermined trigger; A second calculation unit that calculates the number of blinks of the detection target person in a second predetermined time based on the image captured by the image capture unit; Equipped with The threshold setting unit sets the threshold based on the initial drowsiness index value and an initial number of blinks calculated for the first time after the predetermined trigger; The threshold setting unit is When the initial drowsiness index value is less than a predetermined value, the threshold value is set lower than when the initial drowsiness index value is equal to or greater than the predetermined value. When the initial drowsiness index value is less than the predetermined value and the initial number of blinks calculated for the first time after the predetermined trigger is less than the predetermined number, the threshold value is set lower than when the initial drowsiness index value is less than the predetermined value and the initial number of blinks is equal to or greater than the predetermined number. Drowsiness detection device.

5. The drowsiness detection device according to any one of claims 1 to 4, Detection of the start of operation is set as the predetermined trigger. Drowsiness detection device.

6. On the computer, a first calculation unit that calculates a drowsiness index value that is a time during which the detection subject has his / her eyes closed during a first predetermined time or a value corresponding to the time, based on an image of the detection subject's face captured by the image capture unit; a drowsiness detection unit that detects drowsiness of the detection subject when the drowsiness index value is equal to or greater than a threshold value; a threshold setting unit that sets the threshold based on an initial drowsiness index value that is calculated for the first time after a predetermined trigger; a second calculation unit that calculates the number of blinks of the detection subject in a second predetermined time based on the image captured by the image capture unit; The threshold setting unit sets the threshold based on the initial drowsiness index value and an initial number of blinks calculated for the first time after the predetermined trigger. A program for detecting drowsiness.

Images