Eyelid opening degree detection device, eyelid opening degree detection method, and drowsiness determination system

Abstract

This level of eyelid opening detection device comprises: a vehicle-interior video acquisition unit (311) that acquires video of an occupant inside a vehicle; a level of eyelid opening calculation unit (312) that calculates, on the basis of the video of the occupant, a level of eyelid opening which indicates the opening state of the eye of the occupant for a plurality of frame images included in the video; a histogram creation unit (313) that creates, on the basis of the calculated level of eyelid opening, a histogram for the level of eyelid opening for the opening state of the eye of the occupant; a histogram evaluation unit (314) that evaluates whether the created histogram includes false detection data, and outputs the result of the evaluation as a histogram evaluation result; and a personal-feature-amount calculation unit (315) that calculates a personal feature amount of the occupant in accordance with the output histogram evaluation result.

Description

[Technical Field]

[0001] The present disclosure relates to eyelid opening degree detection technology. [Background technology]

[0002] There is a technology for determining whether a driver or other occupant is in a state suitable for driving a vehicle by calculating an index value indicating the degree to which the occupant's eyes are open and making a determination using the calculated index value. For example, Patent Document 1 discloses a dozing-off determination device and an eyelid detection device constituting the dozing-off determination device, and one embodiment of the eyelid detection device includes an eye-opening degree calculation unit. Section 0054 of Patent Document 1 describes that the eye-opening degree calculation unit uses information about the driver's eyes in a normal state to calculate the vertical distance of the eye area in the normal state, and sets this distance as the reference distance for 100% eye-opening. Note that the degree of eye-opening is also referred to as the eyelid opening degree, and the term eyelid opening degree is used in this disclosure. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2018 / 134875 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with conventional technology, when calculating the vertical distance of the occupant's eye area, false detections can occur due to makeup or other factors applied near the occupant's eyes, making it impossible to properly calculate the eyelid opening degree used as a standard.

[0005] The present disclosure has been made to solve such problems, and aims to provide an eyelid opening degree detection technique that can appropriately calculate an eyelid opening degree to be used as a standard. [Means for solving the problem]

[0006] An eyelid opening degree detection device according to one aspect of an embodiment of the present disclosure includes an interior image acquisition unit that acquires an image of an occupant inside a vehicle; an eyelid opening degree calculation unit that calculates, based on the image of the occupant, an eyelid opening degree that indicates the degree of eye opening of the occupant for a plurality of frame images included in the image; a histogram creation unit that creates a histogram of the eyelid opening degree when the occupant has their eyes open, based on the calculated eyelid opening degree; a histogram evaluation unit that evaluates whether the created histogram includes false detection data by analyzing the created histogram and outputs the result of the evaluation as a histogram evaluation result; and an individual feature calculation unit that calculates, based on the output histogram evaluation result, an individual feature related to the eyelid opening degree of the occupant, The histogram evaluation unit evaluates that the created histogram contains falsely detected data when two peaks are present in the histogram, and the personal feature calculation unit, when it is evaluated that the created histogram contains falsely detected data, calculates the most frequent eyelid opening degree as the personal feature of the occupant if the calculated eyelid opening degree contains a peak with an eyelid opening degree greater than the most frequent modal eyelid opening degree, and calculates the eyelid opening degree of the peak with the smaller eyelid opening degree as the personal feature of the occupant if the calculated eyelid opening degree contains a peak with an eyelid opening degree smaller than the most frequent eyelid opening degree. . [Effects of the Invention]

[0007] According to the eyelid opening degree detection technology according to the embodiment of the present disclosure, it is possible to appropriately calculate the eyelid opening degree to be used as a standard. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram showing an example of the configuration of an eyelid opening degree detection device and a drowsiness determination system; [Figure 2A] FIG. 1 is a diagram illustrating an example of the hardware configuration of an eyelid opening detection device and a drowsiness determination system. [Figure 2B] FIG. 1 is a diagram illustrating an example of the hardware configuration of an eyelid opening detection device and a drowsiness determination system. [Figure 3] 10 is a flowchart showing the operation of the eyelid opening degree detection device. [Figure 4] FIG. 2 is a schematic diagram of an eye for explaining the degree of eyelid opening. [Figure 5] 10 is an example of a histogram of flattening as the degree of eyelid opening. DETAILED DESCRIPTION OF THE INVENTION

[0009] Various embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Note that components with the same or similar reference numerals in the drawings have the same or similar configurations or functions, and redundant descriptions of such components will be omitted.

[0010] Embodiment 1 <Configuration> (Drowsiness detection system) An eyelid opening degree detection device and a drowsiness determination system according to a first embodiment of the present disclosure will be described with reference to FIG. 1. FIG. 1 is a block diagram illustrating an example configuration of an eyelid opening degree detection device 31 and a drowsiness determination system 30 according to the first embodiment of the present disclosure. As shown in FIG. 1, a vehicle V includes an imaging device 10, a vehicle information acquisition device 20, and a drowsiness determination system 30. The drowsiness determination system 30 also includes an eyelid opening degree detection device 31 and a drowsiness determination device 32. The drowsiness determination device 32 determines whether the occupant is currently drowsy using the occupant's standard eyelid opening degree calculated by the eyelid opening degree detection device 31. For example, the drowsiness determination device 32 determines that the occupant is drowsy when the ratio of the occupant's current eyelid opening degree to the occupant's standard eyelid opening degree is equal to or less than a predetermined threshold. The current eyelid opening degree of the occupant is calculated, for example, by an eyelid opening degree calculation unit 312 (described later), and the current eyelid opening degree calculated by the eyelid opening degree calculation unit 312 is acquired by the drowsiness determination device 32.

[0011] (imaging device) The imaging device 10 is a device for capturing images of occupants of a vehicle V. The imaging device 10 is installed, for example, in the front part of the cabin of the vehicle V, and captures an image of an area including the face of an occupant such as a driver of the vehicle V from the front. The imaging device 10 is configured with one visible light camera, multiple visible light cameras, one infrared camera, or multiple infrared cameras. When the imaging device 10 is configured with an infrared camera, a light source (not shown) is provided that irradiates an area including the driver's face with infrared light for capturing images. This light source is configured, for example, with an LED (Light Emitting Diode). The imaging device 10 outputs a video consisting of the captured multiple frame images to an eyelid opening detection device 31 provided in the drowsiness determination system 30.

[0012] (Vehicle information acquisition device) The vehicle information acquisition device 20 is a sensor that acquires the driving state of the vehicle V. Examples of the driving state include vehicle speed, steering angle, and gear shift information. The vehicle information acquisition device 20 outputs the acquired information about the driving state to an eyelid opening detection device 31 provided in the drowsiness determination system 30.

[0013] (Eyelid opening detection device) The eyelid opening degree detection device 31 is a device for calculating a standard eyelid opening degree of an occupant. Because the degree of eye opening varies from person to person, it is necessary to calculate the eyelid opening degree of the occupant monitored by the drowsiness determination system 30 when the eyes are open. Therefore, the eyelid opening degree detection device 31 calculates the eyelid opening degree when the eyes are open, which is used by the drowsiness determination system 30, i.e., the standard eyelid opening degree. To realize this function, the eyelid opening degree detection device 31 includes an in-vehicle image acquisition unit 311, an eyelid opening degree calculation unit 312, a histogram creation unit 313, a histogram evaluation unit 314, and an individual feature calculation unit 315. The eyelid opening degree detection device 31 also includes a control unit (not shown) as a functional unit that controls the overall operation of the eyelid opening degree detection device 31.

[0014] (In-car video acquisition unit) The in-vehicle image acquisition unit 311 acquires the image output by the imaging device 10 .

[0015] (Lid opening degree calculation unit) The eyelid opening degree calculation unit 312 calculates the eyelid opening degree for each frame image included in the video acquired by the in-vehicle video acquisition unit 311. Here, the eyelid opening degree is an index showing the degree of eye opening. As an example, as shown in FIG. 4, the eyelid opening degree is a flattening ratio calculated by dividing the distance Ev between the line connecting the coordinates of the inner corner and outer corner of the eye and the highest point of the upper eyelid by the distance Eh between the coordinates of the inner corner and outer corner of the eye. Note that the highest point of the upper eyelid is the point (vertex) of the upper eyelid that is farthest from the line connecting the coordinates of the inner corner and outer corner of the eye.

[0016] Eyelid opening degree calculation unit 312 acquires the coordinates of the inner corner of the eye, the coordinates of the outer corner of the eye, and the coordinates of the vertex of the upper eyelid, for example, using the results of machine learning. That is, eyelid opening degree calculation unit 312 acquires position information such as the coordinates of the inner corner of the eye by inputting each frame image into a trained model that has undergone machine learning to learn the relationship between an image including a face and feature points of the inner corner of the eye, the outer corner of the eye, and the upper eyelid. Eyelid opening degree calculation unit 312 may detect a face region using a known algorithm such as Haar-Like, and calculate feature points from the detected face region on a program basis using known image processing technology.

[0017] Furthermore, the eyelid opening degree calculation unit 312 may acquire position information such as the coordinates of the inner corner of the eye by combining a machine learning model and a program-based model. For example, the machine learning model may be used to acquire position information of the inner corner and outer corner of the eye, and the upper eyelid vertex may be acquired by image processing. The upper eyelid vertex may be acquired by, for example, detecting the edge of the upper eyelid through image processing using a differential filter, detecting the upper eyelid line by performing curve fitting on the detected points, and calculating the coordinates of the upper eyelid vertex.

[0018] The eyelid opening degree may be determined by the inter-eyelid distance, which is the distance between the upper and lower eyelids. For example, the distance between the apex of the upper eyelid and the apex of the lower eyelid may be used. In this case, the apex of the lower eyelid may be calculated using the same method as that used to calculate the apex of the upper eyelid.

[0019] For frames associated with samples that are not suitable as eyelid opening degree samples, the eyelid opening degree calculation unit 312 does not need to calculate the eyelid opening degree. For example, if the occupant is wearing eyeglasses, a landscape reflection may be reflected in the lenses of the eyeglasses, which may hinder the calculation of the eyelid opening degree. Therefore, for frames in which an obstructing factor such as a landscape reflection exists, the eyelid opening degree calculation unit 312 does not need to calculate the eyelid opening degree. To detect such landscape reflections, the eyelid opening degree calculation unit 312 may be configured, for example, to acquire both eyes of the occupant from the in-vehicle image acquisition unit 311, and may determine that a landscape reflection exists when only one of the occupant's eyes can be acquired. Since occupants typically sit on either the left or right door of the vehicle, landscape reflections often occur only on one of the lenses of the eyeglasses. Therefore, when only one of the occupant's eyes can be acquired, it can be estimated that a landscape reflection has occurred on the glass of the eyeglasses in front of the other eye. The eyelid opening degree calculation unit 312 does not need to calculate the eyelid opening degree for samples in which the reliability of the eyelid opening degree is considered to be low due to obstructive factors other than the scenery reflection described above.

[0020] (Histogram creation section) When it is determined that the vehicle is traveling, histogram creation unit 313 creates a histogram of the eyelid opening degree when the occupant's eyes are open, from the eyelid opening degree calculated by eyelid opening degree calculation unit 312. Based on the created histogram of eyelid opening degree, a standard eyelid opening degree is calculated in subsequent processing.

[0021] Since the purpose of creating a histogram is to calculate the eyelid opening degree to be used as a standard, samples when the occupant's eyes are not open are inappropriate or unnecessary for the created histogram. Therefore, if the eyelid opening degree value of a certain frame is equal to or less than a predetermined threshold, the histogram creation unit 313 creates an eyelid opening degree histogram without using the eyelid opening degree of that frame. For example, if the eyelid opening degree value is equal to or less than the predetermined threshold of 0.05, the eyelid opening degree histogram is created without using that eyelid opening degree value. By performing threshold processing on the eyelid opening degree in this way, a histogram of the eyelid opening degree in the open-eye state can be created.

[0022] The histogram creation unit 313 determines whether the vehicle is traveling based on information about the traveling state acquired by the vehicle information acquisition device 20. For example, if the vehicle speed is equal to or greater than a predetermined threshold value, such as 10 km / h, 15 km / h, or 20 km / h, it may be determined that the vehicle is traveling. Regarding the steering angle, for example, if the steering angle is within a predetermined range, such as ±10 degrees, from the front of the vehicle as a reference, it may be determined that the vehicle is traveling. Regarding the shift information, if the shift mode is selected, it may be determined that the vehicle is traveling. It may also be determined that the vehicle is traveling if two or more conditions related to the vehicle speed, the steering angle, and the shift information are satisfied.

[0023] Fig. 5 shows an example of a histogram created by histogram creation unit 313. Fig. 5 shows an example of a histogram when flattening is used as the eyelid opening degree. That is, the histogram has flattening on the horizontal axis and frequency on the vertical axis. The histogram in Fig. 5 has two peaks at flattening values ​​of 0.21 and 0.42. When the inter-eyelid distance is used as the eyelid opening degree, a histogram using the inter-eyelid distance as a variable is created.

[0024] The two peaks are due to the fact that the samples used to create the histogram included false positives, where the upper eyelid was not correctly detected. Examples of factors that can cause false positives include eyelashes, eyelash extensions, eye makeup, bulging of the upper eyelid area, and deep-set eyes. At least one of these factors will cause a false positive.

[0025] (Histogram evaluation section) The histogram evaluation unit 314 evaluates whether the histogram created by the histogram creation unit 313 contains falsely detected data. This evaluation is performed based on whether the histogram contains one peak. If the histogram contains one peak, the histogram evaluation unit 314 evaluates that the histogram does not contain falsely detected data. On the other hand, if the histogram contains more than one peak, i.e., if the histogram contains two peaks, the histogram evaluation unit 314 evaluates that the histogram contains falsely detected data. The histogram evaluation unit 314 outputs an evaluation result indicating the result of the evaluation of whether the histogram contains falsely detected data as a histogram evaluation result.

[0026] (Personal feature calculation unit) The personal feature calculation unit 315 calculates the personal feature of the occupant based on the histogram created by the histogram creation unit 313 and in accordance with the histogram evaluation result of the histogram evaluation unit 314. The personal feature of the occupant means the standard of the occupant's eyelid opening degree.

[0027] If the histogram evaluation result indicates that the created histogram does not contain any false detection data, the individual feature calculation unit 315 calculates the most frequent modal eyelid opening value as the individual feature of the occupant.

[0028] On the other hand, if the histogram evaluation result indicates that the created histogram contains false detection data, the personal feature calculation unit 315 calculates the personal feature of the occupant according to the relationship between the most frequent peak of eyelid openness and the less frequent peaks of eyelid openness. Specifically, if there is a peak whose eyelid openness value is greater than the most frequent peak of eyelid openness, the personal feature calculation unit 315 calculates the value of the most frequent peak of eyelid openness as the personal feature of the occupant. Conversely, if there is a peak whose eyelid openness value is smaller than the most frequent peak of eyelid openness, the personal feature calculation unit 315 calculates the value of the peak whose eyelid openness value is smaller as the personal feature of the occupant.

[0029] The false detections caused by the aforementioned false detection factors all occur in the area above the actual upper eyelid line. Therefore, if the created histogram contains two peaks, the peak with the smaller eyelid opening degree value is considered to be the peak of the upper eyelid opening degree, and the peak with the larger eyelid opening degree value is considered to be the peak due to false detection. Therefore, if the histogram contains two peaks, the personal feature calculation unit 315 calculates the smaller of the eyelid opening degree values ​​forming the two peaks as the personal feature of the occupant. By calculating the personal feature of the occupant in this manner, it is possible to appropriately calculate the eyelid opening degree to be used as a standard.

[0030] Using the individual feature calculated by the individual feature calculation unit 315, the drowsiness determination device 32 determines whether the occupant is currently feeling drowsy.

[0031] Next, an example of the hardware configuration of eyelid opening degree detection device 31 will be described with reference to Figures 2A and 2B. Each functional unit of eyelid opening degree detection device 31 is realized by a processing circuitry. The processing circuitry may be a dedicated processing circuit 100a as shown in Figure 2A, or a processor 100b that executes a program stored in memory 100c as shown in Figure 2B.

[0032] When the processing circuitry is a dedicated processing circuit 100a, the dedicated processing circuit 100a may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination thereof. Each functional unit may be realized by a plurality of separate processing circuits, or each functional unit may be realized together by a single processing circuit.

[0033] When the processing circuitry is a processor 100b, each functional unit is realized by software, firmware, or a combination of software and firmware. Software and firmware are written as programs and stored in memory 100c. The processor 100b realizes each functional unit by reading and executing the programs stored in the memory. Examples of memory 100c include non-volatile or volatile semiconductor memory such as random access memory (RAM), read-only memory (ROM), flash memory, erasable programmable read-only memory (EPROM), and electrically erasable programmable read-only memory (EEPROM), as well as magnetic disks, flexible disks, optical disks, compact disks, minidisks, and DVDs.

[0034] It is also possible to implement some of the multiple functional units using dedicated hardware, and other units using software or firmware. In this way, the processing circuit can implement each functional unit using hardware, software, firmware, or a combination of these.

[0035] The hardware of the drowsiness determination device 32 can be configured in the same way as the hardware of the eyelid opening degree detection device 31.

[0036] <Operation> Next, the operation of the eyelid opening degree detection device 31 will be described with reference to the flowchart of FIG.

[0037] In step ST0, the in-vehicle image acquisition unit 311 acquires, from the imaging device 10, an image of the occupants inside the vehicle V captured by the imaging device 10.

[0038] In step ST1, a control unit (not shown) of the eyelid opening degree detection device 31 determines whether or not calculation of the personal feature amount has been completed. That is, it determines whether or not calculation of the personal feature amount has been performed by the personal feature amount calculation unit 315. If calculation of the personal feature amount has been completed, the process ends. If calculation of the personal feature amount has not been completed, the process proceeds to step ST2.

[0039] In step ST2, the eyelid opening degree calculation unit 312 calculates the eyelid opening degree for multiple frames included in the acquired video, and the histogram creation unit 313 creates a histogram of the eyelid opening degree from the multiple eyelid opening degrees calculated by the eyelid opening degree calculation unit 312.

[0040] In step ST3, the histogram creation unit 313 determines whether or not the accumulation of the eyelid openness degree has been completed. That is, the histogram creation unit 313 determines whether or not to end the creation of the histogram of the eyelid openness degree. The condition for ending the creation of the histogram is that a predetermined number of samples, for example, several hundred samples, have been obtained. If the accumulation of the eyelid openness degree has not been completed, the process returns to step ST0, and an image for calculating the eyelid openness degree is acquired. If the accumulation of the eyelid openness degree has been completed, the process proceeds to step ST4.

[0041] In step ST4, the histogram evaluation unit 314 searches for peaks in the histogram of eyelid opening degrees created by the histogram creation unit 313. That is, the histogram evaluation unit 314 searches for peaks and counts the number of peaks.

[0042] In step ST5, the histogram evaluation unit 314 determines whether the number of peaks obtained by the search is one. This determination determines whether the created histogram of eyelid opening degree contains false positive data. If there is more than one peak, i.e., if there are two peaks, the histogram evaluation unit 314 determines that the histogram contains false positive data, and the process proceeds to step ST6. On the other hand, if there is one peak, the histogram evaluation unit 314 determines that the histogram does not contain false positive data, and the process proceeds to step ST7.

[0043] In step ST6, the individual feature calculation unit 315 determines whether or not there is a peak on the side where the flattening ratio (eyelid opening degree) is smaller than the most frequent flattening ratio (the most frequent eyelid opening degree). That is, the histogram evaluation unit 314 determines whether or not other non-frequent peaks are formed at positions where the flattening ratio value is smaller than the most frequent flattening ratio value. If the result of the determination is No, the process proceeds to step ST7. If the result of the determination is Yes, the process proceeds to step ST8.

[0044] In step ST7, the individual feature calculation section 315 calculates the value of the most frequent flattening as the individual feature.

[0045] In step ST8, the individual characteristic amount calculation unit 315 calculates the value of the flattening of the other peak as the individual characteristic amount. That is, the individual characteristic amount calculation unit 315 calculates the value of the flattening of the other peak that does not have the highest frequency as the individual characteristic amount of the occupant.

[0046] <Additional Notes> Some aspects of the various embodiments described above are summarized below.

[0047] (Appendix 1) The eyelid opening degree detection device (100) of Supplementary Note 1 includes an in-vehicle image acquisition unit (311) that acquires an image of an occupant inside a vehicle; an eyelid opening degree calculation unit (312) that calculates, based on the image of the occupant, an eyelid opening degree that indicates the degree to which the occupant's eyes are open for a plurality of frame images included in the image; a histogram creation unit (313) that creates, based on the calculated eyelid opening degree, a histogram of the eyelid opening degree when the occupant has their eyes open; a histogram evaluation unit (314) that evaluates whether the created histogram contains false detection data and outputs the evaluation result as a histogram evaluation result; and an individual feature calculation unit (315) that calculates individual feature amounts of the occupant according to the output histogram evaluation result.

[0048] (Appendix 2) The eyelid opening degree detection device of Appendix 2 is the eyelid opening degree detection device described in Appendix 1, wherein the eyelid opening degree is a flattening ratio obtained by dividing the distance between the straight line connecting the inner corner and outer corner of the occupant's eye and the top of the upper eyelid by the distance of the straight line connecting the inner corner and outer corner of the eye.

[0049] (Appendix 3) The eyelid opening degree detection device of Supplementary Note 3 is the eyelid opening degree detection device described in Supplementary Note 1, wherein the eyelid opening degree is an inter-eyelid distance between an upper eyelid and a lower eyelid of the occupant. 2. The eyelid opening detection device according to claim 1.

[0050] (Appendix 4) The eyelid opening degree detection device of Appendix 4 is an eyelid opening degree detection device described in any one of Appendixes 1 to 3, wherein the histogram creation unit creates the histogram when it determines that the vehicle is moving based on the vehicle speed, steering angle, or shift information of the vehicle acquired by a vehicle information acquisition device.

[0051] (Appendix 5) The eyelid opening degree detection device of Supplementary Note 5 is an eyelid opening degree detection device described in any one of Supplementary Notes 1 to 4, wherein the histogram evaluation unit evaluates that the created histogram does not contain false detection data if there is only one peak in the histogram, and the personal feature calculation unit, if it has evaluated that the created histogram does not contain false detection data, calculates the most frequent eyelid opening degree among the calculated eyelid opening degrees as the personal feature of the occupant.

[0052] (Appendix 6) The eyelid opening detection device of Supplementary Note 6 is the eyelid opening detection device described in any one of Supplementary Notes 1 to 4, wherein the histogram evaluation unit evaluates that the created histogram contains false detection data when two peaks are present in the histogram, and when it is evaluated that the created histogram contains false detection data, the personal feature calculation unit calculates the most frequent eyelid opening degree as the personal feature of the occupant if the calculated eyelid opening degree contains a peak with a greater eyelid opening degree than a most frequent modal eyelid opening degree, and calculates the eyelid opening degree of the peak with the smaller eyelid opening degree as the personal feature of the occupant if the calculated eyelid opening degree contains a peak with a smaller eyelid opening degree than the most frequent modal eyelid opening degree.

[0053] (Appendix 7) The drowsiness determination system of Appendix 7 includes an eyelid opening detection device (31) described in any one of Appendixes 1 to 6, and a drowsiness determination device (32) that determines the drowsiness of the occupant based on the calculated personal characteristics of the occupant.

[0054] (Appendix 8) The eyelid opening degree detection method of Supplementary Note 8 is an eyelid opening degree detection method performed by an eyelid opening degree detection device (31) including an in-vehicle image acquisition unit (311), an eyelid opening degree calculation unit (312), a histogram creation unit (313), a histogram evaluation unit (314), and an individual feature amount calculation unit (315), the method including: a step (ST0) in which the in-vehicle image acquisition unit acquires an image of an occupant inside a vehicle; and a step (ST1) in which the eyelid opening degree calculation unit calculates, based on the image of the occupant, an eyelid opening degree indicating an eye opening degree of the occupant for a plurality of frame images included in the image. T2), a step (ST2) in which the histogram creation unit creates a histogram of the eyelid opening degree when the occupant has their eyes open based on the calculated eyelid opening degree, steps (ST4 to ST6) in which the histogram evaluation unit evaluates whether the created histogram contains false detection data and outputs the evaluation result as a histogram evaluation result, and steps (ST7 to ST8) in which the personal feature calculation unit calculates the personal feature of the occupant in accordance with the output histogram evaluation result.

[0055] It is possible to combine the embodiments, and to modify or omit each embodiment as appropriate. [Industrial Applicability]

[0056] The eyelid opening degree detection technology of the present disclosure can be used as a technology for obtaining a standard eyelid opening degree of a passenger such as a driver, which is used in a PMS (Passenger Monitoring System). [Explanation of symbols]

[0057] 10 Imaging device, 20 Vehicle information acquisition device, 30 Drowsiness determination system, 31 Eyelid opening detection device, 32 Drowsiness determination device, 100a Processing circuit, 100b Processor, 100c Memory, 311 In-vehicle image acquisition unit, 312 Eyelid opening calculation unit, 313 Histogram creation unit, 314 Histogram evaluation unit, 315 Personal feature calculation unit.

Claims

1. an interior image acquisition unit that acquires an image of an occupant inside the vehicle; an eyelid opening degree calculation unit that calculates an eyelid opening degree indicating an eye opening degree of the occupant for a plurality of frame images included in the video based on the video of the occupant; a histogram creation unit that creates a histogram of the eyelid opening degree when the occupant's eyes are open based on the calculated eyelid opening degree; a histogram evaluation unit that evaluates whether the generated histogram includes erroneous detection data by analyzing the generated histogram and outputs the evaluation result as a histogram evaluation result; an individual feature amount calculation unit that calculates an individual feature amount related to the eyelid opening degree of the occupant based on the histogram in accordance with the output histogram evaluation result; Equipped with the eyelid opening degree is a flattening ratio obtained by dividing the distance between a straight line connecting the inner corner and outer corner of the occupant's eye and the top of the upper eyelid by the distance between the straight line connecting the inner corner and outer corner of the eye, the histogram evaluation unit evaluates that the created histogram includes the erroneous detection data when two peaks exist in the histogram; When it is evaluated that the created histogram includes the erroneous detection data, if the calculated eyelid opening degree has a peak that is larger than a most frequent modal eyelid opening degree, the personal feature calculation unit calculates the most frequent eyelid opening degree as the personal feature of the occupant, and when the calculated eyelid opening degree has a peak that is smaller than the most frequent eyelid opening degree, the personal feature calculation unit calculates the eyelid opening degree of the peak that is smaller than the most frequent eyelid opening degree as the personal feature of the occupant. Eyelid opening detection device.

2. the histogram creation unit creates the histogram when it is determined that the vehicle is traveling based on the vehicle speed, steering wheel angle, or shift information of the vehicle acquired by a vehicle information acquisition device.

2. The eyelid opening degree detecting device according to claim 1.

3. the histogram evaluation unit evaluates that the created histogram does not include false detection data when only one peak exists in the histogram; When it is determined that the created histogram does not include erroneous detection data, the personal characteristic calculation unit calculates a most frequent eyelid opening degree among the calculated eyelid opening degrees as the personal characteristic of the occupant.

2. The eyelid opening degree detecting device according to claim 1.

4. the histogram evaluation unit evaluates that the created histogram does not include false detection data when only one peak exists in the histogram; When it is determined that the created histogram does not include erroneous detection data, the personal characteristic calculation unit calculates a most frequent eyelid opening degree among the calculated eyelid opening degrees as the personal characteristic of the occupant.

3. The eyelid opening detection device according to claim 2.

5. an interior image acquisition unit that acquires an image of an occupant inside the vehicle; an eyelid opening degree calculation unit that calculates an eyelid opening degree indicating an eye opening degree of the occupant for a plurality of frame images included in the video based on the video of the occupant; a histogram creation unit that creates a histogram of the eyelid opening degree when the occupant's eyes are open based on the calculated eyelid opening degree; a histogram evaluation unit that evaluates whether the generated histogram includes erroneous detection data by analyzing the generated histogram and outputs the evaluation result as a histogram evaluation result; an individual feature amount calculation unit that calculates an individual feature amount related to the eyelid opening degree of the occupant based on the histogram in accordance with the output histogram evaluation result; Equipped with the histogram evaluation unit evaluates that the created histogram includes false detection data when two peaks exist in the histogram; When it is evaluated that the created histogram contains erroneous detection data, if the calculated eyelid opening degree has a peak that is larger than a most frequent eyelid opening degree, the personal feature calculation unit calculates the most frequent eyelid opening degree as the personal feature of the occupant, and when the calculated eyelid opening degree has a peak that is smaller than the most frequent eyelid opening degree, the personal feature calculation unit calculates the eyelid opening degree of the peak that is smaller than the most frequent eyelid opening degree as the personal feature of the occupant. Eyelid opening detection device.

6. An eyelid opening degree detection device according to any one of claims 1 to 5, a drowsiness determination device that determines drowsiness of the occupant based on the calculated personal characteristic amount of the occupant; A drowsiness determination system comprising:

7. An eyelid opening degree detection method performed by an eyelid opening degree detection device including an in-vehicle image acquisition unit, an eyelid opening degree calculation unit, a histogram creation unit, a histogram evaluation unit, and an individual feature amount calculation unit, a step in which the vehicle interior image acquisition unit acquires an image of an occupant inside the vehicle; the eyelid opening degree calculation unit calculating, based on the video of the occupant, an eyelid opening degree indicating an eye opening degree of the occupant for a plurality of frame images included in the video; a step in which the histogram creation unit creates a histogram of the eyelid opening degree when the occupant's eyes are open, based on the calculated eyelid opening degree; the histogram evaluation unit evaluating whether the generated histogram includes erroneous detection data by analyzing the generated histogram, and outputting the evaluation result as a histogram evaluation result; a step in which the personal feature calculation unit calculates a personal feature related to an eyelid opening degree of the occupant based on the histogram in accordance with the output histogram evaluation result; Equipped with the eyelid opening degree is an inter-lid distance between the upper and lower eyelids of the occupant's eyes, the histogram evaluation unit evaluates that the created histogram includes false detection data when two peaks exist in the histogram; When it is evaluated that the created histogram contains erroneous detection data, if the calculated eyelid opening degree has a peak that is larger than a most frequent eyelid opening degree, the personal feature calculation unit calculates the most frequent eyelid opening degree as the personal feature of the occupant, and when the calculated eyelid opening degree has a peak that is smaller than the most frequent eyelid opening degree, the personal feature calculation unit calculates the eyelid opening degree of the peak that is smaller than the most frequent eyelid opening degree as the personal feature of the occupant. Eyelid opening detection method.

Images