Eye-closing detection program, eye-closing alarm device, and eye-closing detection method

Abstract

It suppresses false detections of driver drowsiness and accurately detects drivers with their eyes closed. [Solution] The eye-closing alarm device (1) comprises an information processing device (20) and an inner camera (2). The information processing device (20) includes an eye-closing determination unit (230) that determines whether a person captured in the video data captured by the inner camera (2) has their eyes closed, and a post-processing unit (250) that extracts video data in which there is a relatively high probability that the person has their eyes closed as candidate eye-closing video data, extracts eye-closing determination target data by excluding eye-closing video data candidates that satisfy predetermined exclusion conditions in order to detect false detection of eye-closing, and executes control to issue an alarm for eye-closing based on the eye-closing determination target data.

Claims

1. This is an eye-closing detection program, Computers, An eye-closing determination unit determines whether a subject has their eyes closed based on video data of the subject to be determined to be in an eye-closing state, A program for detecting closed eyes, which functions as a post-processing unit to extract video data in which the subject is determined to have a relatively high probability of having their eyes closed, as candidate video data for closed eyes, and to extract data for which closed eyes are to be detected by excluding from the candidate video data for closed eyes video data that satisfy predetermined exclusion conditions in order to detect false detections of closed eyes.

2. An eye-closing detection program according to claim 1, The post-processing unit is an eye-closing detection program that determines that the subject is in an eye-closing state when the eye-closing determination target data is continuously detected for a predetermined time.

3. An eye-closing detection program according to claim 1, The aforementioned computer, A face detection model, trained using a dataset in which facial landmarks of people captured in video data are annotated, is further configured to function as a face detection unit that inputs video data of the subject and detects the facial landmarks of the subject. The face detection model infers at least one of the following tasks based on the detected landmarks: determining whether the subject is wearing sunglasses, determining whether they are wearing a mask, and determining whether they are covering their face with their hands. The post-processing unit is an eye-closed detection program that extracts eye-closed detection target data by excluding candidate eye-closed video data in which the subject is determined to be wearing sunglasses, wearing a mask, or covering their face with their hands, based on the processing results of the face detection model.

4. An eye-closing detection program according to claim 3, The face detection unit calculates the degree of mouth opening and closing of the subject based on the landmarks detected by the face detection model and passes this information to the post-processing unit. The post-processing unit is an eye-closing detection program that, when it determines that the subject is yawning based on the degree of opening and closing of the mouth, excludes the candidate eye-closing video data and extracts the eye-closing detection target data.

5. An eye-closing detection program according to claim 3, The aforementioned computer, It can also function as a face cropping section that crops a portion of the video data. The face detection unit identifies the face coordinates of the landmark in a two-dimensional planar coordinate system defined in the image based on the video data in which the subject was captured. The face cropping unit generates face image data by identifying and cropping the region containing the landmark from the video data in which the subject was captured, based on the face coordinates. The eye-closing detection unit is an eye-closing detection program that determines whether the subject captured in the facial image data has their eyes closed.

6. An eye-closing detection program according to claim 5, The eye-closing detection unit includes an eye-closing trained model trained using training data that includes a dataset in which images of a person with their eyes closed are annotated as eye-closing images and images of a person with their eyes open are annotated as normal images. The eye-closing detection program inputs the face image data into the eye-closing trained model and outputs a pseudo-eye-closing probability that indicates the likelihood that the subject is in a pseudo-eye-closing state, where it is presumed that the subject has their eyes closed.

7. An eye-closing detection program according to claim 6, The aforementioned computer, The system includes a pre-trained model for distracted viewing, which has been trained using machine learning with training data that includes a dataset in which images of a person looking downwards are treated as distracted viewing images and images of a person's face facing forward are treated as normal images. This pre-trained model for distracted viewing is then input to the aforementioned face image data and further functions as a distracted viewing detection unit that outputs the probability of the subject looking away. The post-processing unit is an eye-blinding detection program that, if the difference between the pseudo-eye-blinding accuracy and the distracted-looking accuracy is greater than or equal to a predetermined accuracy threshold, excludes the candidate eye-blinding video data and extracts the data to be judged as eye-blinding.

8. An eye-closing detection program according to claim 1, The post-processing unit is an eye-closed detection program that acquires speed information, and if it determines that the device is in a stopped state based on the speed information, it excludes the candidate eye-closed video data and extracts the data to be used for eye-closed detection.

9. An eye-closing detection program according to claim 1, The post-processing unit acquires speed information, and if it determines based on the speed information that the vehicle is in a moving state, it executes control to issue an alarm when it detects eye closure based on the eye closure detection target data, and if it determines based on the speed information that the vehicle is in a stopped state, it does not execute control to issue an alarm even if it detects eye closure based on the eye closure detection target data, wherein the post-processing unit acquires speed information, and if it determines based on the speed information that the vehicle is in a moving state, it executes control to issue an alarm.

10. It is an eye-closing alarm device, The system comprises a camera that captures video data and an information processing device that performs eye-closing detection processing based on the video data, The aforementioned information processing device is The aforementioned video data includes an eye-closed detection unit that determines whether the person captured in the video data has their eyes closed, A post-processing unit extracts video data in which the person is relatively likely to have their eyes closed as candidate eye-closed video data, excludes eye-closed video data candidates that satisfy predetermined exclusion conditions to detect false detections of eye-closed eyes, extracts data to be determined as eye-closed, and executes control to issue an alarm for eye-closed eyes based on the data to be determined as eye-closed. An eye-closing alarm device equipped with [a specific feature].

11. An eye-closing alarm device according to claim 10, The aforementioned camera is configured separately from the information processing device and is an eye-closing alarm device.

12. An eye-closing alarm device according to claim 10, The aforementioned camera is installed in a position and orientation to capture images of the driver of the vehicle from the front upper side inside the vehicle where the information processing device is mounted, and is part of the eye-blinding warning device.

13. An eye-closing alarm device according to claim 10, The aforementioned information processing device is an eye-closing alarm device installed in a drive recorder.

14. An eye-closing alarm device according to claim 10, An eye-closing alarm device comprising the aforementioned information processing device and the aforementioned camera as an integrated unit.

15. A method for detecting closed eyes, Computers A blindness detection step that determines whether the person captured in the video data has their eyes closed, A method for detecting closed eyes, comprising: a post-processing step of extracting video data in which the person is relatively likely to have their eyes closed as candidate video data of closed eyes; and from the candidate video data of closed eyes, excluding those video data of closed eyes that satisfy predetermined exclusion conditions in order to detect false detection of closed eyes, in order to extract data to be judged as closed eyes.

Images