A driver fatigue state detection method and system based on machine vision

Abstract

The present invention relates to the field of fatigue detection, and more specifically, to a method and system for detecting driver fatigue status based on machine vision. The method comprises: obtaining a multi-frame pre-processed facial image of the driver and marking key points; taking any sampling moment as the target frame, constructing a time frame with the target frame as the midpoint and a preset period as the length, taking the sampling moments other than the target frame in the time frame as the reference frame, and calculating the initial fatigue of the target frame facial image based on the displacement of the key points in the reference frame and the target frame in the time frame; calculating the eye state evaluation based on the fluctuation of the Euclidean distance between the key points in the time frame and the initial fatigue, and generating and sending an alarm signal in response to the eye state evaluation being greater than the preset fatigue threshold, thereby completing the fatigue state detection. The technical solution of the present invention can improve the accuracy of the fatigue state detection results.

Description

Technical Field

[0001] The present invention relates to the field of fatigue detection, and more particularly to a method and system for detecting driver fatigue based on machine vision. Background Art

[0002] With modern society placing a high priority on road safety, driver fatigue has become a significant risk factor for traffic accidents. Research has shown that fatigue is a major cause of traffic accidents, particularly during long driving periods and at night, when drivers can become distracted and unresponsive, leading to accidents. Therefore, effective monitoring and early warning of driver fatigue has become a hot topic in both academia and industry.

[0003] In recent years, with the advancement of computer vision and deep learning technologies, machine vision-based fatigue detection methods have gradually emerged. This method uses visual devices such as cameras to capture facial or eye images of the driver, combining image processing and deep learning techniques to monitor the driver's condition in real time. This approach not only eliminates the need for wearable equipment in traditional methods, but also offers greater accuracy and robustness because visual information is less susceptible to environmental interference.

[0004] The existing Chinese patent application document with publication number CN114332829A discloses a driver fatigue detection method based on edge devices. By segmenting the target area image, the contours of the eyes and mouth are obtained; the correlation values of the eyes and mouth are calculated to determine the status of the eyes and mouth; and based on the adaptive fatigue judgment rules and the status of the eyes and mouth, whether the driver is driving fatigued is determined.

[0005] However, fatigue is a long-term, dynamically changing process, and its manifestations do not occur instantaneously but gradually intensify over time. Therefore, judging fatigue by analyzing eye and mouth values based on only a few frames ignores the dynamic nature of this process and can lead to inaccurate fatigue detection results. Summary of the Invention

[0006] In order to solve the problem of inaccurate fatigue state detection results, the present invention proposes a driver fatigue state detection method and system based on machine vision.

[0007] In a first aspect, the present invention discloses a method for detecting driver fatigue status based on machine vision, comprising: obtaining a multi-frame preprocessed facial image of the driver and marking key points; taking any sampling moment as a target frame, constructing a time frame with the target frame as the midpoint and a preset time period as the length, taking the sampling moments other than the target frame in the time frame as reference frames, and calculating the initial fatigue of the facial image of the target frame according to the displacement between the reference frame and the key points in the target frame in the time frame; calculating an eye state assessment according to the fluctuation of the Euclidean distance between the key points in the time frame and the initial fatigue, and generating and sending an alarm signal in response to the eye state assessment being greater than a preset fatigue threshold, thereby completing fatigue state detection.

[0008] By analyzing the position changes of facial feature points in real time, the physiological reactions of fatigue can be detected more accurately, providing drivers with early fatigue warnings and avoiding safety accidents caused by fatigue, thereby effectively improving driving safety and overall road safety.

[0009] Preferably, obtaining the preprocessed facial image includes: using an industrial camera to shoot facial video of a preset time period, using OpenCV to decode the facial video into continuous frames of facial original images, using an adaptive filtering algorithm to remove noise from each frame of the facial original image, and enhancing the details of the facial original image through a contrast enhancement algorithm; obtaining a facial monitoring frame in each frame of the facial original image through a RetinaFace facial detection algorithm, and cropping according to the position of the facial detection frame to obtain a facial image containing only the driver's face.

[0010] By eliminating noise and enhancing image details, facial features are made more obvious, providing more stable and clear data support for the positioning of facial key points, and providing strong guarantees for the accuracy and real-time performance of the fatigue detection system.

[0011] Preferably, the key points include: the left corner of the left eye key point, the right corner of the left eye key point, the key point directly above the left eyeball, the key point directly below the left eyeball, the left eye pupil key point, the left corner of the right eye key point, the right corner of the right eye key point, the key point directly above the right eyeball, the key point directly below the right eyeball and the right eye pupil key point.

[0012] Preferably, the initial fatigue degree satisfies the relationship:

[0013] , Indicates the target frame Initial fatigue level of the facial image, Represents a reference frame The left pupil key point and target frame in the face image The displacement between the left pupil key points in the facial image, Represents a reference frame The left pupil key point and target frame in the face image The displacement between the left pupil key points in the facial image, Represents a reference frame The right pupil key point in the face image and the target frame The displacement between the right pupil key points in the facial image, Represents a reference frame The right pupil key point in the face image and the target frame The displacement between the right pupil key points in the facial image, Indicates the time frame length, Represents the normalization function.

[0014] By comparing eye displacement between the target frame and multiple reference frames, it is possible to effectively reflect physiological changes in the eyes at different time points and identify signs of fatigue. Specifically, by calculating the changes in pupil position of the left and right eyes, it is possible to capture subtle changes in eye movement when fatigue occurs, which are usually caused by prolonged concentration or decreased attention.

[0015] Preferably, obtaining the eye state evaluation includes: constructing a first distance sequence, a second distance sequence, a third distance sequence, and a fourth distance sequence of a time frame; and the eye state evaluation satisfies the relationship:

[0016] , Indicates the target frame Eye status assessment of facial images, represents the variance, represents the first distance sequence, represents the second distance sequence, represents the third distance sequence, represents the fourth distance sequence, represents the eye corner weight, represents the eyeball weight, represents the fatigue weight, Indicates the target frame Initial fatigue level of the facial image, Represents the normalization function.

[0017] By comprehensively considering the displacement changes of the corners of the eyes and key points of the eyeballs, as well as fatigue assessment, the open and closed state of the eyes can be assessed more accurately, thus providing more efficient and accurate results for fatigue monitoring. Specifically, by calculating the distances between the key points of the left and right corners of the eyes and the upper and lower eyeball positions, and analyzing the changes in these distances within the time frame, it is possible to capture subtle changes in the eyes and reflect the dynamic process of eye opening and closing. The displacement changes of the corners of the eyes have a greater impact on the open and closed state of the eyes, while the upper and lower displacement of the eyeballs can more sensitively detect the fatigue perception state of the eyes.

[0018] Preferably, constructing the first distance sequence, the second distance sequence, the third distance sequence, and the fourth distance sequence of the time frame includes: for the facial image at any frame sampling moment in the time frame, calculating the first distance between the left corner key point of the left eye and the right corner key point of the left eye, the second distance between the left corner key point of the right eye and the right corner key point of the right eye, the third distance between the key point directly above the left eyeball and the key point directly below the left eyeball, and the fourth distance between the key point directly above the right eyeball and the key point directly below the right eyeball; traversing to obtain the first distance, the second distance, the third distance, and the fourth distance at each sampling moment, and constructing the first distance sequence, the second distance sequence, the third distance sequence, and the fourth distance sequence of the time frame.

[0019] Preferably, the method further includes: selecting facial images with eye state evaluation greater than a preset fatigue threshold as candidate images, and for any candidate image, calculating the fifth distance between the annotated key point directly above the mouth and the key point directly below the mouth; constructing a fifth distance sequence for all candidate images, and obtaining all fifth distance minimum points; and calculating the fatigue degree of the candidate image, where the fatigue degree satisfies the relationship:

[0020] , Represents the candidate graph Fatigue, Represents the candidate graph The fifth distance, and Represent the maximum and minimum values of the fifth distance respectively, Represents the candidate graph The time difference between the two nearest fifth distance minimum points on the left and right, Represents the candidate graph Eye status assessment; complete fatigue status detection according to fatigue level.

[0021] It can effectively capture subtle facial changes caused by fatigue, thereby timely identifying the driver's fatigue status, reducing the safety risks brought by fatigue driving, enhancing the reliability of the real-time warning system, and ensuring driving safety.

[0022] Preferably, the fatigue state detection based on fatigue level includes: in response to the fatigue level being within a first threshold range, considering that the driver at the sampling moment corresponding to the candidate image is in a state of mild fatigue; in response to the fatigue level being within a second threshold range, considering that the driver at the sampling moment corresponding to the candidate image is in a state of moderate fatigue; in response to the fatigue level being within a third threshold range, considering that the driver at the sampling moment corresponding to the candidate image is in a state of severe fatigue.

[0023] In a second aspect, the present invention discloses a driver fatigue status detection system based on machine vision, comprising: a processor; and a memory, wherein the memory stores computer instructions. When the computer instructions are executed by the processor, the system executes the above-mentioned driver fatigue status detection method based on machine vision.

[0024] Beneficial effects of the present invention:

[0025] This invention processes and analyzes multiple facial images, combining key point locations and Euclidean distances to accurately calculate a driver's initial fatigue level and eye state, effectively identifying signs of fatigue. If the eye state assessment exceeds a set threshold, the system promptly issues an alarm, mitigating safety hazards associated with fatigued driving.

[0026] Furthermore, by analyzing the distance between key points of the mouth in the candidate image and combining it with fatigue calculation, the driver's fatigue level can be graded and accurately assessed. This system not only monitors the driver's condition in real time but also dynamically adjusts detection sensitivity, ensuring timely response to varying fatigue levels, significantly improving road safety and protecting driver health. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an illustrative and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0028] Figure 1 This is a flowchart of a method for detecting driver fatigue status based on machine vision in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work shall fall within the scope of protection of the present invention.

[0030] It should be understood that when the terms "first," "second," and the like are used in the claims, description, and drawings of the present invention, they are merely used to distinguish between different objects, rather than to describe a specific order. The terms "comprise" and "comprising" used in the description and claims of the present invention indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0031] The present invention provides a method for detecting driver fatigue based on machine vision. Figure 1 As shown, a method for detecting driver fatigue status based on machine vision includes steps S1 to S3, which are described in detail below.

[0032] S1, obtain the driver's multi-frame pre-processed facial image and mark the key points.

[0033] In one embodiment, an industrial camera is used to capture clear facial images, thereby improving the accuracy of fatigue detection. To this end, the camera should be installed in a suitable position in front of the driver, typically above the dashboard, to ensure full coverage of the driver's face and effectively capture relevant data.

[0034] After capturing facial video for a preset period using an industrial camera, the video is first decoded using OpenCV, converting it into a continuous stream of raw facial frames. An adaptive filtering algorithm is then used to remove noise from each frame, improving image quality. A contrast enhancement algorithm is also applied to enhance image detail, making facial features more clearly visible.

[0035] The RetinaFace facial detection algorithm is used to detect facial monitoring frames in each frame of the original facial image. The image is cropped according to the position of these detection frames, and finally a high-quality facial image containing only the driver's face is obtained, providing clear basic data for subsequent fatigue detection.

[0036] The Labeled Faces in the Wild (LFPW) dataset is used to annotate facial keypoints. LFPW is a widely used dataset for facial keypoint detection and facial feature analysis. Each image in this dataset is annotated with 29 facial keypoints, covering key facial features such as the eyes, nose, mouth, and outline.

[0037] The present invention preferably selects the key points of the left corner of the left eye, the right corner of the left eye, the key points directly above the left eyeball, the key points directly below the left eyeball, the key points of the left eye pupil, the key points of the left corner of the right eye, the right corner of the right eye, the key points directly above the right eyeball, the key points directly below the right eyeball and the key points of the right eye pupil.

[0038] The eyes are one of the most common areas of the body that can easily reflect fatigue. Their close connection to the brain makes them a key indicator of fatigue. When fatigued, the eyes exhibit a series of distinct physiological and behavioral changes, which are often closely linked to decreased brain alertness and physiological responses in the body.

[0039] S2: Take any sampling moment as the target frame, construct a time frame with the target frame as the midpoint and the preset time period as the length, take the sampling moments other than the target frame in the time frame as the reference frame, and calculate the initial fatigue degree of the target frame face image based on the displacement of the key points in the reference frame and the target frame in the time frame.

[0040] In one embodiment, for any facial image, a coordinate system is constructed with the pixel point in the lower left corner as the coordinate origin, the horizontal right direction as the horizontal axis, and the vertical upward direction as the vertical axis.

[0041] A time frame is constructed with the target frame as the midpoint and the preset time period as the length. The sampling moments in the time frame except the target frame are used as reference frames to obtain the position coordinates of all key points in the facial image of the target frame. Similarly, the position coordinates of all key points in the facial image of each reference frame are obtained.

[0042] The calculation method of the displacement of the key point is the distance calculation result between the two points. For example, the coordinates of the left pupil key point in the facial image of the reference frame are , the coordinates of the left pupil key point in the face image of the target frame are , the displacement is the coordinate and coordinates The distance between them.

[0043] The initial fatigue degree satisfies the relationship:

[0044] , Indicates the target frame Initial fatigue level of the facial image, Represents a reference frame The left pupil key point and target frame in the face image The displacement between the left pupil key points in the facial image, Represents a reference frame The left pupil key point and target frame in the face image The displacement between the left pupil key points in the facial image, Represents a reference frame The right pupil key point in the face image and the target frame The displacement between the right pupil key points in the facial image, Represents a reference frame The right pupil key point in the face image and the target frame The displacement between the right pupil key points in the facial image, Indicates the time frame length, Represents the normalization function.

[0045] By comparing eye displacement between the target frame and multiple reference frames, it is possible to effectively reflect physiological changes in the eyes at different time points and identify signs of fatigue. Specifically, by calculating the changes in pupil position of the left and right eyes, it is possible to capture subtle changes in eye movement when fatigue occurs, which are usually caused by prolonged concentration or decreased attention.

[0046] As fatigue increases, eye movements change significantly, increasing the amount of displacement. By normalizing these displacements, a comprehensive fatigue assessment can be obtained, enabling real-time monitoring of fatigue levels in practical applications.

[0047] S3, calculate the eye state assessment based on the fluctuation of the Euclidean distance between key points in the time frame and the initial fatigue level, and in response to the eye state assessment being greater than a preset fatigue threshold, generate and send an alarm signal to complete the fatigue state detection.

[0048] It's important to note that changes in key points directly above and below the eyeballs more significantly reflect the eye's openness than changes in key points at the inner and outer corners. This is because eye movement primarily involves vertical displacement, while changes in the inner and outer corners are more influenced by factors like facial expression and eyelid shape, making them less likely to directly reflect the degree of eye openness.

[0049] Based on this characteristic, by combining eye keypoints of varying importance and leveraging the variations of these keypoints in adjacent frames within a single original image, the eye's open / closed state can be more accurately assessed. In particular, when eyes become fatigued, changes in eyelid droop and eye opening are more pronounced. Therefore, the displacement of keypoints in the upper and lower eye positions can more sensitively capture whether the eyes are closed or nearly closed, providing a more accurate assessment metric for fatigue detection.

[0050] In one embodiment, for a facial image at any frame sampling moment in a time frame, a first distance between a left eye left corner key point and a right eye right corner key point of a left eye, a second distance between a left eye left corner key point and a right eye right corner key point of a right eye, a third distance between a key point directly above the left eyeball and a key point directly below the left eyeball, and a fourth distance between a key point directly above the right eyeball and a key point directly below the right eyeball are calculated.

[0051] The first distance, second distance, third distance and fourth distance of each sampling moment are obtained by traversal, and the first distance sequence, second distance sequence, third distance sequence and fourth distance sequence of the time frame are constructed.

[0052] Eye status assessment satisfies the relationship:

[0053] , Indicates the target frame Eye status assessment of facial images, represents the variance, represents the first distance sequence, represents the second distance sequence, represents the third distance sequence, represents the fourth distance sequence, represents the eye corner weight, represents the eyeball weight, represents the fatigue weight, Indicates the target frame Initial fatigue level of the facial image, Represents the normalization function.

[0054] The sum of the corner weight, eyeball weight, and fatigue weight is 1. Because the changes in the key points just above the eyeball and just below the eyeball can more significantly reflect the eye opening and closing status than the changes in the corner key points, the eyeball weight value is greater than the corner weight value. For example, .

[0055] By comprehensively considering the displacement changes of key points of the corners of the eyes and the eyeball, as well as fatigue assessment, the open and closed state of the eyes can be assessed more accurately, thus providing more efficient and accurate results for fatigue monitoring. Specifically, by calculating the distances between the key points of the left and right corners of the eyes and the upper and lower eyeball positions, and analyzing the changes in these distances within the time frame, it is possible to capture subtle changes in the eyes and reflect the dynamic process of eye opening and closing. Displacement changes in the corners of the eyes have a greater impact on the open and closed state of the eyes, while the upper and lower displacement of the eyeballs can more sensitively detect the fatigue perception state of the eyes. Combining the variance of these distances and the initial fatigue level, by assigning different weight parameters (such as corner of the eye, eyeball, and fatigue weights), the degree of eye fatigue can be more accurately assessed based on different physiological characteristics.

[0056] In response to the eye state assessment being greater than a preset fatigue threshold, an alarm signal is generated and sent to complete fatigue state detection.

[0057] In another embodiment, if the driver's mouth is not blocked, fatigue status detection is performed based on key points of the mouth. Blockages include wearing a mask, etc.

[0058] It should be noted that under normal circumstances, when a person is not speaking, the Euclidean distance between the key points directly above and below the mouth is small, close to closed; while when speaking, the movement of the mouth will cause the distance between these two key points to increase, but it is usually still small.

[0059] However, when a person yawns in a fatigued state, the Euclidean distance between key points directly above and below the mouth increases significantly due to the extreme opening of the mouth. This is because when yawning, the mouth moves widely and the facial muscles relax, causing the lips to fully open, resulting in a larger Euclidean distance. Furthermore, this change in mouth opening is often accompanied by a time delay, which is longer than the rapid response during normal speech.

[0060] Specifically, when the Euclidean distance between the key points directly above and below the mouth reaches its maximum value, it often takes a period of time to change, which is closely related to the physiological response caused by fatigue.

[0061] Facial images with eye state assessments greater than a preset fatigue threshold are selected as candidate images. For any candidate image, the fifth distance between the annotated key point directly above the mouth and the key point directly below the mouth is calculated; a fifth distance sequence of all candidate images is constructed, and all fifth distance minimum points are obtained.

[0062] Calculate the fatigue of the candidate graph, and the fatigue satisfies the relationship:

[0063] , Represents the candidate graph Fatigue, Represents the candidate graph The fifth distance, and Represent the maximum and minimum values of the fifth distance respectively, Represents the candidate graph The time difference between the two nearest fifth distance minimum points on the left and right, Represents the candidate graph Assessment of eye status.

[0064] The fatigue state detection is completed according to the fatigue level. In response to the fatigue level being within the first threshold range, it is considered that the driver at the sampling moment corresponding to the candidate image is in a state of mild fatigue. At this time, a short fatigue driving alarm should be initiated to remind the driver to relieve fatigue in time, such as by taking a rest, adjusting the driving posture or taking a short rest activity; in response to the fatigue level being within the second threshold range, it is considered that the driver at the sampling moment corresponding to the candidate image is in a state of moderate fatigue. At this time, a continuous fatigue driving alarm should be initiated, and the driver should be prompted to reduce the speed and activate the vehicle's assisted driving function to assist the driver in controlling the vehicle and ensure driving safety; in response to the fatigue level being within the third threshold range, it is considered that the driver at the sampling moment corresponding to the candidate image is in a state of severe fatigue. At this time, a strong fatigue driving alarm should be initiated, and the vehicle's automatic driving function should be activated to control the vehicle to park safely on the roadside. The driver should be strongly reminded to stop and rest immediately, and it is recommended to stop in a safe place to relieve fatigue.

[0065] Exemplarily, the first threshold range is greater than 0.35 and less than 0.5, the second threshold range is greater than or equal to 0.5 and less than 0.7, and the third threshold range is greater than or equal to 0.7 and less than or equal to 1.

[0066] An embodiment of the present invention also discloses a driver fatigue status detection system based on machine vision, including a processor and a memory, wherein the memory stores computer program instructions. When the computer program instructions are executed by the processor, a driver fatigue status detection method based on machine vision according to the present invention is implemented.

[0067] The above system also includes other components well known to those skilled in the art, such as a communication bus and a communication interface. The configuration and functions of these components are known in the art and will not be described in detail here.

[0068] In the present invention, the aforementioned memory may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, a computer-readable storage medium may be any suitable magnetic storage medium or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), or any other medium that can be used to store the required information and can be accessed by an application, module, or both. Any such computer storage medium may be part of, accessible to, or connectable to a device.

[0069] While this specification has shown and described several embodiments of the present invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Numerous modifications, variations, and alternatives will occur to those skilled in the art without departing from the concept and spirit of the present invention. It should be understood that various alternatives to the embodiments of the present invention described herein may be employed in practicing the present invention.

[0070] The above are all preferred embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for detecting driver fatigue based on machine vision, characterized in that: include: Obtain the driver's multi-frame pre-processed facial image and mark the key points; The key points include: The left corner of the left eye, the right corner of the left eye, the key point just above the left eyeball, the key point just below the left eyeball, the left eye pupil key point, the left corner of the right eye, the right corner of the right eye, the key point just above the right eyeball, the key point just below the right eyeball, and the right eye pupil key point; Take any sampling moment as the target frame, construct a time frame with the target frame as the midpoint and the preset period as the length, take the sampling moments other than the target frame in the time frame as the reference frame, and calculate the initial fatigue degree of the target frame face image based on the displacement between the reference frame and the key points in the target frame in the time frame, satisfying the relationship: , Indicates the target frame The initial fatigue level of the facial image, Represents a reference frame The left pupil key point and target frame in the face image The displacement between the left pupil key points in the facial image, Represents a reference frame The left pupil key point and target frame in the face image The displacement between the left pupil key points in the facial image, Represents a reference frame The right pupil key point in the face image and the target frame The displacement between the right pupil key points in the facial image, Represents a reference frame The right pupil key point in the face image and the target frame The displacement between the right pupil key points in the facial image, Indicates the time frame length, represents the normalization function; Eye state assessment is calculated based on the fluctuation of the Euclidean distance between key points in the time frame and the initial fatigue level, including: Constructing a first distance sequence, a second distance sequence, a third distance sequence, and a fourth distance sequence of the time frame; Eye status assessment satisfies the relationship: , Indicates the target frame Eye status assessment of facial images, represents the variance, represents the first distance sequence, represents the second distance sequence, represents the third distance sequence, represents the fourth distance sequence, represents the eye corner weight, represents the eyeball weight, represents the fatigue weight, Indicates the target frame The initial fatigue level of the facial image, represents the normalization function; In response to the eye state assessment being greater than a preset fatigue threshold, an alarm signal is generated and sent to complete fatigue state detection.

2. The method for detecting driver fatigue based on machine vision according to claim 1, characterized in that: Obtaining the preprocessed facial image includes: An industrial camera is used to shoot facial videos for a preset period of time. The facial videos are decoded into continuous frames of facial raw images using OpenCV. An adaptive filtering algorithm is used to remove noise from each frame of the facial raw image. A contrast enhancement algorithm is used to enhance the details of the facial raw image. The facial detection frame in each frame of the original facial image is obtained through the RetinaFace facial detection algorithm, and the facial image containing only the driver's face is cropped according to the position of the facial detection frame.

3. The method for detecting driver fatigue based on machine vision according to claim 1, characterized in that: The first distance sequence, the second distance sequence, the third distance sequence and the fourth distance sequence of constructing the time frame include: For the facial image at any frame sampling moment in the time frame, calculate the first distance between the left eye corner key point and the right eye corner key point of the left eye, the second distance between the left eye corner key point and the right eye corner key point of the right eye, the third distance between the key point directly above the left eyeball and the key point directly below the left eyeball, and the fourth distance between the key point directly above the right eyeball and the key point directly below the right eyeball; The first distance, second distance, third distance and fourth distance of each sampling moment are obtained by traversal, and the first distance sequence, second distance sequence, third distance sequence and fourth distance sequence of the time frame are constructed.

4. The method for detecting driver fatigue based on machine vision according to claim 1, characterized in that: Also includes: The facial images with an eye state assessment greater than a preset fatigue threshold are selected as candidate images. For any candidate image, the fifth distance between the annotated key point just above the mouth and the key point just below the mouth is calculated; Construct the fifth distance sequence of all candidate graphs and obtain all the fifth distance minimum points; Calculate the fatigue of the candidate graph, and the fatigue satisfies the relationship: , Represents the candidate graph Fatigue, Represents the candidate graph The fifth distance, and Represent the maximum and minimum values of the fifth distance respectively, Represents the candidate graph The time difference between the two nearest fifth distance minimum points on the left and right, Represents the candidate graph Assessment of eye status; Complete fatigue status detection based on fatigue level.

5. The method for detecting driver fatigue based on machine vision according to claim 4, characterized in that: The fatigue state detection according to the fatigue degree includes: In response to the fatigue level being within a first threshold range, it is determined that the driver at the sampling moment corresponding to the candidate graph is in a state of mild fatigue; In response to the fatigue level being within a second threshold range, determining that the driver at the sampling moment corresponding to the candidate graph is in a state of moderate fatigue; In response to the fatigue level being within the third threshold range, it is considered that the driver at the sampling moment corresponding to the candidate map is in a state of severe fatigue.

6. A driver fatigue status detection system based on machine vision, characterized in that: include: processor; as well as A memory storing computer instructions, wherein when the computer instructions are executed by a processor, the system executes a driver fatigue state detection method based on machine vision according to any one of claims 1 to 5.

Images