Driver fatigue state detection method and system based on machine vision

Abstract

The invention relates to the field of fatigue detection, in particular to a driver fatigue state detection method and system based on machine vision. The method comprises the following steps: acquiring a multi-frame preprocessed facial image of a driver, and marking key points; any sampling moment is used as a target frame, the target frame is used as a midpoint, a preset time period is used as a length to construct a time frame, sampling moments except the target frame in the time frame are used as reference frames, and the initial fatigue degree of the face image of the target frame is calculated according to the displacement amount of key points in the reference frame and the target frame in the time frame; and calculating eye state evaluation according to the fluctuation of the Euclidean distance between the key points in the time frame and the initial fatigue degree, and generating and sending an alarm signal in response to the fact that the eye state evaluation is greater than a preset fatigue threshold value so as to complete fatigue state detection. According to the technical scheme, the precision of the fatigue state detection result can be improved.

Description

Technical Field

[0001] The present invention relates to the field of fatigue detection. More specifically, the present invention relates to a method and system for detecting a driver's fatigue state based on machine vision. Background Art

[0002] With the high attention paid to road safety in modern society, the problem of driver fatigue driving has become an important hidden danger of traffic accidents. Research shows that fatigue driving is one of the main causes of traffic accidents. Especially during long - term driving and night driving, drivers are prone to phenomena such as inattentiveness and slow reaction, which can lead to traffic accidents. Therefore, how to effectively monitor and warn the fatigue state of drivers has become a research hotspot in academia and industry.

[0003] In recent years, with the development of computer vision and deep - learning technologies, fatigue detection methods based on machine vision have gradually emerged. This method uses visual devices such as cameras to obtain facial images or eye images of drivers, and combines image - processing and deep - learning technologies to monitor the driver's state in real - time. This method can not only eliminate the trouble of wearing devices in traditional methods, but also has higher accuracy and robustness because visual information is not easily interfered by environmental factors.

[0004] The Chinese patent application document with the publication number CN114332829A discloses a method for detecting driver fatigue based on edge devices. By segmenting the target - area image, the contours of the eyes and mouth are obtained; the relevant values of the eyes and mouth are calculated to judge the states of the eyes and mouth; according to the adaptive fatigue - determination rule and the states of the eyes and mouth, it is determined whether the driver is fatigued.

[0005] However, the fatigue state is a long - term and dynamically changing process, and its manifestation does not occur instantaneously, but gradually intensifies over time. Therefore, judging the fatigue state only by analyzing the relevant values of the eyes and mouth in a small number of frames ignores the dynamic characteristics of this process, and there will be problems with inaccurate fatigue - state detection results. Summary of the Invention

[0006] To solve the problem of inaccurate fatigue - state detection results, the present invention proposes a method and system for detecting a driver's fatigue state based on machine vision.

[0007] In a first aspect, the present invention discloses a method for detecting a driver's fatigue state based on machine vision, including: obtaining multiple preprocessed facial images of the driver and annotating key points; taking any sampling moment as a target frame, constructing a time frame with the target frame as the midpoint and a preset time period as the length, taking the sampling moments other than the target frame in the time frame as reference frames, and calculating the initial fatigue degree of the facial image of the target frame according to the displacement of the key points between the reference frames and the target frame within the time frame; calculating an eye state assessment based on the fluctuation of the Euclidean distance between the key points within the time frame and the initial fatigue degree, and generating and sending an alarm signal in response to the eye state assessment being greater than a preset fatigue threshold to complete the fatigue state detection.

[0008] By analyzing the position changes of facial feature points in real time, the physiological reactions of fatigue can be detected more accurately, providing early fatigue warnings for drivers and avoiding safety accidents caused by fatigue, thereby effectively improving driving safety and the overall safety of road driving.

[0009] Preferably, obtaining the preprocessed facial image includes: using an industrial camera to capture a facial video for a preset time period, decoding the facial video into a continuous frame of facial original images using OpenCV, removing the noise of each frame of facial original image using an adaptive filtering algorithm, and enhancing the details of the facial original image through a contrast enhancement algorithm; obtaining a facial monitoring frame in each frame of facial original image through the RetinaFace facial detection algorithm, and cropping according to the position of the facial detection frame to obtain a facial image containing only the driver's face.

[0010] By eliminating noise and enhancing image details, the facial features become more obvious, thus providing more stable and clear data support for facial key point positioning, and providing a strong guarantee for the accuracy and real-time performance of the fatigue detection system.

[0011] Preferably, the key points include: the key point at the left corner of the left eye, the key point at the right corner of the left eye, the key point directly above the left eye ball, the key point directly below the left eye ball, the key point of the left eye pupil, the key point at the left corner of the right eye, the key point at the right corner of the right eye, the key point directly above the right eye ball, the key point directly below the right eye ball, and the key point of the right eye pupil.

[0012] Preferably, the initial fatigue degree satisfies the relational expression: , represents the initial fatigue degree of the facial image of the target frame , represents the reference frame in the facial image of the left eye pupil key point and the target frame in the facial image of the displacement between the left eye pupil key points, represents the reference frame in the facial image of the left eye pupil key point and the target frame The displacement amount between the left eye pupil key points in the facial image of denotes the reference frame The displacement amount between the right eye pupil key point in the facial image of and the right eye pupil key point in the facial image of the target frame denotes the reference frame The displacement amount between the right eye pupil key point in the facial image of and the right eye pupil key point in the facial image of the target frame denotes the time frame length denotes the normalization function.

[0013] By comparing the eye displacements between the target frame and multiple reference frames, it can effectively reflect the physiological changes of the eyes at different time points, and then identify the signs of fatigue. Specifically, by calculating the changes in the positions of the left and right eye pupils, the minute movement changes of the eyes in the fatigue state can be captured, which are usually caused by long-term concentration or decreased attention.

[0014] Preferably, obtaining the eye state assessment includes: constructing the first distance sequence, the second distance sequence, the third distance sequence, and the fourth distance sequence of the time frame; the eye state assessment satisfies the relational expression: , denotes the target frame The eye state assessment of the facial image denotes variance denotes the first distance sequence denotes the second distance sequence denotes the third distance sequence denotes the fourth distance sequence denotes the canthus weight denotes the eyeball weight denotes the fatigue weight denotes the target frame The initial fatigue degree of the facial image denotes the normalization function.

[0015] By comprehensively considering the displacement changes of the canthus and eyeball key points, as well as the fatigue degree assessment, the opening and closing state of the eyes can be evaluated more accurately, thus providing more efficient and accurate results for fatigue monitoring. Specifically, by calculating the key point distances of the left and right canthi, the upper and lower eyeballs, and analyzing the changes of these distances within the time frame, the minute changes of the eyes can be captured, reflecting the dynamic process of the eyes opening and closing. The displacement changes of the canthus have a greater impact on the opening and closing state of the eyes, while the up and down displacement of the eyeballs can more sensitively detect the fatigue perception state of the eyes.

[0016] Preferably, the first distance sequence, second distance sequence, third distance sequence, and fourth distance sequence for constructing the time frame include: for a facial map at any frame sampling moment in the time frame, calculate the first distance between the key point at the left corner of the left eye and the key point at the right corner of the left eye, the second distance between the key point at the left corner of the right eye and the key point at the right corner of the right eye, the third distance between the key point directly above the left eye ball and the key point directly below the left eye ball, and the fourth distance between the key point directly above the right eye ball and the key point directly below the right eye ball; traverse to obtain the first distance, second distance, third distance, and fourth distance at each sampling moment, and construct the first distance sequence, second distance sequence, third distance sequence, and fourth distance sequence of the time frame.

[0017] Preferably, it further includes: taking the facial map with an eye state evaluation greater than a preset fatigue threshold as a candidate map, and for any candidate map, calculate the fifth distance between the key point directly above the marked mouth and the key point directly below the mouth; construct the fifth distance sequence of all candidate maps, and obtain all minimum value points of the fifth distance; calculate the fatigue degree of the candidate map, and the fatigue degree satisfies the relational expression: , represents the fatigue degree of candidate map , represents the fifth distance of candidate map , and respectively represent the maximum value and minimum value of the fifth distance, represents the time difference between the two closest minimum value points of the fifth distance on the left and right of candidate map , represents the eye state evaluation of candidate map ; complete the fatigue state detection according to the fatigue degree.

[0018] It can effectively capture the minute facial changes caused by fatigue, thereby timely identifying the fatigue situation of the driver, reducing the safety risks brought by fatigue driving, enhancing the reliability of the real-time warning system, and ensuring driving safety.

[0019] Preferably, the fatigue state detection according to the fatigue degree includes: in response to the fatigue degree being within the first threshold range, it is considered that the driver at the sampling moment corresponding to the candidate map is in a state of mild fatigue; in response to the fatigue degree being within the second threshold range, it is considered that the driver at the sampling moment corresponding to the candidate map is in a state of moderate fatigue; in response to the fatigue degree being within the third threshold range, it is considered that the driver at the sampling moment corresponding to the candidate map is in a state of severe fatigue.

[0020] In a second aspect, the present invention discloses a driver fatigue state detection system based on machine vision, comprising: a processor; and a memory storing computer instructions which, when run by the processor, cause the system to execute the above-mentioned method for detecting a driver's fatigue state based on machine vision.

[0021] Advantages of the present invention: Through the processing and analysis of multiple frames of facial images, combined with the changes in the positions of key points and Euclidean distances, the present invention accurately calculates the initial fatigue degree and eye state of the driver, thereby effectively identifying fatigue signs. When the eye state evaluation exceeds the set threshold, the system promptly issues an alarm signal to avoid potential safety hazards caused by fatigue driving.

[0022] Furthermore, through the analysis of the distances between the key points of the mouth in the candidate images, combined with fatigue degree calculation, it is possible to hierarchically judge the fatigue degree of the driver and achieve accurate fatigue state evaluation. It can not only monitor the driver's state in real time but also dynamically adjust the detection sensitivity, ensuring timely response at different fatigue levels, greatly improving the safety of road driving and the health protection of the driver. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] By referring to the accompanying drawings and reading the following detailed description, the above and other objects, features, and advantages of the exemplary embodiments of the present invention will become readily understood. In the drawings, several embodiments of the present invention are shown by way of example and not limitation, and the same or corresponding reference numerals indicate the same or corresponding parts, wherein: Figure 1 is a flowchart of a method for detecting a driver's fatigue state based on machine vision according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0025] It should be understood that when the claims, specifications, and drawings of the present invention use terms such as "first" and "second", they are only used to distinguish different objects and not to describe a specific order. The terms "comprising" and "including" used in the specifications and claims of the present invention indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0026] The present invention provides a method for detecting the fatigue state of a driver based on machine vision. As Figure 1 shown, a method for detecting the fatigue state of a driver based on machine vision includes steps S1 - S3, which are specifically described below.

[0027] S1, obtain multiple pre - processed facial images of the driver and mark key points.

[0028] In one embodiment, an industrial camera is selected for shooting to ensure that clear facial images can be captured, thereby improving the accuracy of fatigue detection. For this purpose, the camera should be installed at a suitable position in front of the driver, usually above the dashboard, to ensure that the driver's face can be fully covered, so as to effectively obtain relevant data.

[0029] After shooting the facial video within a preset time period by the industrial camera, first use OpenCV to decode the video and convert it into a continuous sequence of raw facial image frames. Then, an adaptive filtering algorithm is used to remove the noise in each frame of the image, improving the image quality. At the same time, a contrast enhancement algorithm is applied to enhance the details of the image, making the facial features more clearly visible.

[0030] Use the RetinaFace face detection algorithm to detect the face monitoring frames in each frame of the raw facial image, and perform image cropping based on the positions of these detection frames, finally obtaining high - quality facial images containing only the driver's face, providing clear basic data for subsequent fatigue detection.

[0031] When marking key points on the face, the LFPW (Labeled Faces in the Wild) dataset is used. The LFPW dataset is an important dataset widely used in research on face key point detection and face feature analysis. Each image in this dataset is marked with 29 face key points, and these key points cover the main features of the face, such as eyes, nose, mouth, and contours, etc.

[0032] The present invention preferably selects the key points at the left - hand side corner of the left eye, the right - hand side corner of the left eye, directly above the center of the left eye, directly below the center of the left eye, the center of the left pupil, the left - hand side corner of the right eye, the right - hand side corner of the right eye, directly above the center of the right eye, directly below the center of the right eye, and the center of the right pupil.

[0033] The eyes are one of the parts of the human body that are most likely to reflect the fatigue state. Because of the close connection between the eyes and the brain, they become key indicators of fatigue signals. When a person is fatigued, the eyes will show a series of obvious physiological and behavioral changes, which are usually closely related to the reduced alertness of the brain and the body's physiological responses.

[0034] S2. Take any sampling moment as the target frame, construct a time frame with the target frame as the midpoint and a preset time period as the length, take the sampling moments other than the target frame in the time frame as reference frames, and calculate the initial fatigue degree of the facial map of the target frame according to the displacement of the key points between the reference frames and the target frame within the time frame.

[0035] In one embodiment, for any facial map, a coordinate system is constructed with the pixel point in the lower left corner as the origin of coordinates, the horizontal right direction as the horizontal axis, and the vertical upward direction as the vertical axis.

[0036] Construct a time frame with the target frame as the midpoint and a preset time period as the length, take the sampling moments other than the target frame in the time frame as reference frames, obtain the position coordinates of all key points in the facial map of the target frame, and similarly obtain the position coordinates of all key points in the facial map of each reference frame.

[0037] The calculation method of the displacement of the key points is the calculation result of the distance between two points. Exemplarily, the coordinate of the left eye pupil key point in the facial map of the reference frame is and the coordinate of the left eye pupil key point in the facial map of the target frame is , and the displacement is the coordinate and the coordinate The distance between them.

[0038] The initial fatigue degree satisfies the relational expression: , represents the initial fatigue degree of the facial map of the target frame , represents the displacement between the left eye pupil key point in the facial map of the reference frame and the left eye pupil key point in the facial map of the target frame , represents the displacement between the left eye pupil key point in the facial map of the reference frame and the left eye pupil key point in the facial map of the target frame , represents the displacement between the right eye pupil key point in the facial map of the reference frame and the right eye pupil key point in the facial map of the target frame , represents the displacement between the right eye pupil key point in the facial map of the reference frame and the right eye pupil key point in the facial map of the target frame , represents the time frame length, represents the normalization function.

[0039] By comparing the eye displacements between the target frame and multiple reference frames, the physiological changes of the eyes at different time points can be effectively reflected, and then the signs of fatigue can be identified. Specifically, by calculating the changes in the pupil positions of the left and right eyes, the minute movement changes of the eyes in the fatigued state can be captured, which are usually caused by long-term concentration or decreased attention.

[0040] When the sense of fatigue increases, the eye movements will change significantly, resulting in an increase in the displacement amount. By normalizing these displacement amounts, a comprehensive fatigue assessment value can be obtained, which can monitor the fatigue level in real time in practical applications.

[0041] S3. Calculate the eye state assessment based on the fluctuations of the Euclidean distances between the key points within the time frame and the initial fatigue degree. In response to the eye state assessment being greater than the preset fatigue threshold, generate and send an alarm signal to complete the fatigue state detection.

[0042] It should be noted that the changes in the key points directly above and below the eyeball can more significantly reflect the eye opening and closing state compared to the changes in the key points at the inner and outer corners of the eyes. This is because the movement of the eyeball is mainly reflected in the displacement in the up and down directions, while the changes at the inner and outer corners of the eyes are more affected by factors such as facial expressions and eyelid shapes, and it is difficult to directly reflect the degree of eye opening and closing.

[0043] Based on this characteristic, by combining the eye key points of different importance and using the changes of these key points in the adjacent images in the single-frame original image, the eye opening and closing state can be evaluated more precisely. Especially when the eyes are fatigued, the drooping of the eyelids and the changes in the eye-opening amplitude are more obvious. Therefore, the displacement of the key points at the upper and lower positions of the eyeball can more sensitively capture whether the eyes are in a closed or nearly closed state, providing a more accurate evaluation index for fatigue detection.

[0044] In one embodiment, for the facial image at any frame sampling moment in the time frame, calculate the first distance between the key point at the left corner of the left eye and the key point at the right corner of the left eye, the second distance between the key point at the left corner of the right eye and the key point at the right corner of the right eye, the third distance between the key point directly above the left eyeball and the key point directly below the left eyeball, and the fourth distance between the key point directly above the right eyeball and the key point directly below the right eyeball; Traverse to obtain the first distance, the second distance, the third distance, and the fourth distance at each sampling moment, and construct the first distance sequence, the second distance sequence, the third distance sequence, and the fourth distance sequence of the time frame.

[0045] The eye state assessment satisfies the relational expression: , represents the eye state assessment of the target frame facial image, represents the variance, represents the first distance sequence, represents the second distance sequence, represents the third distance sequence, represents the fourth distance sequence, represents the canthus weight, represents the eyeball weight, represents the fatigue weight, represents the target frame the initial fatigue degree of the facial image, represents the normalization function.

[0046] The sum of the canthus weight, the eyeball weight, and the fatigue weight is 1. Since the changes of the key points directly above and directly below the eyeball can more significantly reflect the eye opening and closing state than the changes of the canthus key points, the value of the eyeball weight is greater than the value of the canthus weight. Exemplarily, .

[0047] By comprehensively considering the displacement changes of the canthus and eyeball key points and the fatigue degree evaluation, the eye opening and closing state can be evaluated more accurately, thus providing more efficient and accurate results for fatigue monitoring. Specifically, by calculating the key point distances of the left and right canthi and the upper and lower eyeball positions and analyzing the changes of these distances within the time frame, the minute changes of the eyes can be captured, reflecting the dynamic process of eye opening and closing. The displacement change of the canthus has a greater impact on the eye opening and closing state, while the up and down displacement of the eyeball can more sensitively detect the fatigue perception state of the eyes. Combining the variances of these distances and the initial fatigue degree, and by assigning different weight parameters (such as canthus, eyeball, and fatigue weights), the fatigue degree of the eyes can be evaluated more precisely according to different physiological characteristics.

[0048] In response to the eye state evaluation being greater than the preset fatigue threshold, an alarm signal is generated and sent to complete the fatigue state detection.

[0049] In another embodiment, it further includes: if the driver's mouth is not blocked, the fatigue state detection needs to be combined with the mouth key points. The blocking situations include behaviors such as wearing a mask.

[0050] It should be noted that under normal circumstances, when a person is in a non-speaking state, the Euclidean distance between the key points directly above and directly below the mouth is relatively small, close to being closed; while when speaking, the movement of the mouth will cause the distance between these two key points to increase, but it is usually still relatively small.

[0051] However, in a fatigued state, when a person yawns, due to the extreme opening of the mouth, the Euclidean distance between the key points directly above and below the mouth will increase significantly. This is because when yawning, the mouth moves a large amount and the facial muscles relax, causing the lips to open fully, forming a large Euclidean distance. At the same time, this change in the mouth opening state is usually accompanied by a time delay, which is longer than the rapid response during normal speech.

[0052] Specifically, when the Euclidean distance between the key points directly above and below the mouth reaches its maximum value, it often requires a period of change, which is closely related to the physiological response caused by fatigue.

[0053] Use facial images with eye state evaluations greater than the preset fatigue threshold as candidate images. For any candidate image, calculate the fifth distance between the key point directly above the labeled mouth and the key point directly below the mouth; construct a sequence of the fifth distances for all candidate images and obtain all the minimum value points of the fifth distances.

[0054] Calculate the fatigue degree of the candidate image, and the fatigue degree satisfies the relational expression: , represents the fatigue degree of candidate image , represents the fifth distance of candidate image , and represent the maximum and minimum values of the fifth distance respectively, represents the time difference between the two closest minimum value points of the fifth distance on the left and right of candidate image , represents the eye state evaluation of candidate image .

[0055] Complete the detection of the fatigued state based on the fatigue degree. In response to the fatigue degree being within the first threshold range, it is considered that the driver at the sampling moment corresponding to the candidate image is in a mildly fatigued state. At this time, a short-term fatigue driving alert should be initiated to remind the driver to relieve fatigue in a timely manner, such as by taking a break, adjusting the driving posture, or performing short rest activities; in response to the fatigue degree being within the second threshold range, it is considered that the driver at the sampling moment corresponding to the candidate image is in a moderately fatigued state. At this time, a continuous fatigue driving alert should be initiated, and the driver should be prompted to reduce the vehicle speed and activate the vehicle's assisted driving function to assist the driver in controlling the vehicle and ensure driving safety; in response to the fatigue degree being within the third threshold range, it is considered that the driver at the sampling moment corresponding to the candidate image is in a severely fatigued state. At this time, a strong fatigue driving alert should be initiated, and the vehicle's autonomous driving function should be activated to control the vehicle to safely pull over to the side of the road, strongly remind the driver to immediately stop and rest, and recommend parking at a safe location to relieve fatigue.

[0056] Exemplarily, the first threshold range is greater than 0.35 and less than 0.5, the second threshold range is greater than or equal to 0.5 and less than 0.7, and the third threshold range is greater than or equal to 0.7 and less than or equal to 1.

[0057] An embodiment of the present invention also discloses a driver fatigue state detection system based on machine vision, including a processor and a memory. The memory stores computer program instructions, and when the computer program instructions are executed by the processor, a method for detecting a driver's fatigue state based on machine vision according to the present invention is implemented.

[0058] The above system also includes other components well-known to those skilled in the art, such as a communication bus and a communication interface. Their settings and functions are known in the art, so they will not be elaborated here.

[0059] In the present invention, the aforementioned memory can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or component. For example, a computer-readable storage medium can be any suitable magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory (RRAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), an enhanced dynamic random access memory (EDRAM), a high-bandwidth memory (HBM), a hybrid memory cube (HMC), etc., or any other medium that can be used to store the required information and can be accessed by an application program, a module, or both. Any such computer storage medium can be part of the device or accessible or connectable to the device.

[0060] Although this specification has shown and described multiple embodiments of the present invention, it is obvious to those skilled in the art that such embodiments are provided only by way of example. Those skilled in the art will think of many changes, alterations, and alternative ways without departing from the spirit and idea of the present invention. It should be understood that various alternative solutions of the embodiments of the present invention described herein can be adopted in the process of practicing the present invention.

[0061] The above are all preferred embodiments of the present invention. The protection scope of the present invention is not limited by this. Therefore, all equivalent changes made according to the structure, shape, and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for detecting driver fatigue status based on machine vision, characterized in that: include: Obtain the driver's multi-frame pre-processed facial image and mark the key points; Take any sampling moment as the target frame, construct a time frame with the target frame as the midpoint and the preset time period as the length, take the sampling moments other than the target frame in the time frame as the reference frame, and calculate the initial fatigue degree of the target frame face image according to the displacement of the key points in the reference frame and the target frame in the time frame; The eye state assessment is calculated based on the fluctuation of the Euclidean distance between key points in the time frame and the initial fatigue level. In response to the eye state assessment being greater than a preset fatigue threshold, an alarm signal is generated and sent to complete fatigue state detection.

2. The method for detecting driver fatigue status based on machine vision according to claim 1, characterized in that: Acquiring the preprocessed facial image includes: An industrial camera is used to shoot facial videos at a preset time period, and OpenCV is used to decode the facial videos into continuous frames of facial original images. An adaptive filtering algorithm is used to remove noise from each frame of the facial original image, and a contrast enhancement algorithm is used to enhance the details of the facial original image. The facial detection frame in each frame of the original facial image is obtained through the RetinaFace facial detection algorithm, and the facial detection frame is cropped according to the position of the facial detection frame to obtain a facial image containing only the driver's face.

3. The method for detecting driver fatigue status based on machine vision according to claim 1, characterized in that: The key points include: The key points of the left corner of the left eye, the right corner of the left eye, the key points directly above the left eyeball, the key points directly below the left eyeball, the key points of the left eye pupil, the key points of the left corner of the right eye, the right corner of the right eye, the key points directly above the right eyeball, the key points directly below the right eyeball and the key points of the right eye pupil.

4. The method for detecting driver fatigue status based on machine vision according to claim 3, characterized in that: The initial fatigue degree satisfies the relationship: , Indicates the target frame The initial fatigue level of the face graph, Represents the reference frame The left pupil key point and the target frame in the face image The displacement between the left pupil key points in the facial image, Represents the reference frame The left pupil key point and the target frame in the face image The displacement between the left pupil key points in the facial image, Represents the reference frame The right pupil key point and the target frame in the face image The displacement between the key points of the right pupil in the face image, Represents the reference frame The right pupil key point and the target frame in the face image The displacement between the key points of the right pupil in the face image, Indicates the time frame length, Represents the normalization function.

5. The method for detecting driver fatigue status based on machine vision according to claim 3, characterized in that: Obtaining the eye status assessment includes: Constructing a first distance sequence, a second distance sequence, a third distance sequence and a fourth distance sequence of the time frame; Eye status assessment satisfies the relationship: , Indicates the target frame Eye status assessment of facial images, represents the variance, represents the first distance sequence, represents the second distance sequence, represents the third distance sequence, represents the fourth distance sequence, represents the eye corner weight, represents the eye weight, represents the fatigue weight, Indicates the target frame The initial fatigue level of the face graph, Represents the normalization function.

6. The method for detecting driver fatigue status based on machine vision according to claim 5, characterized in that: The first distance sequence, the second distance sequence, the third distance sequence and the fourth distance sequence of constructing the time frame include: For the facial image at any frame sampling moment in the time frame, calculate the first distance between the left eye corner key point and the right eye corner key point of the left eye, the second distance between the left eye corner key point and the right eye corner key point of the right eye, the third distance between the key point directly above the left eyeball and the key point directly below the left eyeball, and the fourth distance between the key point directly above the right eyeball and the key point directly below the right eyeball; The first distance, the second distance, the third distance and the fourth distance of each sampling moment are obtained by traversing, and the first distance sequence, the second distance sequence, the third distance sequence and the fourth distance sequence of the time frame are constructed.

7. The method for detecting driver fatigue status based on machine vision according to claim 1, characterized in that: Also includes: The facial images with eye state evaluation greater than a preset fatigue threshold are taken as candidate images. For any candidate image, the fifth distance between the key point just above the mouth and the key point just below the mouth after annotation is calculated; Construct the fifth distance sequence of all candidate graphs and obtain all the fifth distance minimum points; Calculate the fatigue degree of the candidate graph, and the fatigue degree satisfies the relationship: , Represents the candidate graph of fatigue, Represents the candidate graph The fifth distance, and Respectively represent the maximum and minimum values ​​of the fifth distance, Represents the candidate graph The time difference between the two nearest fifth distance minimum points on the left and right, Represents the candidate graph Assessment of eye status; The fatigue status detection is completed according to the fatigue degree.

8. The method for detecting driver fatigue status based on machine vision according to claim 7, characterized in that: The fatigue state detection according to the fatigue degree comprises: In response to the fatigue level being within the first threshold range, it is considered that the driver at the sampling moment corresponding to the candidate graph is in a state of mild fatigue; In response to the fatigue level being within the second threshold range, it is considered that the driver at the sampling moment corresponding to the candidate graph is in a state of moderate fatigue; In response to the fatigue level being within the third threshold range, it is considered that the driver at the sampling moment corresponding to the candidate graph is in a state of severe fatigue.

9. A driver fatigue status detection system based on machine vision, characterized in that: include: processor; as well as A memory storing computer instructions, wherein when the computer instructions are executed by a processor, the system executes a driver fatigue state detection method based on machine vision according to any one of claims 1 to 8.

Images