A Driver Fatigue Detection Method and Device

Abstract

The present invention discloses a driver fatigue detection method and device. The method includes: obtaining facial feature data of the current driver; performing truth value statistics on the eyelid distance feature data and the blink feature data in the facial feature data to obtain the normal value of the eyelid distance and the percentage of blink frames of the current driver; processing the eyelid distance feature data and the blink feature data according to the normal value of the eyelid distance to obtain the normalized eyelid distance and the percentage of closed-eye frames; correcting the preset fatigue level division conditions according to the percentage of blink frames to obtain the corrected fatigue level division conditions; performing fatigue level evaluation according to the normalized eyelid distance, the percentage of closed-eye frames, and the corrected fatigue level division conditions, and outputting the fatigue level evaluation result of the current driver. The present invention uses multiple features that eliminate individual differences for fatigue evaluation, can avoid missed detection and false detection, and the evaluation result is more accurate.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a driver fatigue detection method, device, vehicle, electronic equipment and storage medium. Background Art

[0002] With the development of traffic, various traffic accidents are also increasing, of which about 30% are caused by driver fatigue. Therefore, it is of great significance to detect the driver's fatigue level and give targeted warnings to the driver to reduce the occurrence of traffic accidents and maintain the safety of life and property.

[0003] Existing driver fatigue detection technologies can be roughly divided into two categories: contact and non-contact. The contact type mainly obtains relevant parameter indicators of the driver by wearing corresponding sensors, such as heart rate, electromyography, electrodermal, electroencephalogram, respiratory rate and other parameters to judge the driver's fatigue status. The non-contact type mainly judges the driver's fatigue status by statistically analyzing the driver's driving behavior information and visual information. The driving behavior information mainly includes driving trajectory, speed, acceleration, steering wheel status, etc. to judge the driver's fatigue status. The visual information mainly includes the degree of eye opening, the position angle of the mouth and the head, etc. When evaluating eye closing movements, the existing technology mostly evaluates the degree of eye opening based on the eye aspect ratio (EAR). However, the eye aspect ratio EAR of each person in the natural state is different. The same judgment standard cannot adapt to different people, and it is easy to have false detection and missed detection. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the first aspect of the present invention provides a driver fatigue detection method, comprising:

[0005] Acquire facial feature data of the current driver; wherein the facial feature data includes eyelid distance feature data and blink feature data;

[0006] Performing true value statistics on the eyelid distance feature data and the blink feature data to obtain the normal value of the eyelid distance and the percentage of blink frames of the current driver;

[0007] Processing the eyelid distance feature data and the blink feature data according to the eyelid distance normal value to obtain normalized eyelid distance and eye-closing frame number percentage;

[0008] Correcting the preset fatigue level classification condition according to the percentage of blink frames to obtain a corrected fatigue level classification condition;

[0009] Based on the normalized eyelid distance, the percentage of closed-eye frames, and the modified fatigue level classification conditions, perform fatigue level assessment and output the fatigue level assessment result of the current driver.

[0010] Further, the true value statistics of the eyelid distance feature data and the blink feature data to obtain the normal value of the eyelid distance and the percentage of blink frame numbers of the current driver includes:

[0011] Obtain a first sample set; wherein, the first sample set includes the eyelid distances of the current driver collected within a first preset time window, and the first sample set is collected under a first startup condition;

[0012] Sort the eyelid distances in the first sample set from largest to smallest, and select a preset number of eyelid distances ranked at the front;

[0013] Take the mean of the selected eyelid distances as the normal value of the eyelid distance of the current driver;

[0014] After obtaining the normal value of the eyelid distance, obtain a second sample set; wherein, the second sample set includes the eyelid distances of the current driver collected within a second preset time window, and the second sample set is continuously collected under a second startup condition;

[0015] Obtain the maximum number of closed-eye frames corresponding to the blink action according to the camera frame rate, and increase the maximum number of closed-eye frames by a preset number of frames to obtain the upper limit of the continuous closed-eye frame number corresponding to the blink action;

[0016] Calculate the eye closure degree according to the normal value of the eyelid distance and the eyelid distance of the current driver;

[0017] Compare the size of the eye closure degree with the closure degree threshold, and filter out the closed-eye frames in the second sample set according to the comparison result;

[0018] Compare the number of consecutive closed-eye frames in the closed-eye frame screening result with the upper limit of the consecutive closed-eye frame number;

[0019] According to the comparison result, regard the consecutive closed-eye frames exceeding the upper limit of the consecutive closed-eye frame number as closed-eye frames, and regard the consecutive closed-eye frames not exceeding the upper limit of the consecutive closed-eye frame number as blink frames;

[0020] Take the ratio of the number of blink frames to the total number of image frames in the second sample set as the percentage of blink frame numbers of the current driver.

[0021] Further, the obtaining of the first sample set includes:

[0022] Obtain and determine in real time whether the current vehicle speed meets the first start condition or the first exit condition; wherein, the first start condition is that the current vehicle speed is greater than the first preset start vehicle speed, the first exit condition is that the current vehicle speed is less than the first preset exit vehicle speed, and the first preset start vehicle speed is greater than or equal to the first preset exit vehicle speed;

[0023] When the first start condition is met, collect the eyelid distance feature data of the current driver until the collected statistical information fills the first preset time window;

[0024] When the first exit condition is met, the collected statistical information is not cleared, and the process turns to the step of obtaining and determining in real time whether the current vehicle speed meets the second start condition;

[0025] Further, the obtaining of the second sample set includes:

[0026] Obtain and determine in real time whether the current vehicle speed meets the second start condition; wherein, the second start condition is that the current vehicle speed is greater than the second preset start vehicle speed, the second exit condition is that the current vehicle speed is less than the second preset exit vehicle speed or the situation of failure to obtain blink feature data occurs within the second preset time window, the second preset start vehicle speed is greater than the second preset exit vehicle speed, and the second preset exit vehicle speed is greater than or equal to the first preset start vehicle speed;

[0027] When the second start condition is met, determine whether the second exit condition is met;

[0028] When the second exit condition is not met, collect the blink feature data of the current driver until the collected statistical information fills the second preset time window;

[0029] When the second exit condition is met, clear the collected statistical information, and turn to the step of obtaining and determining in real time whether the current vehicle speed meets the second start condition.

[0030] Further, the facial features include continuous eye - closing features; the method further includes: performing fatigue level assessment according to the number of eye - closing frames and the corrected fatigue level division condition / the preset fatigue level division condition, specifically including:

[0031] Compare the number of eye - closing frames with the secondary fatigue threshold p1 and the sleep threshold p2 corresponding to the continuous eye - closing feature;

[0032] When the number of eye - closing frames corresponding to the continuous eye - closing feature is greater than the secondary fatigue threshold p1 and less than the sleep threshold p2, determine that the fatigue level of the current driver is secondary fatigue;

[0033] When the number of closed - eye frames corresponding to the continuous closed - eye feature is greater than the sleep threshold p2, it is determined that the current driver is in a sleep state.

[0034] Further, the processing of the eyelid distance feature data and the blink feature data according to the normal value of the eyelid distance to obtain the normalized eyelid distance and the percentage of closed - eye frames includes:

[0035] Calculating the normalized eyelid distance of the current driver according to the eyelid distance in the first sample set and the normal value of the eyelid distance of the current driver;

[0036] Calculating the eye closure degree corresponding to each image frame according to the normal value of the eyelid distance and the eyelid distance of the current driver;

[0037] Comparing the eye closure degree with the closure degree threshold, and screening out the closed - eye frames in the second sample set according to the comparison result;

[0038] Calculating the ratio of the number of closed - eye frames to the total number of image frames in the second sample set to obtain the percentage of closed - eye frames.

[0039] Further, the modifying the preset fatigue level classification conditions according to the percentage of blink frames includes:

[0040] Judging whether the percentage of blink frames is greater than the first correction value and less than the second correction value; wherein, the first correction value is the product of the first correction coefficient s1 and the primary fatigue threshold m1 corresponding to the closed - eye duration feature, and the second correction value is the product of the second correction coefficient s2 and the secondary fatigue threshold m2 corresponding to the closed - eye duration feature;

[0041] If so, updating the value of the primary fatigue threshold m1 corresponding to the percentage of blink frames to the product of the third correction coefficient s3 and the primary fatigue threshold m1, and updating the value of the secondary fatigue threshold m2 to the product of the fourth correction coefficient s4 and the secondary fatigue threshold m2;

[0042] If not, keeping the values of the primary fatigue threshold m1 and the secondary fatigue threshold m2 corresponding to the percentage of blink frames unchanged.

[0043] Further, the facial feature data includes a yawn feature; the method further includes: performing fatigue level assessment according to the yawn feature and the modified fatigue level classification conditions / the preset fatigue level classification conditions, specifically including:

[0044] Obtaining the number of yawns within the fifth preset time window;

[0045] Judging whether the number of yawns within the fifth preset time window is greater than the primary fatigue threshold q1 corresponding to the yawn feature;

[0046] If it is greater than, determine that the current driver's fatigue level is first-level fatigue.

[0047] Further, before correcting the preset fatigue level classification conditions according to the percentage of blink frame numbers, the following steps are also included:

[0048] Obtain a third sample set; wherein, the third sample set includes the eyelid distances of a certain number of people collected within a third preset time window in a normal state.

[0049] Obtain the default value of the normal eyelid distance; wherein, the default value of the normal eyelid distance is set according to the eyelid distance of ordinary people in a normal state.

[0050] Based on the eyelid distances in the third sample set, the size of the third preset time window, and the default value of the normal eyelid distance, obtain the normalized average eyelid distance.

[0051] Based on the normalized average eyelid distance, determine that the first-level fatigue threshold corresponding to the eyelid distance feature is n1, and the second-level fatigue threshold is n2.

[0052] Obtain a fourth sample set; wherein, the fourth sample set includes the blink frame numbers of a certain number of people continuously collected within a fourth preset time window in a normal state.

[0053] Calculate the default value of the percentage of blink frame numbers based on the blink frame numbers in the fourth sample set, the size of the fourth preset time window, and the camera frame rate.

[0054] Based on the default value of the percentage of blink frame numbers, determine that the first-level fatigue threshold corresponding to the closed-eye duration feature is m1, and the second-level fatigue threshold is m2.

[0055] Further, the fatigue level assessment according to the normalized eyelid distance, the percentage of closed-eye frame numbers, and the corrected fatigue level classification conditions includes:

[0056] Compare the size of the normalized eyelid distance with the first-level fatigue threshold n1 and the second-level fatigue threshold n2 corresponding to the eyelid distance feature.

[0057] When the normalized eyelid distance is greater than the first-level fatigue threshold n1 and less than the second-level fatigue threshold n2, determine that the current driver's fatigue level is first-level fatigue.

[0058] When the normalized eyelid distance is greater than the second-level fatigue threshold n2, determine that the current driver's fatigue level is second-level fatigue.

[0059] And, compare the percentage of the number of closed-eye frames with the first-level fatigue threshold m1 and the second-level fatigue threshold m2 corresponding to the blinking feature;

[0060] When the percentage of the number of closed-eye frames is greater than the first-level fatigue threshold m1 and less than the second-level fatigue threshold m2, determine that the fatigue level of the current driver is first-level fatigue;

[0061] When the percentage of the number of closed-eye frames is greater than the second-level fatigue threshold m2, determine that the fatigue level of the current driver is second-level fatigue.

[0062] A second aspect of the present invention provides a driver fatigue detection device, including the following modules:

[0063] A data acquisition module, configured to acquire the facial feature data of the current driver; wherein, the facial feature data includes eyelid distance feature data and blinking feature data;

[0064] A true value statistics module, configured to perform true value statistics on the eyelid distance feature data and the blinking feature data to obtain the normal eyelid distance value and the percentage of the number of blinking frames of the current driver;

[0065] A data processing module, configured to process the eyelid distance feature data and the blinking feature data according to the normal eyelid distance value to obtain a normalized eyelid distance and a percentage of the number of closed-eye frames;

[0066] A parameter update module, configured to correct the preset fatigue level division condition according to the percentage of the number of blinking frames to obtain a corrected fatigue level division condition;

[0067] A fatigue level evaluation module, configured to perform fatigue level evaluation according to the normalized eyelid distance, the percentage of the number of closed-eye frames, and the corrected fatigue level division condition, and output a fatigue level evaluation result of the current driver.

[0068] A third aspect of the present invention provides a vehicle, and the vehicle is provided with the driver fatigue detection device described in the second aspect of the present invention.

[0069] A fourth aspect of the present invention provides an electronic device, the electronic device includes a processor and a memory, and at least one instruction, at least one program, a code set or an instruction set is stored in the memory, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the driver fatigue detection method proposed in the first aspect of the present invention.

[0070] A fifth aspect of the present invention provides a computer-readable storage medium, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the driver fatigue detection method as described in the first aspect of the present invention.

[0071] Implementing the present invention has the following beneficial effects:

[0072] A driver fatigue detection method, device, vehicle, electronic device and storage medium provided by the present invention establish a reliable true value statistical system, eliminate individual differences in the eye features of drivers, and use this as the basis for fatigue detection. Then, the PERCLOS algorithm is used for fatigue judgment, and through multi-feature fusion evaluation, false detection and missed detection are avoided to a certain extent, and the fatigue level of the driver can be evaluated more accurately.

[0073] Additional aspects and advantages of the present invention will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0075] Figure 1 is a flowchart of the driver fatigue detection method provided by an embodiment of the present invention;

[0076] Figure 2 is a flowchart of step S120 provided by an embodiment of the present invention;

[0077] Figure 3 is a true value statistical flowchart provided by an embodiment of the present invention;

[0078] Figure 4 is a flowchart of step S130 provided by an embodiment of the present invention;

[0079] Figure 5 is a flowchart of modifying the preset fatigue level division conditions provided by an embodiment of the present invention;

[0080] Figure 6 is a flowchart of determining the default values of relevant system parameters provided by an embodiment of the present invention;

[0081] Figure 7 is a structural block diagram of the driver fatigue detection device provided by an embodiment of the present invention. Detailed implementation manners

[0082] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without making creative efforts shall fall within the protection scope of the present invention. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals indicate the same or similar elements or elements having the same or similar functions throughout.

[0083] Embodiment

[0084] The embodiments of the present invention optimize the closing-eye logic. The optimized solution takes into account the individual differences in the eye characteristics of the driver, and introduces two eye characteristics, namely the eyelid distance and the percentage of the number of blinking frames, into the closing-eye logic. Even if the eye characteristics of different drivers are different, it does not affect the judgment of the closing-eye logic.

[0085] The existing closing-eye frame judgment method first uses the eye aspect ratio (EAR) to judge blinking, and then judges whether the current frame is in the closing-eye state according to this parameter according to the PERCLOS method. However, the initial value of the eye aspect ratio of each person in the normal state is different, and the action habits of different people are also different. Directly judging the closing-eye based on the eye aspect ratio will cause misrecognition due to individual differences.

[0086] To solve this problem, the embodiments of the present invention introduce the eye characteristic of the eyelid distance into the closing-eye frame judgment. The judgment process is as follows: First, count the eyelid width (distance) of the current driver in the normal state, and then calculate the eye closing degree according to the normal value of the eyelid distance. When the eye closing degree exceeds a certain threshold, it is determined that the current frame is a closing-eye frame.

[0087] According to relevant research, when a person is fatigued, eye characteristics will be manifested: the blinking frequency increases and the blinking speed decreases. According to the PERCLOS algorithm, when the percentage of the eye closing time within a unit time exceeds a certain threshold, it can be used as a basis for fatigue judgment. However, the number of closing-eye frames obtained by the above closing-eye logic judgment includes the closing-eye in the blinking action. Therefore, the blinking frequency and the number of blinking frames have a certain influence on the judgment of the PERCLOS algorithm. There must be differences in the blinking frequency and speed among individuals, and sometimes abnormal situations different from ordinary people occur, which will inevitably affect the accuracy of the fatigue level evaluation results. To make the evaluation results more accurate, the embodiments of the present invention introduce the eye characteristic of the percentage of the number of blinking frames.

[0088] Figure 1It is a flowchart of the driver fatigue detection method provided by an embodiment of the present invention. This specification provides method operation steps such as in the embodiment or flowchart, but based on routine or non-creative labor, there may be more or fewer operation steps. The step sequence listed in the embodiment is only one way among the execution sequences of numerous steps and does not represent the only execution sequence. When the actual system or server product executes, it can be executed in the method sequence shown in the embodiment or the drawings, or in parallel (for example, in an environment with parallel processors or multi-threaded processing).

[0089] Specifically, as Figure 1 shown, the method may include the following steps:

[0090] S110: Obtain the facial feature data of the current driver; wherein, the facial feature data includes eyelid distance feature data and blink feature data;

[0091] Obtaining the facial feature data of the current driver includes: obtaining an image frame containing the facial image of the current driver, and identifying the image frame to obtain multiple facial features of the current driver.

[0092] S120: Perform truth value statistics on the eyelid distance feature data and the blink feature data to obtain the normal eyelid distance value and the percentage of blink frame numbers of the current driver;

[0093] S130: Process the eyelid distance feature data and the blink feature data according to the normal eyelid distance value to obtain the normalized eyelid distance and the percentage of closed-eye frame numbers;

[0094] S140: Modify the preset fatigue level division conditions according to the percentage of blink frame numbers to obtain the modified fatigue level division conditions;

[0095] S150: Perform fatigue level evaluation according to the normalized eyelid distance, the percentage of closed-eye frame numbers, and the modified fatigue level division conditions, and output the fatigue level evaluation result of the current driver.

[0096] Figure 2 It is a flowchart of step S120 provided by an embodiment of the present invention. Specifically, as Figure 2 shown, performing truth value statistics on the eyelid distance feature data and the blink feature data to obtain the normal eyelid distance value and the percentage of blink frame numbers of the current driver includes the following steps:

[0097] S121: Obtain the first sample set; wherein, the first sample set includes the eyelid distances of the current driver collected within the first preset time window, and the first sample set is collected under the first startup condition; the first preset time window is a sliding window; for example, the size of the third preset time window is 5 min, and the first sample set is the eyelid distances for 5 min.

[0098] S122: Sort the eyelid distances in the first sample set from largest to smallest, and select a preset number of eyelid distances at the front of the sorting.

[0099] In some embodiments, the preset number here can be determined according to the frame rate. For example, the preset number can be 30f, where f is the frame rate of the camera; in some embodiments, the preset number here can be a specific value. For example, the preset number can be 500.

[0100] It should be noted that the above examples are only used to illustrate the embodiments of the present invention and should not be regarded as a limitation on the protection scope of this specification.

[0101] S123: Take the mean of the selected eyelid distances as the normal value of the eyelid distance of the current driver.

[0102] In a specific example, the process of calculating the normal value of the eyelid distance includes: from the collected 5-minute eyelid distance data, select the maximum values of the first 30f frames in the sorting, and take the mean of these values to obtain the normal value of the eyelid distance eyeW of the current driver. After obtaining the normal value of the eyelid distance eyeW, it can be used for blink logic judgment. The calculation formula for the normal value of the eyelid distance is as follows:

[0103] eyeW = (eyeW1 + eyeW2 + … + eyeW 30f ) / 30f;

[0104] Where, eyeW refers to the normal value of the eyelid distance of the current driver;

[0105] eyeW i refers to the i-th eyelid distance of the current driver, where i = 1, 2, …, 30f;

[0106] 30f is the value of the preset number recorded in step S122;

[0107] f refers to the frame rate of the camera.

[0108] S124: After obtaining the normal value of the eyelid distance, obtain a second sample set; where, the second sample set includes the eyelid distances of the current driver collected within a second preset time window, and the second sample set is continuously collected under a second start condition; the second preset time window is a sliding window.

[0109] S125: Obtain the maximum number of closed-eye frames corresponding to the blink action according to the frame rate of the camera, and increase the maximum number of closed-eye frames by a preset number of frames to obtain the upper limit of the continuous closed-eye frames corresponding to the blink action.

[0110] The blinking speed of a person is generally about 200 ms. According to the camera frame rate f, the number of frames eyeBlinkF1 required for an average person to blink in the current environment is obtained. Among them, eyeBlinkF1 is the number of blinks of most people under normal circumstances, determined by the mean of the mode.

[0111] According to the differences among different people, a preset number of frames c is floated upward. If the number of consecutive closed-eye frames is within (eyeBlinkF1 + c) frames, it is considered a blinking action. Among them, c needs to be determined through debugging. Setting a certain redundancy c is to ensure that blinking actions will not be missed.

[0112] S126: Calculate the eye closure degree based on the normal value of the eyelid distance and the current driver's eyelid distance;

[0113] Specifically, the calculation formula for the eye closure degree is: eyeClose = 1 - eyeC / eyeW;

[0114] Among them, eyeClose is the eye closure degree, representing the degree of closure of the current driver's eyes;

[0115] eyeC is the current eyelid distance, that is, the detection value of the current vision algorithm;

[0116] eyeW is the normal value of the eyelid distance.

[0117] S127: Compare the size of the eye closure degree with the closure degree threshold, and screen out the closed-eye frames in the second sample set according to the comparison result;

[0118] According to the PERCLOS method, the two indicators P70 and P80 can best reflect a person's closed-eye state. Therefore, in one embodiment, an intermediate value eyeF = 75% is taken as the threshold for opening and closing the eyes. When the eye closure degree eyeClose >= 75%, the current frame is a closed-eye frame, otherwise it is a non-closed-eye frame.

[0119] S128: Compare the number of consecutive closed-eye frames in the closed-eye frame screening result with the upper limit of the number of consecutive closed-eye frames;

[0120] S129: According to the comparison result, regard the consecutive closed-eye frames exceeding the upper limit of the number of consecutive closed-eye frames as closed-eye frames, and regard the consecutive closed-eye frames not exceeding the upper limit of the number of consecutive closed-eye frames as blinking frames;

[0121] S12X: Take the ratio of the number of blinking frames to the total number of image frames in the second sample set as the percentage of the number of blinking frames of the current driver.

[0122] That is to say, the truth value statistics of the eye features of the current driver in the embodiments of the present invention are divided into two stages. The first stage (corresponding to steps S131 - S133) statistics the normal eyelid distance of the current driver, and the second stage (corresponding to steps S134 - S136) statistics the number of blinks of the current driver within a fixed time.

[0123] To ensure that the truth value statistics are carried out only when the driver is in a stable and normal driving state, the vehicle speed information needs to be received. Considering that the recorded data obtained when the driver is in the driving state is generally relatively valid, the main purpose of setting the first start condition and the first exit condition in the first stage is to determine whether the vehicle has started according to the first start condition and the first exit condition, so as to avoid collecting a lot of invalid data when the driver is not ready (i.e., in a non - driving state). Figure 3 is the truth value statistics flowchart provided by the embodiments of the present invention. Please refer to Figure 3 , to obtain the first sample set, including:

[0124] Obtain and judge in real time whether the current vehicle speed meets the first start condition or the first exit condition; wherein, the first start condition is that the current vehicle speed is greater than the first preset start vehicle speed, the first exit condition is that the current vehicle speed is less than the first preset exit vehicle speed, and the first preset start vehicle speed is greater than or equal to the first preset exit vehicle speed; Figure 3 shows the situation where both the first preset start vehicle speed and the first preset exit vehicle speed are 5 km / h. In practical applications, other values can also be used for the first preset start vehicle speed and the first preset exit vehicle speed. For example, the first preset start vehicle speed is 7 km / h and the first preset exit vehicle speed is 5 km / h.

[0125] When the first start condition is met, collect the eyelid distance feature data of the current driver until the collected statistical information fills the first preset time window;

[0126] When the first exit condition is met, the collected statistical information is not cleared, and it turns to the step of obtaining and judging in real time whether the current vehicle speed meets the second start condition;

[0127] Since the second stage needs to count the number of blinks of the driver within a period of time, it is best not to miss the recorded data, so the vehicle speed needs to be relatively high to ensure that the driver is always within the range of the camera. Please continue to refer to Figure 3 , to obtain the second sample set, including:

[0128] Obtain and judge in real time whether the current vehicle speed meets the second start condition;

[0129] Among them, the second start condition is that the current vehicle speed is greater than the second preset start vehicle speed, and the second exit condition is that the current vehicle speed is less than the second preset exit vehicle speed or the situation of failure to obtain blink feature data occurs within the second preset time window. The second preset start vehicle speed is greater than the second preset exit vehicle speed, and the second preset exit vehicle speed is greater than or equal to the first preset start vehicle speed.

[0130] Figure 3 Illustrates the situation where the second preset start vehicle speed is 10 km / h and the second preset exit vehicle speed is 5 km / h. In practical applications, other values can also be used for the second preset start vehicle speed and the second preset exit vehicle speed. For example, the second preset start vehicle speed is 12 km / h and the second preset exit vehicle speed is 6 km / h.

[0131] When the second start condition is satisfied, it is judged whether the second exit condition is satisfied;

[0132] When the second exit condition is not satisfied, the blink feature data of the current driver is collected until the collected statistical information fills the second preset time window;

[0133] When the second exit condition is satisfied, the collected statistical information is cleared, and it turns to the step of real-time obtaining and judging whether the current vehicle speed satisfies the second start condition. That is to say, in the second stage opening state, if the failure to obtain human eye information occurs within 5 minutes, the statistical information is cleared; until the eye information can be obtained again, the statistical timing restarts and the face information is collected again.

[0134] Figure 4 is the flowchart of step S130 provided by the embodiment of the present invention. Specifically, as Figure 4 shown, the eyelid distance feature data and blink feature data are processed according to the normal value of the eyelid distance to obtain the normalized eyelid distance and the percentage of the number of closed-eye frames, including the following steps:

[0135] S131: Calculate the normalized eyelid distance of the current driver according to the eyelid distance in the first sample set and the normal value of the eyelid distance of the current driver;

[0136] Specifically, set the size of the first preset time window to eyeN, and count the normalized eyelid distance eyeAvrN within the first preset time window; eyeAvrN = the sum of the eyelid distances within the window / eyeN * the normal value of the eyelid distance; among them, eyeN can take 30 s, and can also be other values set according to actual needs.

[0137] S132: Calculate the eye closure degree corresponding to each image frame according to the normal value of the eyelid distance and the eyelid distance of the current driver;

[0138] S133: Compare the eye closure degree with the closure degree threshold, and screen out the closed-eye frames in the second sample set according to the comparison result;

[0139] S134: Calculate the ratio of the number of closed-eye frames to the total number of image frames in the second sample set to obtain the percentage of closed-eye frames.

[0140] Specifically, set the size of the second preset time window as eyeM, and count the percentage of closed-eye frames Mx within the second preset time window; Mx = the number of closed-eye frames in the second sample set / eyeM; where the value of eyeM can be 30s or other values set according to actual needs.

[0141] Based on the default system parameters, the PERCLOS algorithm is used to calculate the ratio of the number of closed-eye frames and the blinking frequency of the driver within a period of time, and the fatigue state of the driver can be judged. However, the scenarios covered by this method are limited. It can only report the corresponding fatigue state under limited parameter ratios and cannot adapt to changes in the population. Therefore, it is necessary to correct the system parameters to make them suitable for the current driver.

[0142] Figure 5 It is a flowchart for correcting the preset fatigue level classification conditions provided by an embodiment of the present invention. Specifically, as Figure 5 shown, correcting the preset fatigue level classification conditions according to the percentage of blinking frames includes:

[0143] S141: Judge whether the percentage of blinking frames is greater than the first correction value and less than the second correction value; where the first correction value is the product of the first correction coefficient s1 and the primary fatigue threshold m1 corresponding to the closed-eye duration feature, and the second correction value is the product of the second correction coefficient s2 and the secondary fatigue threshold m2 corresponding to the closed-eye duration feature;

[0144] Where: s1 is the coefficient when the blinking time is too long and within the normal value range

[0145] s2 is the abnormal blinking time situation.

[0146] s3, s4 are the correction coefficients for m1 and m2 when the blinking time is too long.

[0147] s1, s2, s3, s4 are all calibrated from a large amount of data statistics.

[0148] S142: If so, update the value of the primary fatigue threshold m1 corresponding to the percentage of blinking frames to the product of the third correction coefficient s3 and the primary fatigue threshold m1, and update the value of the secondary fatigue threshold m2 to the product of the fourth correction coefficient s4 and the secondary fatigue threshold m2;

[0149] In practical applications, some users blink too frequently under normal circumstances, and the percentage of closed-eye frame counts obtained when they are not fatigued is already close to or reaches the threshold, resulting in false alarms. For this group of people, the threshold needs to be increased. Therefore, when s1*eyeBlinkDFT < eyeBlinkFC < s2*eyeBlinkDFT, it is considered that the current driver's blinking time exceeds that of ordinary people, and the primary fatigue threshold m1 and the secondary fatigue threshold m2 may not be applicable to the current driver. The corresponding primary fatigue threshold and secondary fatigue threshold need to be corrected: m1 = s3*m1, m2 = s4*m2.

[0150] S143: If not, keep the values of the primary fatigue threshold m1 and the secondary fatigue threshold m2 corresponding to the percentage of blinking frame counts unchanged.

[0151] When eyeBlinkFC < s1*eyeBlinkDFT, it is considered that the current driver's blinking is the same as that of ordinary people, and the primary fatigue threshold m1 and the secondary fatigue threshold m2 can be judged using the default values.

[0152] Generally, fatigue will cause an increase in the number of blinks. If the driver is already fatigued when getting in the car, the blink statistics at this time will be inaccurate, possibly far greater than the number of blinks of normal people, resulting in abnormal situations. Adjusting the threshold at this time will cause missed reports. Therefore, when eyeBlinkFC > s2*eyeBlinkDFT, it is considered that the current driver's blinking action is an abnormal situation. To prevent abnormal threshold adjustment, for this situation, the default values m1 and m2 are still used. The specific query is shown in Table 1 below.

[0153] Table 1 Threshold Change

[0154] Relationship between eyeBlinkFC and eyeBlinkDFT Threshold change eyeBlinkFC<s1*eyeBlinkDFT m1 = m1, m2 = m2 s1*eyeBlinkDFT < eyeBlinkFC < s2*eyeBlinkDFT m1 = s3 * m1, m2 = s4 * m2 eyeBlinkFC > s2 * eyeBlinkDFT m1 = m1, m2 = m2

[0155] Before the truth value system is calculated, the default values of relevant system parameters need to be set to ensure the normal operation of fatigue assessment. In some embodiments, before correcting the preset fatigue level classification conditions according to the percentage of blinking frame counts, it also includes setting the default values of relevant system parameters. Figure 6 It is the flowchart for determining the default values of relevant system parameters provided by the embodiments of the present invention. The specific relevant steps are as Figure 6 shown:

[0156] S161: Obtain the third sample set; where the third sample set includes the eyelid distances of a certain number of people in a normal state collected within the third preset time window; it should be noted that when people are in a normal state, the eyelid distances are mostly in their normal situations at most times.

[0157] S162: Obtain the default value of the normal eyelid distance; wherein, the default value of the normal eyelid distance, eyeWidthDFT, is set according to the eyelid distance of ordinary people in the normal state;

[0158] S163: Obtain the normalized average value of the eyelid distance based on the eyelid distance in the third sample set, the size of the third preset time window, and the default value of the normal eyelid distance;

[0159] Normalizing the eyelid distance helps to eliminate the differences in eyelid distance among individuals and facilitates the setting of fatigue thresholds.

[0160] S164: Determine that the first-level fatigue threshold corresponding to the eyelid distance feature is n1 and the second-level fatigue threshold is n2 according to the normalized average value of the eyelid distance;

[0161] S165: Obtain the fourth sample set; wherein, the fourth sample set includes the number of blinks of a certain number of people continuously collected within the fourth preset time window in the normal state; for example, the size of the fourth preset time window can be 5 min, and obtaining the fourth sample set specifically means counting the number of blinks of a certain data set of people within 5 minutes.

[0162] S166: Calculate the default value of the blink frame percentage according to the number of blinks in the fourth sample set, the size of the fourth preset time window, and the camera frame rate; for example, step S166 can be to calculate the mode of the samples in the fourth sample set, and set the default value of the blink frame percentage eyeBlinkDFT within the fourth preset time according to the average value of the modes. The calculation formula for the default value of the blink frame percentage is as follows:

[0163] eyeBlinkDFT = average value / t * f,

[0164] wherein, eyeBlinkDFT is the default value of the blink frame percentage;

[0165] The average value refers to the average of the modes of the samples in the fourth sample set;

[0166] t is the size of the fourth preset time window, for example, it can be 5 min;

[0167] f is the camera frame rate.

[0168] S167: Determine that the first-level fatigue threshold corresponding to the closed-eye duration feature is m1 and the second-level fatigue threshold is m2 according to the default value of the blink frame percentage.

[0169] Furthermore, fatigue level assessment is performed according to the normalized eyelid distance, the closed-eye frame percentage, and the corrected fatigue level division conditions, including:

[0170] Compare the magnitudes of the normalized eyelid distance eyeAvrN with the first-level fatigue threshold n1 and the second-level fatigue threshold n2 corresponding to the eyelid distance feature;

[0171] When the normalized eyelid distance eyeAvrN is greater than the first-level fatigue threshold n1 and less than the second-level fatigue threshold n2, determine that the current driver's fatigue level is first-level fatigue; that is, if n1 < eyeAvrN < n2, it is first-level fatigue;

[0172] When the normalized eyelid distance eyeAvrN is greater than the second-level fatigue threshold n2, determine that the current driver's fatigue level is second-level fatigue; that is, if eyeAvrN > n2, it is second-level fatigue.

[0173] And, compare the percentage of closed-eye frames with the first-level fatigue threshold m1 and the second-level fatigue threshold m2 corresponding to the blinking feature;

[0174] When the percentage of closed-eye frames Mx is greater than the first-level fatigue threshold m1 and less than the second-level fatigue threshold m2, determine that the current driver's fatigue level is first-level fatigue; that is, if m1 < Mx < m2, it is first-level fatigue;

[0175] When the percentage of closed-eye frames Mx is greater than the second-level fatigue threshold m2, determine that the current driver's fatigue level is second-level fatigue; that is, Mx > m2, it is second-level fatigue.

[0176] The fatigue level assessment based on the yawn state includes: setting a sliding window mouthM and counting the number of yawns mouthQ within the window; if mouthQ > q1, it is first-level fatigue; that is, when the number of yawns exceeds the threshold within a period of time, report first-level fatigue, where mouthM can take 60s and can also be other values according to actual needs.

[0177] Yawning is also the most obvious manifestation feature of a person's fatigue state. Therefore, in one embodiment, the facial feature data may also include the yawn feature; the method further includes: performing fatigue level assessment according to the yawn feature and the corrected fatigue level division conditions / preset fatigue level division conditions. Since yawning is actually a manifestation of a person's resistance to fatigue and indicates that the driver is relatively awake to a certain extent at this time, only the first-level fatigue threshold q1 is set for the yawn feature, and the yawn state is only used to judge first-level fatigue. The specific steps for assessing the fatigue state according to the yawn feature are as follows:

[0178] Obtain the number of yawns within the fifth preset time window;

[0179] Judge whether the number of yawns within the fifth preset time window is greater than the first-level fatigue threshold q1 corresponding to the yawn feature;

[0180] If it is greater than, determine that the current driver's fatigue level is primary fatigue.

[0181] If the driver has consecutive eye closures and may enter the sleep state, adding the condition of consecutive eye closures can cover more situations. In one embodiment, the facial features may further include the feature of consecutive eye closures. The method further includes: performing fatigue level assessment according to the number of eye closure frames and the corrected fatigue level division condition / preset fatigue level division condition, specifically including the following steps:

[0182] Compare the number of eye closure frames with the secondary fatigue threshold p1 and the sleep threshold p2 corresponding to the feature of consecutive eye closures;

[0183] When the number of eye closure frames corresponding to the feature of consecutive eye closures is greater than the secondary fatigue threshold p1 and less than the sleep threshold p2, determine that the current driver's fatigue level is secondary fatigue;

[0184] When the number of eye closure frames corresponding to the feature of consecutive eye closures is greater than the sleep threshold p2, determine that the current driver is in the sleep state.

[0185] When the camera frame rate is known, the number of eye closure frames and the consecutive eye closure time can be converted into each other. Therefore, in an alternative embodiment, the fatigue threshold can also be the consecutive eye closure time. Set the first consecutive eye closure duration eyeT1 and the second consecutive eye closure duration eyeT2 for the consecutive eye closure time, where eyeT1 is less than eyeT2. If the consecutive eye closure time exceeds eyeT1, it is secondary fatigue; if it exceeds eyeT2, it is the sleep state. In a specific example, eyeT1 can be taken as 2s; eyeT2 can be taken as 3s. According to actual needs, the values of eyeT1 and eyeT2 can also be other values.

[0186] Preferably, the fatigue assessment result can be to output the maximum value among multiple fatigue level assessment results as the fatigue level of the current driver. For example, if the fatigue level obtained based on the normalized eyelid distance assessment is level 2 and the fatigue level obtained based on the yawn assessment is level 1, then determine that the fatigue level of the current driver is level 2. For example, if the fatigue level obtained based on the normalized eyelid distance assessment is level 2 and the fatigue level obtained based on the consecutive eye closure duration assessment is sleep, then determine that the fatigue level of the current driver is sleep.

[0187] The driver fatigue detection method provided by the embodiments of the present invention, from a statistical perspective, performs multi-feature fusion assessment based on at least two features among multiple facial features, such as eyelid distance, percentage of blink frames, number of yawns, consecutive eye closure features, etc., to avoid misdetection and missed detection to a certain extent and improve the detection accuracy.

[0188] It should be noted that the present invention is not limited by the described order of actions, because according to the present invention, certain steps can be carried out in other orders or simultaneously.

[0189] Figure 7 is a structural block diagram of a driver fatigue detection device provided by an embodiment of the present invention. Specifically, as Figure 7 shown, the device may include the following modules:

[0190] A data acquisition module 201, configured to acquire facial feature data of the current driver; wherein, the facial feature data includes eyelid distance feature data and blink feature data;

[0191] Specifically, acquiring the facial feature data of the current driver includes: acquiring an image frame containing the facial image of the current driver, and identifying the image frame to obtain multiple facial features of the current driver.

[0192] A true value statistics module 202, configured to perform true value statistics on the eyelid distance feature data and the blink feature data to obtain the normal eyelid distance value and the percentage of blink frames of the current driver;

[0193] A data processing module 203, configured to process the eyelid distance feature data and the blink feature data according to the normal eyelid distance value to obtain a normalized eyelid distance and a percentage of closed-eye frames;

[0194] A parameter update module 204, configured to correct the preset fatigue level classification conditions according to the percentage of blink frames to obtain the corrected fatigue level classification conditions;

[0195] A fatigue level evaluation module 205, configured to perform fatigue level evaluation according to the normalized eyelid distance, the percentage of closed-eye frames, and the corrected fatigue level classification conditions, and output the fatigue level evaluation result of the current driver.

[0196] An embodiment of the present invention further provides a vehicle configured with the driver fatigue detection device provided by the above device embodiment. It should be noted that without departing from the scope of the present disclosure, the vehicle of the present invention can be a truck, a sport utility vehicle, a van, a recreational vehicle, or any other type of vehicle.

[0197] An embodiment of the present invention further provides an electronic device, which includes a processor and a memory. At least one instruction, at least one program, a code set, or an instruction set is stored in the memory, and at least one instruction, at least one program, a code set, or an instruction set is loaded and executed by the processor to implement the driver fatigue detection method as in the method embodiment.

[0198] An embodiment of the present invention further provides a storage medium, which can be arranged in a server to store at least one instruction, at least one program, a code set or an instruction set related to the driver fatigue detection method in the method embodiment. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the driver fatigue detection method provided in the above method embodiment.

[0199] Optionally, in this embodiment, the above storage medium may be located in at least one of multiple network servers in a computer network. Optionally, in this embodiment, the above storage medium may include, but is not limited to: various media that can store program codes such as USB flash drives, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), mobile hard disks, magnetic disks or optical discs.

[0200] As can be seen from the embodiments of the driver fatigue detection method, device, vehicle, electronic device or storage medium provided by the present invention above, a reliable true value statistical system is established in the embodiments of the present invention, and the individual differences of the driver's eye features are eliminated. Taking this as the basis for fatigue detection, and then using the PERCLOS algorithm to judge fatigue, through multi-feature fusion evaluation, false detections and missed detections are avoided to a certain extent, and the fatigue level of the driver can be evaluated more accurately.

[0201] It should be noted that: the above sequence of embodiments of the present invention is only for description and does not represent the superiority or inferiority of the embodiments. And the above specific embodiments of this specification have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims may be executed in a different order from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0202] Each embodiment in this specification is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and the key point of each embodiment is to illustrate the differences from other embodiments. In particular, for the device and server embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0203] Those of ordinary skill in the art can understand that all or part of the steps to implement the above embodiments can be completed by hardware or by a program instructing relevant hardware. The program can be stored in a computer-readable storage medium, and the above-mentioned storage medium can be a read-only memory, a magnetic disk, an optical disk, etc.

[0204] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A driver fatigue detection method, characterized in that, Including: Obtaining the facial feature data of the current driver; wherein, the facial feature data includes eyelid distance feature data and blink feature data; Performing truth value statistics on the eyelid distance feature data and the blink feature data to obtain the normal eyelid distance value and the percentage of blink frames of the current driver; Processing the eyelid distance feature data and the blink feature data according to the normal eyelid distance value to obtain the normalized eyelid distance and the percentage of closed-eye frames, wherein the closed-eye frames corresponding to the percentage of closed-eye frames are consecutive closed-eye frames exceeding the upper limit of the consecutive closed-eye frame number, and the blink frames corresponding to the percentage of blink frames are consecutive blink frames not exceeding the upper limit of the consecutive closed-eye frame number; Judging whether the percentage of blink frames is greater than a first correction value and less than a second correction value; wherein, the first correction value is the product of a first correction coefficient s1 and a first-level fatigue threshold m1 corresponding to the percentage of blink frames, the second correction value is the product of a second correction coefficient s2 and a second-level fatigue threshold m2 corresponding to the percentage of blink frames, and the first-level fatigue threshold m1 and the second-level fatigue threshold m2 are preset fatigue level division conditions; If so, updating the value of the first-level fatigue threshold m1 corresponding to the percentage of blink frames to the product of a third correction coefficient s3 and the first-level fatigue threshold m1, and updating the value of the second-level fatigue threshold m2 to the product of a fourth correction coefficient s4 and the second-level fatigue threshold m2 to obtain the corrected fatigue level division conditions; If not, keeping the values of the first-level fatigue threshold m1 and the second-level fatigue threshold m2 corresponding to the percentage of blink frames unchanged to obtain the corrected fatigue level division conditions; Performing fatigue level evaluation according to the normalized eyelid distance, the percentage of closed-eye frames and the corrected fatigue level division conditions, and outputting the fatigue level evaluation result of the current driver.

2. The method according to claim 1, characterized in that The performing truth value statistics on the eyelid distance feature data and the blink feature data to obtain the normal eyelid distance value and the percentage of blink frames of the current driver includes: Obtaining a first sample set; wherein, the first sample set includes the eyelid distances of the current driver collected within a first preset time window, and the first sample set is collected under a first startup condition; Sorting the eyelid distances in the first sample set from large to small, and selecting a preset number of eyelid distances ranked in the front; Taking the mean value of the selected eyelid distances as the normal eyelid distance value of the current driver; After obtaining the normal eyelid distance value, obtaining a second sample set; wherein, the second sample set includes the eyelid distances of the current driver collected within a second preset time window, and the second sample set is continuously collected under a second startup condition; Obtaining the maximum number of closed-eye frames corresponding to the blink action according to the camera frame rate, and floating the maximum number of closed-eye frames by a preset number of frames to obtain the upper limit of the consecutive closed-eye frame number corresponding to the blink action; Calculating the eye closure degree according to the normal eyelid distance value and the eyelid distance of the current driver; Comparing the size of the eye closure degree with the closure degree threshold, and screening out the closed-eye frames in the second sample set according to the comparison result; Compare the number of consecutive closed-eye frames in the closed-eye frame screening result with the upper limit of the number of consecutive closed-eye frames; According to the comparison result, use the consecutive closed-eye frames that exceed the upper limit of the number of consecutive closed-eye frames as closed-eye frames, and use the consecutive closed-eye frames that do not exceed the upper limit of the number of consecutive closed-eye frames as blink frames; Use the ratio of the number of blink frames to the total number of image frames in the second sample set as the percentage of blink frames of the current driver.

3. The method according to claim 2, wherein The obtaining of the first sample set includes: Obtain and judge in real time whether the current vehicle speed meets the first start condition or the first exit condition; wherein, the first start condition is that the current vehicle speed is greater than the first preset start vehicle speed, the first exit condition is that the current vehicle speed is less than the first preset exit vehicle speed, and the first preset start vehicle speed is greater than or equal to the first preset exit vehicle speed; When the first start condition is met, collect the eyelid distance feature data of the current driver until the collected statistical information fills the first preset time window; When the first exit condition is met, the collected statistical information is not cleared, and turn to the step of obtaining and judging in real time whether the current vehicle speed meets the second start condition; And The obtaining of the second sample set includes: Obtain and judge in real time whether the current vehicle speed meets the second start condition; wherein, the second start condition is that the current vehicle speed is greater than the second preset start vehicle speed, the second exit condition is that the current vehicle speed is less than the second preset exit vehicle speed or a failure occurs in obtaining blink feature data within the second preset time window, the second preset start vehicle speed is greater than the second preset exit vehicle speed, and the second preset exit vehicle speed is greater than or equal to the first preset start vehicle speed; When the second start condition is met, judge whether the second exit condition is met; When the second exit condition is not met, collect the blink feature data of the current driver until the collected statistical information fills the second preset time window; When the second exit condition is met, clear the collected statistical information and turn to the step of obtaining and judging in real time whether the current vehicle speed meets the second start condition.

4. The method according to claim 2, wherein The facial features include continuous closed-eye features; The method further includes: performing fatigue level assessment according to the number of closed-eye frames and the corrected fatigue level division condition or the preset fatigue level division condition, specifically including: Compare the number of closed-eye frames with the secondary fatigue threshold p1 and the sleep threshold p2 corresponding to the continuous closed-eye feature; When the number of closed-eye frames corresponding to the continuous closed-eye feature is greater than the secondary fatigue threshold p1 and less than the sleep threshold p2, determine that the fatigue level of the current driver is secondary fatigue; When the number of closed-eye frames corresponding to the continuous closed-eye feature is greater than the sleep threshold p2, determine that the current driver is in a sleeping state.

5. The method according to claim 2, wherein The processing of the eyelid distance feature data and the blink feature data according to the eyelid distance normal value to obtain the normalized eyelid distance and the percentage of the number of closed-eye frames includes: Calculate the normalized eyelid distance of the current driver according to the eyelid distance in the first sample set and the normal value of the eyelid distance of the current driver; Calculate the eye closure degree corresponding to each image frame according to the normal value of the eyelid distance and the eyelid distance of the current driver; Compare the eye closure degree with the closure degree threshold, and screen out the closed-eye frames in the second sample set according to the comparison result; Calculate the ratio of the number of closed-eye frames to the total number of image frames in the second sample set to obtain the percentage of closed-eye frames.

6. The method according to claim 1, characterized in that, The facial feature data includes yawn features; the method further includes: performing fatigue level assessment according to the yawn features and the modified fatigue level division conditions or the preset fatigue level division conditions, specifically including: Obtain the number of yawns within the fifth preset time window; Determine whether the number of yawns within the fifth preset time window is greater than the first-level fatigue threshold q1 corresponding to the yawn feature; If it is greater, determine that the fatigue level of the current driver is first-level fatigue.

7. The method according to claim 1, wherein Before modifying the preset fatigue level division conditions according to the percentage of blink frames, it further includes: Obtain a third sample set; wherein, the third sample set includes the eyelid distances of a certain number of people in a normal state collected within a third preset time window; Obtain the default value of the normal value of the eyelid distance; wherein, the default value of the normal value of the eyelid distance is set according to the eyelid distance of the general population in a normal state; According to the eyelid distances in the third sample set, the size of the third preset time window, and the default value of the normal value of the eyelid distance, obtain the average value of the normalized eyelid distances; Determine that the first-level fatigue threshold corresponding to the eyelid distance feature is n1 and the second-level fatigue threshold is n2 according to the average value of the normalized eyelid distances; Obtain a fourth sample set; wherein, the fourth sample set includes the number of blink frames of a certain number of people in a normal state continuously collected within a fourth preset time window; Calculate the default value of the percentage of blink frames according to the number of blink frames in the fourth sample set, the size of the fourth preset time window, and the camera frame rate; Determine that the first-level fatigue threshold corresponding to the percentage of blink frames is m1 and the second-level fatigue threshold is m2 according to the default value of the percentage of blink frames.

8. The method according to claim 1, characterized in that, The fatigue level assessment according to the normalized eyelid distance, the percentage of closed-eye frames, and the modified fatigue level division conditions includes: Compare the percentage of closed-eye frames with the first-level fatigue threshold m1 and the second-level fatigue threshold m2 corresponding to the percentage of blink frames; When the percentage of closed-eye frames is greater than the first-level fatigue threshold m1 and less than the second-level fatigue threshold m2, determine that the fatigue level of the current driver is first-level fatigue; When the percentage of closed-eye frames is greater than the second-level fatigue threshold m2, determine that the fatigue level of the current driver is second-level fatigue.

9. A driver fatigue detection device, characterized in that, Including: A data acquisition module for acquiring the facial feature data of the current driver; wherein, the facial feature data includes eyelid distance feature data and blink feature data; A true value statistics module, configured to perform true value statistics on the eyelid distance feature data and the blink feature data to obtain the normal value of the eyelid distance and the percentage of blink frames of the current driver; A data processing module, configured to process the eyelid distance feature data and the blink feature data according to the normal value of the eyelid distance to obtain a normalized eyelid distance and a percentage of closed-eye frames, wherein the closed-eye frames corresponding to the percentage of closed-eye frames are consecutive closed-eye frames exceeding the upper limit of the consecutive closed-eye frame number, and the blink frames corresponding to the percentage of blink frames are consecutive blink frames not exceeding the upper limit of the consecutive closed-eye frame number; Determine whether the percentage of blink frames is greater than a first correction value and less than a second correction value; wherein, the first correction value is the product of a first correction coefficient s1 and a primary fatigue threshold m1 corresponding to the percentage of blink frames, and the second correction value is the product of a second correction coefficient s2 and a secondary fatigue threshold m2 corresponding to the percentage of blink frames; If so, update the value of the primary fatigue threshold m1 corresponding to the percentage of blink frames to the product of a third correction coefficient s3 and the primary fatigue threshold m1, and update the value of the secondary fatigue threshold m2 to the product of a fourth correction coefficient s4 and the secondary fatigue threshold m2 to obtain a corrected fatigue level classification condition; A parameter update module, configured to, if not, keep the values of the primary fatigue threshold m1 and the secondary fatigue threshold m2 corresponding to the percentage of blink frames unchanged to obtain a corrected fatigue level classification condition; A fatigue level evaluation module, configured to perform fatigue level evaluation according to the normalized eyelid distance, the percentage of closed-eye frames, and the corrected fatigue level classification condition, and output a fatigue level evaluation result of the current driver.

10. A vehicle, characterized in that, The vehicle is equipped with the driver fatigue detection device described in claim 9.

11. An electronic device, characterized in that, The electronic device includes a processor and a memory, and at least one instruction, at least one program, a code set, or an instruction set is stored in the memory, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the driver fatigue detection method described in any one of claims 1-8.

12. A computer-readable storage medium, characterized in that At least one instruction, at least one program, a code set, or an instruction set is stored in the storage medium, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the driver fatigue detection method described in any one of claims 1-8.

Images