KSS drowsiness grade evaluation method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of image processing, and particularly discloses a KSS drowsiness grade evaluation method and device, computer equipment and a storage medium. According to the method, the face image of the to-be-evaluated person can be detected according to the preset drowsiness level influence factors, sliding window statistics is performed on the detection result, the target feature probability value is obtained, the target feature probability value set is generated, and finally the target feature probability value is compared with the preset KSS drowsiness level evaluation rule, so that the evaluation accuracy is improved. And obtaining a KSS drowsiness grade evaluation result. Drowsiness level influence factors can be set according to needs, and the drowsiness level of the to-be-assessed person is comprehensively assessed in combination with a plurality of parameters. The KSS drowsiness level evaluation rule is introduced, accidental errors can be eliminated through statistical analysis and conditional judgment of various data, and the accuracy and stability of evaluation are improved. And secondly, only the to-be-detected image needs to be collected, the to-be-detected image is detected, and the sliding window statistics of the detection result is carried out, so that the complexity of KSS drowsiness level evaluation is reduced.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a KSS sleepiness level assessment method, device, computer equipment and storage medium. Background Art

[0002] Currently, in the automotive industry, some vehicle manufacturers have begun to introduce driver fatigue and attention warning systems (DDAW). These systems typically include sensor and camera-based technologies that monitor the driver's facial expressions, eye activities, and the vehicle's driving status to assess the driver's alertness and fatigue. The Karolinska Sleepiness Scale (KSS) used in DDAW is a commonly used self-report scale for assessing an individual's subjective sleepiness level. However, existing driver fatigue and attention warning systems face some challenges in accurately assessing the KSS sleepiness level: 1. Subjectivity and misjudgment issues: Recognition based on facial expressions and eye movements is often subjective, and the performance of different drivers may vary, which affects the accuracy of the evaluation results.

[0003] 2. Operational complexity: Some systems may be too complex, requiring a large number of sensors and computing resources to achieve accurate assessment of sleepiness levels, increasing the cost and complexity of the system.

[0004] 3. Lack of real-time performance: Some systems do not react fast enough when warning the driver and are unable to detect the driver’s fatigue status in time, which affects the effectiveness of the system.

[0005] Therefore, how to reduce the complexity of KSS sleepiness level assessment and improve the accuracy and real-time performance of KSS sleepiness level assessment has become an urgent problem to be solved. Summary of the invention

[0006] The present application provides a KSS sleepiness level assessment method, apparatus, computer equipment and storage medium to reduce the complexity of KSS sleepiness level assessment and improve the accuracy and real-time performance of KSS sleepiness level assessment.

[0007] In a first aspect, the present application provides a KSS sleepiness level assessment method, the method comprising: Acquire continuous facial images of the person to be evaluated as images to be detected; Detect each frame of the image to be detected to obtain face detection information, eye detection information, expression detection information and face angle detection information; Based on a preset threshold, respectively, sliding window statistics are performed on the target features present in the face detection information, the eye detection information, the expression detection information, and the face angle detection information to obtain a probability value of each of the target features, and generate a probability value set; Based on the preset KSS sleepiness level assessment rule, the probability value of at least one of the target features is obtained in the probability value set, and the probability value of each of the target features is compared with the probability value threshold corresponding to each of the target features to obtain the sleepiness level of the person to be assessed.

[0008] In a second aspect, the present application further provides a KSS sleepiness level assessment device, the device comprising: The image acquisition module to be detected is used to acquire continuous facial images of the person to be evaluated as the image to be detected; The detection information acquisition module is used to detect each frame of the image to be detected and obtain face detection information, eye detection information, expression detection information and face angle detection information; A probability value set acquisition module is used to perform sliding window statistics on the target features present in the face detection information, the eye detection information, the expression detection information and the face angle detection information based on a preset threshold, obtain the probability value of each target feature, and generate a probability value set; The KSS sleepiness level assessment module is used to obtain the probability value of at least one of the target features in the probability value set based on a preset KSS sleepiness level assessment rule, and compare the probability value of each of the target features with the probability value threshold corresponding to each of the target features to obtain the sleepiness level of the person to be assessed.

[0009] In a third aspect, the present application also provides a computer device, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the KSS sleepiness level assessment method as described above when executing the computer program.

[0010] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the KSS sleepiness level assessment method as described above.

[0011] The present application discloses a KSS sleepiness level assessment method, device, computer equipment and storage medium, which obtain continuous facial images of a person to be assessed as images to be detected; detect each frame of the image to be detected to obtain face detection information, eye detection information, expression detection information and face angle detection information; based on a preset threshold, perform sliding window statistics on target features present in the face detection information, eye detection information, expression detection information and face angle detection information, respectively, obtain probability values ​​of each target feature, and generate a probability value set; based on a preset KSS sleepiness level assessment rule, obtain the probability value of at least one target feature in the probability value set, and compare the probability value of each target feature with the probability value threshold corresponding to each target feature to obtain the sleepiness level of the person to be assessed. The present application can detect the facial image of the person to be evaluated according to the preset sleepiness level influencing factors, obtain the detection results, and perform sliding window statistics on the detection results to obtain accurate probability values ​​of target features, and generate a set of target feature probability values, and finally compare the target feature probability values ​​with the preset KSS sleepiness level evaluation rules to obtain the final KSS sleepiness level evaluation results. The sleepiness level influencing factors can be set as needed, and the sleepiness level of the person to be evaluated can be comprehensively evaluated by combining multiple parameters. The introduction of KSS sleepiness level evaluation rules can eliminate accidental errors and improve the accuracy and stability of the evaluation by performing statistical analysis and conditional judgment on a variety of data. Secondly, the present application only needs to collect the image to be detected, detect the image to be detected, and perform sliding window statistics on the detection results, which reduces the complexity of the KSS sleepiness level evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0013] Figure 1 is a schematic flow chart of a first embodiment of a KSS sleepiness level assessment method provided in an embodiment of the present application; Figure 2 is a schematic flow chart of sleepiness level assessment of a KSS sleepiness level assessment method provided in an embodiment of the present application; Figure 3 is a schematic flow chart of a second embodiment of a KSS sleepiness level assessment method provided in an embodiment of the present application; Figure 4 A schematic flow chart of dynamic learning of a human eye closing threshold value of a KSS sleepiness level assessment method provided in an embodiment of the present application; Figure 5 A schematic block diagram of a KSS sleepiness level assessment device provided in an embodiment of the present application; Figure 6 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0014] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0015] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0016] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0017] It should be further understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0018] The embodiments of the present application provide a KSS sleepiness level assessment method, device, computer equipment and storage medium. Among them, the KSS sleepiness level assessment method can be applied to a server, and the sleepiness level influencing factors can be set as needed, and the sleepiness level of the person to be assessed can be comprehensively assessed by combining multiple parameters. The introduction of KSS sleepiness level assessment rules can eliminate accidental errors and improve the accuracy and stability of the assessment by performing statistical analysis and conditional judgment on a variety of data. Secondly, the present application only needs to collect images to be detected, detect the images to be detected, and perform sliding window statistics on the detection results, which reduces the complexity of the KSS sleepiness level assessment. Among them, the server can be an independent server or a server cluster.

[0019] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0020] See also Figure 1 , Figure 1 It is a schematic flow chart of a KSS sleepiness level assessment method provided by an embodiment of the present application. The KSS sleepiness level assessment method can be applied to a server, and the sleepiness level influencing factors can be set as needed, and the sleepiness level of the person to be assessed can be comprehensively assessed by combining multiple parameters. The introduction of KSS sleepiness level assessment rules can eliminate accidental errors and improve the accuracy and stability of the assessment by performing statistical analysis and conditional judgment on a variety of data. Secondly, the present application only needs to collect images to be detected, detect the images to be detected, and perform sliding window statistics on the detection results, which reduces the complexity of the KSS sleepiness level assessment.

[0021] like Figure 1 As shown, the KSS sleepiness level evaluation method specifically includes steps S101 to S104.

[0022] S101, obtaining continuous facial images of the person to be evaluated as images to be detected; In one embodiment, an infrared camera is called to capture an image including the face of the person to be evaluated as the image to be detected.

[0023] In one embodiment, the person to be assessed may be a driver, a machine operator, or other persons who need to undergo a KSS sleepiness level assessment.

[0024] S102, detecting each frame of the image to be detected to obtain face detection information, eye detection information, expression detection information, and face angle detection information; In one embodiment, sleepiness level influencing factors may include facial angle features, yawn features, left eye closure features, and right eye closure features, and may also include other features set by the user according to actual needs.

[0025] In one embodiment, the image to be processed is fed into various detection algorithms to obtain basic features such as face and eyes. Specifically, the face detection algorithm is used to detect face information, including face score, face position, 106 feature points, etc. The eye detection algorithm is used to detect eye information, including left and right eye scores, eye aspect ratio, etc. The yawn detection algorithm is used to detect expression information, including mouth score, yawn score, etc. The face angle detection algorithm is used to detect face angle information, including scores in the four directions of up, down, left, and right.

[0026] S103, based on a preset threshold, respectively performing sliding window statistics on the target features present in the face detection information, the eye detection information, the expression detection information, and the face angle detection information, to obtain a probability value of each of the target features, and generating a probability value set; Furthermore, based on a preset threshold, sliding window statistics are performed on the target features existing in the face detection information, the eye detection information, the expression detection information and the face angle detection information to obtain the probability value of each target feature, including: according to the number of frames of the image to be detected, the face detection information, the eye detection information, the expression detection information and the face angle detection information of each image to be detected are added to a preset short-term sliding window and a long-term sliding window respectively; based on the short-term sliding window, the long-term sliding window and the preset threshold, judging and counting whether each target feature exists in the face detection information, the eye detection information, the expression detection information and the face angle detection information to obtain the statistical result of the short-term sliding window and the statistical result of the long-term sliding window; probability calculation is performed based on the statistical result of the short-term sliding window and the statistical result of the long-term sliding window to obtain the short-term probability value of each target feature in the short-term sliding window and the long-term probability value of each target feature in the long-term sliding window.

[0027] In one embodiment, multi-frame statistics are performed in units of frames, and divided into two groups, short-term and long-term, which are recorded as shortTermData and longTermData respectively. Among them, shortTermData is a 20-frame sliding window, and longTermData is a 600-frame sliding window. Through the feature statistics of shortTermData, the probability value of each feature in the short term can be calculated, and through the feature statistics of longTermData, the probability value of each feature in the long term can be calculated.

[0028] In one embodiment, the sliding window statistics may include the following 13 features: face presence, face leaning to the left, face leaning to the right, head lowered, head raised, left eye present, left eye closed, left eye open, right eye present, right eye closed, right eye open, yawning, and eye rubbing.

[0029] In one embodiment, the facial angle features may include face leaning to the left, face leaning to the right, lowering the head, and raising the head.

[0030] In one embodiment, for the statistics of face presence features: if the face score is greater than a threshold, the face is considered to exist and is added to the statistics queue. Otherwise, the face is considered not to exist and is added to the statistics queue. For the statistics of the face leaning to the left feature: the face angle in the face detection result is added to the statistics queue, and the probability of the face angle reaching a given angle threshold is counted. The threshold can be adjusted based on the algorithm used. Other category statistics: such as "face leaning to the right", "bowing", "raising head", "left eye closed", "right eye closed", etc., refer to the similar implementation method of "face leaning to the left". Logic inference of rubbing eyes: when the presence of a face is detected, if the detected left or right eye score is lower than the threshold, it is considered that the eyes are not detected, that is, the eyes are rubbed.

[0031] In one embodiment, the probability value set of the target feature stores short-term probability values ​​and long-term probability values ​​corresponding to each target feature, wherein the short-term probability value is the probability obtained by short-term sliding window statistics, and the long-term probability value is the probability obtained by long-term sliding window statistics, such as the long-term probability value and short-term probability value of the right-sided face feature, the long-term probability value and short-term probability value of the left-sided face feature, the long-term probability value and short-term probability value of the head-lowering feature, the long-term probability value and short-term probability value of the head-up feature, the long-term probability value and short-term probability value of the left eye feature, the long-term probability value and short-term probability value of the left eye closed feature, the long-term probability value and short-term probability value of the left eye open feature, the long-term probability value and short-term probability value of the right eye feature, the long-term probability value and short-term probability value of the right eye closed feature, the long-term probability value and short-term probability value of the right eye open feature, the long-term probability value and short-term probability value of the yawning feature, the long-term probability value and short-term probability value of the eye rubbing feature, etc.

[0032] In one embodiment, in order to achieve real-time and accurate fatigue detection, a sliding window statistical method is adopted. Whenever a new frame of data is detected, the data is updated in the sliding window and the oldest frame of data is removed.

[0033] In one embodiment, two types of sliding windows are used: a short-term sliding window and a long-term sliding window. The short-term sliding window includes the most recent 20 frames of data, and the long-term sliding window includes the most recent 600 frames of data.

[0034] In one embodiment, for each sliding window, the frequency of occurrence of features of each category is counted. For example: If a face is detected in 10 frames within a short-term sliding window (20 frames), the probability of the face existing is 50%. The statistical methods for other features (such as closed eyes, yawning, etc.) are similar.

[0035] In one embodiment, each time a new frame is detected, the data of the new frame is added to the sliding window, and the oldest frame of data in the sliding window is removed. Then, the occurrence probability of each category in the sliding window is recalculated.

[0036] In one embodiment, through the sliding window statistics mechanism, the short-term sliding window can be used to quickly respond to the driver's short-term state changes. The long-term sliding window can be used to evaluate the driver's overall fatigue trend. The fatigue state of the person to be evaluated is evaluated dynamically and in real time, and a more accurate judgment is made based on short-term and long-term statistical data.

[0037] S104. Based on a preset KSS sleepiness level assessment rule, obtain the probability value of at least one of the target features in the probability value set, and compare the probability value of each of the target features with the probability value threshold corresponding to each of the target features to obtain the sleepiness level of the person to be assessed.

[0038] In one embodiment, the KSS level is simplified into 5 levels, namely, the first level: LEVEL_03, the second level: LEVEL_02, the third level: LEVEL_01, the fourth level: LEVEL_00, and the fifth level: LEVEL_NONE, which respectively represent: LEVEL_NONE: alert; LEVEL_00: neither alert nor sleepy; LEVEL_01: slightly sleepy; LEVEL_02: sleepy, but can stay awake with effort; LEVEL_03: very sleepy.

[0039] In one embodiment, short-term evaluation: evaluates the immediate sleepiness state of the person to be evaluated based on short-term statistical data. Long-term evaluation: evaluates the overall fatigue trend of the person to be evaluated based on long-term statistical data.

[0040] Furthermore, based on the preset KSS sleepiness level assessment rule, obtaining the probability value of at least one of the target features in the probability value set, and comparing the probability value of each of the target features with the probability value threshold corresponding to each of the target features, before obtaining the sleepiness level of the person to be assessed, it also includes: obtaining the long-term probability value of the face existence feature based on the probability value set; when the long-term probability value of the face existence feature is less than the preset face existence probability threshold, reacquiring the image to be detected and performing a target feature probability detection, so that when the long-term probability value of the face existence feature is greater than or equal to the face existence threshold, based on the preset KSS sleepiness level assessment rule, obtaining the probability value of at least one of the target features in the probability value set, and comparing the probability value of each of the target features with the probability value threshold corresponding to each of the target features, to obtain the sleepiness level of the person to be assessed.

[0041] In one embodiment, a face existence check is first performed: if the face existence probability in the long-term data is lower than a threshold, that is, the long-term probability value of the face existence feature is less than a preset face existence probability threshold, the detection is considered to have failed and no further evaluation is performed. The image to be detected needs to be reacquired and detected.

[0042] In one embodiment, Figure 2 As shown, Figure 2 This is a flowchart for sleepiness level assessment. The sleepiness level assessment rules include face presence check. High fatigue state: If the probability of yawning in long-term data exceeds the threshold, or the probability of eyes closed exceeds the threshold, it is judged as a high fatigue state. Special fatigue state: If the probability of eyes open in long-term data is abnormally high, it may mean that the eyes have not blinked for a long time, which is judged as a special fatigue state. Multiple fatigue states: Combine short-term and long-term data to determine whether there are multiple fatigue indicators, such as the probability of closing eyes and the probability of yawning exceeding the threshold at the same time. Face position threshold exceeded state, high closed eyes state.

[0043] Furthermore, based on the preset KSS sleepiness level assessment rule, the probability value of at least one of the target features is obtained in the probability value set, and the probability value of each of the target features is compared with the probability value threshold corresponding to each of the target features to obtain the sleepiness level of the person to be assessed, including: based on the KSS sleepiness level assessment rule, in the probability value set, obtaining the long-term probability value of the yawning feature, the long-term probability value of the left eye being present and the left eye being closed, the long-term probability value of the right eye being present and the right eye being closed, the long-term probability value of the left eye being present and the left eye being open, and the long-term probability value of the right eye being present and the right eye being open. probability value, the long-term probability value of the left-leaning face feature, the long-term probability value of the right-leaning face feature, the long-term probability value of the head-lowering feature, the long-term probability value of the head-raising feature, the long-term probability value of the eye-rubbing feature, the long-term probability value of the left eye presence feature, the short-term probability value of the left eye closed feature, the long-term probability value of the right eye presence feature, and the short-term probability value of the right eye closed feature; the long-term probability value of the yawning feature, the long-term probability value of the left eye presence and the left eye closed, the long-term probability value of the right eye presence and the right eye closed, the long-term probability value of the left eye presence and the left eye open, and the long-term probability value of the right eye presence and the right eye open , respectively compared with a preset first yawning probability threshold, a preset first left eye closing probability threshold, a preset first right eye closing probability threshold, a preset first left eye opening probability value, and a preset first right eye opening probability value to obtain a first comparison result; the long-term probability value of the left-leaning face feature, the long-term probability value of the right-leaning face feature, the long-term probability value of the head-lowering feature, the long-term probability value of the head-raising feature, and the long-term probability value of the eye-rubbing feature are respectively compared with the preset first left-leaning probability threshold, the preset first right-leaning probability threshold, the preset first head-lowering probability threshold, and the preset head-raising probability threshold and a preset first eye rubbing probability threshold to obtain a second comparison result; the long-term probability value of the left eye presence feature, the short-term probability value of the left eye closed feature, the long-term probability value of the right eye presence feature and the short-term probability value of the right eye closed feature are respectively compared with the preset left eye presence probability threshold, the preset second left eye closed probability threshold, the preset right eye presence probability threshold and the preset second right eye closed probability threshold to obtain a third comparison result; based on the first comparison result, the second comparison result and the third comparison result, the sleepiness level of the person to be evaluated is determined.

[0044] In one embodiment, if the first comparison result is that the long-term probability value of the yawning feature is greater than the first yawning probability threshold, or the long-term probability value of the left eye being present and the left eye being closed is greater than the first left eye closed probability threshold, or the long-term probability value of the right eye being present and the right eye being closed is greater than the first right eye closed probability threshold, then it is determined that the person to be evaluated is in a high fatigue state and the sleepiness level is the first level. If the left eye is present and the long-term probability value of the left eye being open is greater than the first left eye open probability threshold, or the right eye is present and the long-term probability value of the right eye being open is greater than the first right eye open probability threshold, then it is determined that the person to be evaluated is in a special fatigue state and the sleepiness level is the first level.

[0045] In a specific embodiment, a very high fatigue index: if the yawning probability in longTermData exceeds a preset yawning probability threshold (such as 0.6), or the left eye exists and the probability of closing the eye exceeds a preset left eye closing probability threshold (such as 0.80), or the right eye exists and the probability of closing the eye exceeds a preset right eye closing probability threshold (such as 0.80), then it is determined to be a very high fatigue state, and the sleepiness level is increased to a maximum of LEVEL_03.

[0046] In a specific embodiment, special fatigue indication: if the left eye exists in the longTermData and the probability of opening the eye exceeds the preset left eye opening probability threshold (such as 0.995), or the right eye exists and the probability of opening the eye exceeds the preset right eye opening probability threshold (such as 0.995), it is determined to be a special fatigue state, and the sleepiness level is increased to LEVEL_03 at most. This rule is for special situations where the eyes do not blink for a long time.

[0047] In one embodiment, if the second comparison result is that the long-term probability value of the left-biased face feature is greater than the first left-biased probability threshold, or the long-term probability value of the right-biased face feature is greater than the first right-biased probability threshold, or the long-term probability value of the head-lowering feature is greater than the first head-lowering probability threshold, or the long-term probability value of the head-raising feature is greater than the first head-raising probability threshold, or the long-term probability value of the eye-rubbing feature is greater than the first eye-rubbing probability threshold, then the sleepiness level of the person to be evaluated is determined to be the first level.

[0048] In a specific embodiment, the face position threshold exceeds the limit: if the probability of the face in longTermData being tilted to the left, tilted to the right, lowering the head, raising the head, or rubbing the eyes exceeds a preset probability threshold (such as 0.8), it is determined that the face position threshold exceeds the limit, and the sleepiness level is increased to a maximum of LEVEL_03.

[0049] In one embodiment, if the third comparison result is that the long-term probability value of the left eye presence feature is greater than the preset left eye presence probability threshold and the short-term probability value of the left eye closed feature is greater than the preset second left eye closed probability threshold, or the long-term probability value of the right eye presence feature is greater than the preset right eye presence probability threshold and the short-term probability value of the right eye closed feature is greater than the preset second right eye closed probability threshold, it is determined that the sleepiness level of the person to be evaluated is the second level.

[0050] In one embodiment, high eye closing probability: if the left eye exists in longTermDat and the left eye closing probability in shortTermData exceeds a preset left eye closing probability threshold (such as 0.99), or the right eye exists and the right eye closing probability in shortTermData exceeds a preset right eye closing probability threshold (such as 0.99), the sleepiness level is increased to a maximum of LEVEL_02.

[0051] Furthermore, the method based on the preset KSS sleepiness level assessment rule obtains the probability value of at least one of the target features in the probability value set, and compares the probability value of each of the target features with the probability value threshold corresponding to each of the target features to obtain the sleepiness level of the person to be assessed, and further includes: based on the KSS sleepiness level assessment rule, comparing the long-term probability value of the left eye existing and the left eye closed, the long-term probability value of the right eye existing and the right eye closed, and the long-term probability value of the yawning feature with a preset third left eye closed probability threshold, a preset third right eye closed probability threshold, and a preset third yawning probability threshold, respectively; if the long-term probability value of the left eye existing and the left eye closed is greater than the third left eye closed probability threshold, or the long-term probability value of the right eye existing and the right eye closed is greater than the third left eye closed probability threshold, If the long-term probability value is greater than the third right eye closing probability threshold, or the long-term probability value of the yawning feature is greater than the third yawning probability threshold, then obtain the short-term probability value of the left eye closing feature, the short-term probability value of the right eye closing feature, the short-term probability value of the yawning feature and the short-term probability value of the eye rubbing feature in the probability value set; compare the short-term probability value of the left eye closing feature, the short-term probability value of the right eye closing feature, the short-term probability value of the yawning feature and the short-term probability value of the eye rubbing feature with the preset fourth left eye closing probability threshold, the preset fourth right eye closing probability threshold, the preset fourth yawning probability threshold and the preset second eye rubbing probability threshold respectively to obtain a fourth comparison result; based on the fourth comparison result, determine the sleepiness level of the person to be evaluated.

[0052] In one embodiment, if the fourth comparison result is that the short-term probability value of the left eye closed feature is greater than the fourth left eye closed probability threshold, or the short-term probability value of the right eye closed feature is greater than the fourth right eye closed probability threshold, or the short-term probability value of the yawning feature is greater than the fourth yawning probability threshold, or the short-term probability value of the eye rubbing feature is greater than the second eye rubbing probability threshold, then the sleepiness level of the person to be evaluated is determined to be the first level; otherwise, the sleepiness level of the person to be evaluated is determined to be the second level.

[0053] In a specific embodiment, multiple fatigue indications (high threshold): if the left eye exists in longTermData and the closing probability exceeds the preset left eye closing threshold (such as 0.50), or the right eye exists and the closing probability exceeds the preset right eye closing threshold (such as 0.50), or the yawning probability exceeds the preset yawning probability threshold (such as 0.4), if the left eye closing probability in shortTermData exceeds the preset left eye closing probability threshold (such as 0.8), or the right eye closing probability exceeds the preset right eye closing probability threshold (such as 0.8), or the eye rubbing probability exceeds the preset eye rubbing probability threshold (such as 0.8), or the yawning probability exceeds the preset yawning probability threshold (such as 0.8), then the sleepiness level is raised to LEVEL_03 at most. Otherwise, the sleepiness level is raised to LEVEL_02 at most.

[0054] Furthermore, the method based on the preset KSS sleepiness level assessment rule obtains the probability value of at least one of the target features in the probability value set, and compares the probability value of each of the target features with the probability value threshold corresponding to each of the target features to obtain the sleepiness level of the person to be assessed, and further includes: based on the KSS sleepiness level assessment rule, comparing the long-term probability value of the left eye existing and the left eye closed, the long-term probability value of the right eye existing and the right eye closed, and the long-term probability value of the yawning feature with the preset fifth left eye closed probability threshold, the preset fifth right eye closed probability threshold, and the preset fifth yawning probability threshold, respectively: if the long-term probability value of the left eye existing and the left eye closed is greater than the fifth left eye closed probability threshold, or the long-term probability value of the right eye existing and the right eye closed is greater than the fifth left eye closed probability threshold, If the long-term probability value is greater than the fifth right eye closing probability threshold, or the long-term probability value of the yawning feature is greater than the fifth yawning probability threshold, then obtain the short-term probability value of the left eye closing feature, the short-term probability value of the right eye closing feature, the short-term probability value of the yawning feature and the short-term probability value of the eye rubbing feature in the probability value set; compare the short-term probability value of the left eye closing feature, the short-term probability value of the right eye closing feature, the short-term probability value of the yawning feature and the short-term probability value of the eye rubbing feature with the preset sixth left eye closing probability threshold, the preset sixth right eye closing probability threshold, the preset sixth yawning probability threshold and the preset third eye rubbing probability threshold respectively to obtain a fifth comparison result; based on the fifth comparison result, determine the sleepiness level of the person to be evaluated.

[0055] In one embodiment, if the fifth comparison result is that the short-term probability value of the left eye closed feature is greater than the sixth left eye closed probability threshold, or the short-term probability value of the right eye closed feature is greater than the sixth right eye closed probability threshold, or the short-term probability value of the yawning feature is greater than the sixth yawning probability threshold, or the short-term probability value of the eye rubbing feature is greater than the third eye rubbing probability threshold, then the sleepiness level of the person to be evaluated is determined to be the second level; otherwise, the sleepiness level of the person to be evaluated is determined to be the third level.

[0056] In a specific embodiment, multiple fatigue indications (middle threshold): if the left eye exists in longTermData and the closing probability exceeds the preset left eye closing threshold (such as 0.30), or the right eye exists and the closing probability exceeds the preset right eye closing probability threshold (such as 0.30), or the yawning probability exceeds the preset yawning probability threshold (such as 0.3), if the left eye closing probability in shortTermData exceeds the preset left eye closing probability threshold (such as 0.8), or the right eye closing probability exceeds the preset right eye closing probability threshold (such as 0.8), or the eye rubbing probability exceeds the preset eye rubbing probability threshold (such as 0.8), or the yawning probability exceeds the preset yawning probability threshold (such as 0.8), then the sleepiness level is raised to LEVEL_02 at most. Otherwise, the sleepiness level is raised to LEVEL_01 at most.

[0057] Furthermore, the method based on the preset KSS sleepiness level assessment rule obtains the probability value of at least one of the target features in the probability value set, and compares the probability value of each of the target features with the probability value threshold corresponding to each of the target features to obtain the sleepiness level of the person to be assessed, and further includes: based on the KSS sleepiness level assessment rule, comparing the long-term probability value of the left eye existing and the left eye closed, the long-term probability value of the right eye existing and the right eye closed, and the long-term probability value of the yawning feature with the preset seventh left eye closed probability threshold, the preset seventh right eye closed probability threshold, and the preset seventh yawning probability threshold, respectively: if the long-term probability value of the left eye existing and the left eye closed is greater than the seventh left eye closed probability threshold, or the long-term probability value of the right eye existing and the right eye closed is greater than the seventh left eye closed probability threshold, If the long-term probability value is greater than the seventh right eye closing probability threshold, or the long-term probability value of the yawning feature is greater than the seventh yawning probability threshold, then obtain the short-term probability value of the left eye closing feature, the short-term probability value of the right eye closing feature, the short-term probability value of the yawning feature and the short-term probability value of the eye rubbing feature in the probability value set; compare the short-term probability value of the left eye closing feature, the short-term probability value of the right eye closing feature, the short-term probability value of the yawning feature and the short-term probability value of the eye rubbing feature with the preset eighth left eye closing probability threshold, the preset eighth right eye closing probability threshold, the preset eighth yawning probability threshold and the preset fourth eye rubbing probability threshold respectively to obtain a sixth comparison result; based on the sixth comparison result, determine the sleepiness level of the person to be evaluated.

[0058] In one embodiment, if the sixth comparison result is that the short-term probability value of the left eye closed feature is greater than the eighth left eye closed probability threshold, or the short-term probability value of the right eye closed feature is greater than the eighth right eye closed probability threshold, or the short-term probability value of the yawning feature is greater than the eighth yawning probability threshold, or the short-term probability value of the eye rubbing feature is greater than the fourth eye rubbing probability threshold, then the sleepiness level of the person to be evaluated is determined to be the third level; otherwise, the sleepiness level of the person to be evaluated is determined to be the fourth level.

[0059] In a specific embodiment, multiple fatigue indications (low threshold): if the left eye exists in longTermData and the closing probability exceeds the preset left eye closing threshold (such as 0.20), or the right eye exists and the closing probability exceeds the preset right eye closing threshold (such as 0.20), or the yawning probability exceeds 0.2, if the left eye closing probability in shortTermData exceeds the preset left eye closing threshold (such as 0.8), or the right eye closing probability exceeds the preset right eye closing threshold (such as 0.8), or the eye rubbing probability exceeds the preset eye rubbing probability threshold (such as 0.8), or the yawning probability exceeds the preset yawning probability threshold (such as 0.8), then the sleepiness level is increased to a maximum of LEVEL_01. Otherwise, the sleepiness level is increased to a maximum of LEVEL_00.

[0060] Furthermore, the method based on the preset KSS sleepiness level assessment rule obtains the probability value of at least one of the target features in the probability value set, and compares the probability value of each of the target features with the probability value threshold corresponding to each of the target features to obtain the sleepiness level of the person to be assessed, and also includes: based on the KSS sleepiness level assessment rule, obtaining the short-term probability value of the left face feature, the short-term probability value of the right face feature, the short-term probability value of the head lowering feature, and the short-term probability value of the head raising feature; comparing the long-term probability value of the eye rubbing feature, the long-term probability value of the yawning feature, the long-term probability value of the left eye existing and the left eye closed, the long-term probability value of the right eye existing and the right eye closed, the long-term probability value of the left face feature, the long-term probability value of the right face feature, the long-term probability value of the head lowering feature, the long-term probability value of the head raising feature, the long-term probability value of the left eye closed The short-term probability value of the eye feature, the short-term probability value of the right eye closed feature, the short-term probability value of the left face feature, the short-term probability value of the right face feature, the short-term probability value of the head lowering feature and the short-term probability value of the head raising feature are respectively compared with the preset sixth eye rubbing probability threshold, the preset ninth yawning probability threshold, the preset ninth left eye closing threshold, the preset ninth right eye closing threshold, the preset second left probability threshold, the preset second right probability threshold, the preset second head lowering probability threshold, the preset second head raising probability threshold, the preset tenth left eye closing probability threshold, the preset tenth right eye closing probability threshold, the preset third left probability threshold, the preset third right probability threshold, the preset third head lowering probability threshold, and the preset third head raising probability threshold to obtain a seventh comparison result; based on the seventh comparison result, the sleepiness level of the person to be evaluated is determined.

[0061] In one embodiment, if the seventh comparison result is that the long-term probability value of the yawning feature is greater than the ninth yawning probability threshold, and the long-term probability value of the eye rubbing feature is greater than the sixth eye rubbing probability threshold, and the long-term probability value of the left eye being present and the left eye being closed is greater than the ninth left eye closed probability threshold, or the long-term probability value of the right eye being present and the right eye being closed is greater than the ninth right eye closed probability threshold, or the long-term probability value of the left-leaning face feature is greater than the second left-leaning face probability threshold, or the long-term probability value of the right-leaning face feature is greater than the second right-leaning face probability threshold, or the long-term probability value of the head-lowering feature is greater than the second head-lowering face probability threshold; If the probability threshold or the long-term probability value of the head-raising feature is greater than the second head-raising probability threshold, or the short-term probability value of the left eye closed feature is greater than the tenth left eye closed probability threshold, or the short-term probability value of the right eye closed feature is greater than the tenth right eye closed probability threshold, or the short-term probability value of the left face feature is greater than the third left face probability threshold, or the short-term probability value of the right face feature is greater than the third right face probability threshold, or the short-term probability value of the head-lowering feature is greater than the third head-lowering probability threshold, or the short-term probability value of the head-raising feature is greater than the third head-raising probability threshold, then the sleepiness level of the person to be evaluated is determined to be the third level.

[0062] In a specific embodiment, the eye closing or yawning threshold exceeds the limit: if the eye rubbing probability in longTermData exceeds the preset eye rubbing probability threshold (such as 0.05), and the yawning probability exceeds the preset yawning probability threshold (such as 0.05), and the left eye exists and the eye closing probability exceeds the preset left eye closing threshold (such as 0.1), or the right eye exists and the eye closing probability exceeds the preset right eye closing threshold (such as 0.1), or the probability of the face leaning to the left, right, lowering the head, or raising the head exceeds the preset probability threshold 0.1, or the left eye closing probability in shortTermData exceeds the preset left eye closing threshold (such as 0.6), or the right eye closing probability exceeds the preset right eye closing threshold (such as 0.6), or the probability of the face leaning to the left, right, lowering the head, or raising the head exceeds the preset probability threshold (such as 0.8), then the sleepiness level is increased to LEVEL_01 at most. This rule mainly considers the situation where multiple fatigue characteristics occur at the same time, but the probability value is low.

[0063] The above-mentioned embodiment provides a KSS sleepiness level assessment method, apparatus, computer equipment and storage medium, which obtains continuous facial images of a person to be assessed as images to be detected; detects each frame of the image to be detected to obtain face detection information, eye detection information, expression detection information and face angle detection information; based on a preset threshold, performs sliding window statistics on the target features present in the face detection information, the eye detection information, the expression detection information and the face angle detection information, respectively, obtains the probability value of each target feature, and generates a probability value set; based on a preset KSS sleepiness level assessment rule, obtains the probability value of at least one target feature in the probability value set, and compares the probability value of each target feature with the probability value threshold corresponding to each target feature to obtain the sleepiness level of the person to be assessed. The present application can detect the facial image of the person to be evaluated according to the preset sleepiness level influencing factors, obtain the detection results, and perform sliding window statistics on the detection results to obtain accurate probability values ​​of target features, and generate a set of target feature probability values, and finally compare the target feature probability values ​​with the preset KSS sleepiness level evaluation rules to obtain the final KSS sleepiness level evaluation results. The sleepiness level influencing factors can be set as needed, and the sleepiness level of the person to be evaluated can be comprehensively evaluated by combining multiple parameters. The introduction of the KSS sleepiness level evaluation rules can be carried out by performing statistical analysis and conditional judgment on a variety of data, and a comprehensive judgment can be made based on a variety of fatigue indication parameters, so as to more accurately evaluate the driver's fatigue level and issue warnings in time; eliminate accidental errors and improve the accuracy and stability of the evaluation. Secondly, the present application only needs to collect the image to be detected, detect the image to be detected, and perform sliding window statistics on the detection results, which reduces the complexity of the KSS sleepiness level evaluation.

[0064] See also Figure 3 , Figure 3 This is a schematic flow chart of a KSS sleepiness level assessment method provided in an embodiment of the present application. The KSS sleepiness level assessment method can be applied to a server to dynamically adjust the eye closure threshold according to the actual situation of the person to be assessed, adapt to different persons to be assessed and eyes of different sizes, and improve the sensitivity and accuracy of fatigue detection.

[0065] like Figure 3 As shown, the KSS sleepiness level evaluation method specifically includes steps S201 to S204.

[0066] S201, acquiring left eye aspect ratio data and right eye aspect ratio data of the person to be evaluated based on the human eye detection information; S202, performing frequency distribution analysis on the left eye aspect ratio data and the right eye aspect ratio data respectively to obtain a first target aspect ratio interval corresponding to the left eye and a second target aspect ratio interval corresponding to the right eye; S203, obtaining a first target eye closing threshold and a second target eye closing threshold based on a preset proportionality coefficient, a first median value of the first target aspect ratio interval, and a second median value of the second target aspect ratio interval; S204. When the first target eye-closing threshold is within the preset eye-closing threshold range, the left eye-closing threshold is updated to the first target eye-closing threshold; and, when the second target eye-closing threshold is within the eye-closing threshold range, the right eye-closing threshold is updated to the second target eye-closing threshold.

[0067] In one embodiment, based on a preset proportionality coefficient and the first target aspect ratio interval and the second target aspect ratio interval, a first target eye closing threshold and a second target eye closing threshold are obtained respectively, including: obtaining a first median value of the first target aspect ratio interval and a second median value of the second target aspect ratio interval respectively; obtaining the first target eye closing threshold based on the first median value and the preset proportionality coefficient; obtaining the second target eye closing threshold based on the second median value and the preset proportionality coefficient.

[0068] In one embodiment, the eye closing threshold is dynamically adjusted based on the real-time detected eye aspect ratio data to improve the accuracy of fatigue detection. The eye closing threshold suitable for the current person to be evaluated is determined by statistically analyzing the frequency distribution of the eye aspect ratio.

[0069] In a specific embodiment, Figure 4 As shown, Figure 4This is a flowchart showing the dynamic learning of the eye closing threshold. 1. Initialization: During initialization, set the statistical interval and the number of statistical sliding window frames. The statistical interval refers to the unit interval of the eye size score. Set the default eye closing threshold range, for example, the eye opening threshold is greater than 0.2, otherwise it is eye closing.

[0070] 2. Data update: The aspect ratio data of the left and right eyes are collected in real time and stored separately according to the eye type (left eye or right eye). The sliding window mechanism is used to store the eye aspect ratio data in the recent period to ensure the real-time and validity of the statistical data.

[0071] 3. Dynamic threshold calculation: In the sliding window, perform frequency distribution analysis on the collected eye aspect ratio data to find the most frequently occurring aspect ratio interval as the target aspect ratio interval, where the left eye corresponds to the first target aspect ratio interval and the right eye corresponds to the second target aspect ratio interval. Calculate the dynamic eye closing threshold based on the aspect ratio interval with the highest frequency. The specific method is to multiply the median of the interval by a preset ratio (such as 0.6) to obtain a new eye closing threshold. Make sure that the calculated eye closing threshold is within the preset eye closing threshold range (such as between 0.1 and 0.2). If the conditions are met, update the dynamic eye closing threshold. If it is not within the preset threshold range, it is an abnormal statistical value and should be discarded.

[0072] 4. Threshold update: Calculate and update the dynamic threshold for the left eye and the right eye respectively to ensure that the threshold of each eye can be adjusted independently to adapt to different eye characteristics. The calculated dynamic threshold is output for subsequent fatigue detection.

[0073] In the above embodiment, the eye closing threshold can be dynamically adjusted according to the actual situation of the person to be evaluated, so as to adapt to different persons to be evaluated and eyes of different sizes, thereby improving the sensitivity and accuracy of fatigue detection.

[0074] See also Figure 5 , Figure 5 The embodiment of the present application provides a schematic block diagram of a KSS sleepiness level assessment device, which is used to execute the aforementioned KSS sleepiness level assessment method. The KSS sleepiness level assessment device can be configured on a server.

[0075] like Figure 5 As shown, the KSS sleepiness level assessment device 400 comprises: The image acquisition module 401 is used to acquire continuous facial images of the person to be evaluated as images to be detected; The detection information acquisition module 402 is used to detect each frame of the image to be detected and obtain face detection information, eye detection information, expression detection information and face angle detection information; The probability value set acquisition module 403 is used to perform sliding window statistics on the target features present in the face detection information, the eye detection information, the expression detection information and the face angle detection information based on a preset threshold, obtain the probability value of each target feature, and generate a probability value set; The KSS sleepiness level assessment module 404 is used to obtain the probability value of at least one of the target features in the probability value set based on a preset KSS sleepiness level assessment rule, and compare the probability value of each of the target features with the probability value threshold corresponding to each of the target features to obtain the sleepiness level of the person to be assessed.

[0076] Furthermore, the preset threshold includes a left eye closed threshold and a right eye closed threshold, and the KSS sleepiness level evaluation device further includes a dynamic threshold generation module, and the dynamic threshold generation module includes: A data acquisition unit, configured to acquire left eye aspect ratio data and right eye aspect ratio data of the person to be evaluated based on the human eye detection information; an interval determination unit, configured to perform frequency distribution analysis on the left eye aspect ratio data and the right eye aspect ratio data, respectively, to obtain a first target aspect ratio interval corresponding to the left eye and a second target aspect ratio interval corresponding to the right eye; a threshold value obtaining unit, configured to obtain a first target eye closing threshold value and a second target eye closing threshold value respectively based on a preset proportionality coefficient and a first median value of the first target aspect ratio interval and a second median value of the second target aspect ratio interval; A threshold updating unit is used to update the left eye closing threshold to the first target eye closing threshold when the first target eye closing threshold falls within a preset eye closing threshold range, and to update the right eye closing threshold to the second target eye closing threshold when the second target eye closing threshold falls within the eye closing threshold range.

[0077] Furthermore, the target features include face presence feature, yawning feature, face leaning to the left feature, face leaning to the right feature, head lowering feature, head raising feature, left eye presence feature, left eye closed feature, left eye open feature, right eye presence feature, right eye closed feature, right eye open feature and eye rubbing feature.

[0078] Furthermore, the probability value set obtaining module 403 includes: An information adding unit, used for adding the face detection information, the eye detection information, the expression detection information and the face angle detection information of each of the images to be detected into a preset short-term sliding window and a long-term sliding window respectively according to the number of frames of the images to be detected; a sliding window statistics unit, configured to judge and count whether each of the target features exists in the face detection information, the eye detection information, the expression detection information, and the face angle detection information, based on the short-term sliding window, the long-term sliding window, and the preset threshold, to obtain a statistical result of the short-term sliding window and a statistical result of the long-term sliding window; The probability value obtaining unit is used to perform probability calculation based on the statistical results of the short-term sliding window and the statistical results of the long-term sliding window, respectively, to obtain the short-term probability value of each target feature in the short-term sliding window and the long-term probability value of each target feature in the long-term sliding window.

[0079] Furthermore, the KSS sleepiness level assessment device further includes: A probability value acquisition unit, used to acquire a long-term probability value of the presence feature of the face based on the probability value set; The probability value comparison unit is used to reacquire the image to be detected and perform target feature probability detection when the long-term probability value of the face existence feature is less than a preset face existence probability threshold, so as to obtain the probability value of at least one target feature in the probability value set based on the preset KSS sleepiness level assessment rule when the long-term probability value of the face existence feature is greater than or equal to the face existence threshold, and compare the probability value of each target feature with the probability value threshold corresponding to each target feature to obtain the sleepiness level of the person to be assessed.

[0080] Furthermore, the KSS sleepiness level assessment module 404 includes: a probability value acquisition unit, for acquiring, based on the KSS sleepiness level evaluation rule, from the probability value set, a long-term probability value of a yawning feature, a long-term probability value of a left eye existing and closed, a long-term probability value of a right eye existing and closed, a long-term probability value of a left eye existing and open, a long-term probability value of a right eye existing and open, a long-term probability value of a face leaning to the left feature, a long-term probability value of a face leaning to the right feature, a long-term probability value of a head lowering feature, a long-term probability value of a head raising feature, a long-term probability value of an eye rubbing feature, a long-term probability value of a left eye existing feature, a short-term probability value of a left eye closed feature, a long-term probability value of a right eye existing feature, and a short-term probability value of a right eye closed feature; a first comparison result obtaining unit, for comparing the long-term probability value of the yawning feature, the long-term probability value of the left eye existing and the left eye closed, the long-term probability value of the right eye existing and the right eye closed, the long-term probability value of the left eye existing and the left eye open, and the long-term probability value of the right eye existing and the right eye open, respectively, with a preset first yawning probability threshold, a preset first left eye closed probability threshold, a preset first right eye closed probability threshold, a preset first left eye open probability value, and a preset first right eye open probability value to obtain a first comparison result; a second comparison result obtaining unit, for comparing the long-term probability value of the left-leaning face feature, the long-term probability value of the right-leaning face feature, the long-term probability value of the head-lowering feature, the long-term probability value of the head-raising feature, and the long-term probability value of the eye-rubbing feature with a preset first left-leaning probability threshold, a preset first right-leaning probability threshold, a preset first head-lowering probability threshold, a preset head-raising probability threshold, and a preset first eye-rubbing probability threshold, respectively, to obtain a second comparison result; a third comparison result obtaining unit, configured to compare the long-term probability value of the left eye presence feature, the short-term probability value of the left eye closed feature, the long-term probability value of the right eye presence feature, and the short-term probability value of the right eye closed feature with a preset left eye presence probability threshold, a preset second left eye closed probability threshold, a preset right eye presence probability threshold, and a preset second right eye closed probability threshold, respectively, to obtain a third comparison result; A sleepiness level determination unit is used to determine the sleepiness level of the person to be evaluated based on the first comparison result, the second comparison result and the third comparison result.

[0081] Furthermore, the KSS sleepiness level assessment module 404 further includes: a probability value comparison unit, for comparing the long-term probability value of the left eye being present and the left eye being closed, the long-term probability value of the right eye being present and the right eye being closed, and the long-term probability value of the yawning feature with a preset third left eye closed probability threshold, a preset third right eye closed probability threshold, and a preset third yawning probability threshold, respectively, based on the KSS sleepiness level evaluation rule; a probability value acquisition unit, configured to acquire, from the probability value set, a short-term probability value of a left eye closed feature, a short-term probability value of a right eye closed feature, a short-term probability value of a yawning feature, and a short-term probability value of an eye rubbing feature if the left eye exists and the long-term probability value of the left eye closed is greater than the third left eye closed probability threshold, or the right eye exists and the long-term probability value of the right eye closed is greater than the third right eye closed probability threshold, or the long-term probability value of the yawning feature is greater than the third yawning probability threshold; a fourth comparison result obtaining unit, configured to compare the short-term probability value of the left eye closing feature, the short-term probability value of the right eye closing feature, the short-term probability value of the yawning feature, and the short-term probability value of the eye rubbing feature with a preset fourth left eye closing probability threshold, a preset fourth right eye closing probability threshold, a preset fourth yawning probability threshold, and a preset second eye rubbing probability threshold, respectively, to obtain a fourth comparison result; A sleepiness level determination unit is used to determine the sleepiness level of the person to be evaluated based on the fourth comparison result.

[0082] Furthermore, the KSS sleepiness level assessment module 404 further includes: a probability value comparison unit, for comparing the long-term probability value of the left eye existing and the left eye closed, the long-term probability value of the right eye existing and the right eye closed, and the long-term probability value of the yawning feature with a preset fifth left eye closed probability threshold, a preset fifth right eye closed probability threshold, and a preset fifth yawning probability threshold, respectively, based on the KSS sleepiness level evaluation rule: a probability value acquisition unit, configured to acquire, from the probability value set, a short-term probability value of a left eye closed feature, a short-term probability value of a right eye closed feature, a short-term probability value of a yawning feature, and a short-term probability value of an eye rubbing feature if the left eye exists and the long-term probability value of the left eye closed is greater than the fifth left eye closed probability threshold, or the right eye exists and the long-term probability value of the right eye closed is greater than the fifth right eye closed probability threshold, or the long-term probability value of the yawning feature is greater than the fifth yawning probability threshold; a fifth comparison result obtaining unit, configured to compare the short-term probability value of the left eye closing feature, the short-term probability value of the right eye closing feature, the short-term probability value of the yawning feature, and the short-term probability value of the eye rubbing feature with a preset sixth left eye closing probability threshold, a preset sixth right eye closing probability threshold, a preset sixth yawning probability threshold, and a preset third eye rubbing probability threshold, respectively, to obtain a fifth comparison result; The sleepiness level determining unit is used to determine the sleepiness level of the person to be evaluated based on the fifth comparison result.

[0083] Furthermore, the KSS sleepiness level assessment module 404 further includes: a probability value comparison unit, for comparing the long-term probability value of the left eye existing and the left eye closed, the long-term probability value of the right eye existing and the right eye closed, and the long-term probability value of the yawning feature with a preset seventh left eye closed probability threshold, a preset seventh right eye closed probability threshold, and a preset seventh yawning probability threshold, respectively, based on the KSS sleepiness level evaluation rule: a probability value acquisition unit, configured to acquire, from the probability value set, a short-term probability value of a left eye closed feature, a short-term probability value of a right eye closed feature, a short-term probability value of a yawning feature, and a short-term probability value of an eye rubbing feature if the left eye exists and the long-term probability value of the left eye closed is greater than the seventh left eye closed probability threshold, or the right eye exists and the long-term probability value of the right eye closed is greater than the seventh right eye closed probability threshold, or the long-term probability value of the yawning feature is greater than the seventh yawning probability threshold; a sixth comparison result obtaining unit, configured to compare the short-term probability value of the left eye closing feature, the short-term probability value of the right eye closing feature, the short-term probability value of the yawning feature, and the short-term probability value of the eye rubbing feature with a preset eighth left eye closing probability threshold, a preset eighth right eye closing probability threshold, a preset eighth yawning probability threshold, and a preset fourth eye rubbing probability threshold, respectively, to obtain a sixth comparison result; The sleepiness level determination unit is used to determine the sleepiness level of the person to be evaluated based on the sixth comparison result.

[0084] Furthermore, the KSS sleepiness level assessment module 404 further includes: A probability value acquisition unit, used to acquire, based on the KSS sleepiness level assessment rule, a short-term probability value of a left-leaning face feature, a short-term probability value of a right-leaning face feature, a short-term probability value of a head-down feature, and a short-term probability value of a head-up feature; The seventh comparison result obtaining unit is used to convert the long-term probability value of the eye rubbing feature, the long-term probability value of the yawning feature, the long-term probability value of the left eye existing and the left eye closed, the long-term probability value of the right eye existing and the right eye closed, the long-term probability value of the left face feature, the long-term probability value of the right face feature, the long-term probability value of the head lowering feature, the long-term probability value of the head raising feature, the short-term probability value of the left eye closed feature, the short-term probability value of the right eye closed feature, the short-term probability value of the left face feature, the short-term probability value of the right face feature, and the short-term probability value of the head lowering feature. The value and the short-term probability value of the head-raising feature are respectively compared with the preset sixth eye rubbing probability threshold, the preset ninth yawning probability threshold, the preset ninth left eye closing threshold, the preset ninth right eye closing threshold, the preset second left probability threshold, the preset second right probability threshold, the preset second head-lowering probability threshold, the preset second head-raising probability threshold, the preset tenth left eye closing probability threshold, the preset tenth right eye closing probability threshold, the preset third left probability threshold, the preset third right probability threshold, the preset third head-lowering probability threshold, and the preset third head-raising probability threshold to obtain a seventh comparison result; A sleepiness level determination unit is used to determine the sleepiness level of the person to be evaluated based on the seventh comparison result.

[0085] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and each module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0086] The above-mentioned device can be implemented in the form of a computer program. Figure 6 Runs on the computer device shown.

[0087] See also Figure 6 , Figure 6 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device may be a server.

[0088] See also Figure 6 The computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0089] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any one of the KSS sleepiness level assessment methods.

[0090] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0091] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any one of the KSS sleepiness level assessment methods.

[0092] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0093] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0094] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps: Acquire continuous facial images of the person to be evaluated as images to be detected; Detect each frame of the image to be detected to obtain face detection information, eye detection information, expression detection information and face angle detection information; Based on a preset threshold, respectively, sliding window statistics are performed on the target features present in the face detection information, the eye detection information, the expression detection information, and the face angle detection information to obtain a probability value of each of the target features, and generate a probability value set; Based on the preset KSS sleepiness level assessment rule, the probability value of at least one of the target features is obtained in the probability value set, and the probability value of each of the target features is compared with the probability value threshold corresponding to each of the target features to obtain the sleepiness level of the person to be assessed.

[0095] In one embodiment, the preset threshold includes a left eye closed threshold and a right eye closed threshold. Before the processor implements sliding window statistics on the target features present in the face detection information, the eye detection information, the expression detection information, and the face angle detection information based on the preset threshold, obtains the probability value of each of the target features, and generates a probability value set, it is also used to implement: Based on the human eye detection information, acquiring left eye aspect ratio data and right eye aspect ratio data of the person to be evaluated; Performing frequency distribution analysis on the left eye aspect ratio data and the right eye aspect ratio data respectively to obtain a first target aspect ratio interval corresponding to the left eye and a second target aspect ratio interval corresponding to the right eye; Based on a preset proportionality coefficient and a first median value of the first target aspect ratio interval and a second median value of the second target aspect ratio interval, respectively obtaining a first target eye closing threshold value and a second target eye closing threshold value; When the first target eye closing threshold is within the preset eye closing threshold range, the left eye closing threshold is updated to the first target eye closing threshold, and when the second target eye closing threshold is within the eye closing threshold range, the right eye closing threshold is updated to the second target eye closing threshold.

[0096] In one embodiment, the target features include a face presence feature, a yawning feature, a face leaning to the left feature, a face leaning to the right feature, a head lowering feature, a head raising feature, a left eye presence feature, a left eye closed feature, a left eye open feature, a right eye presence feature, a right eye closed feature, a right eye open feature, and an eye rubbing feature.

[0097] In one embodiment, the processor performs sliding window statistics on the target features present in the face detection information, the eye detection information, the expression detection information, and the face angle detection information based on a preset threshold to obtain a probability value of each of the target features, and is used to implement: According to the number of frames of the image to be detected, the face detection information, the eye detection information, the expression detection information and the face angle detection information of each image to be detected are added to a preset short-term sliding window and a long-term sliding window respectively; Based on the short-term sliding window, the long-term sliding window and the preset threshold, judging and counting whether each of the target features exists in the face detection information, the eye detection information, the expression detection information and the face angle detection information, respectively, to obtain the statistical results of the short-term sliding window and the statistical results of the long-term sliding window; Probability calculation is performed based on the statistical result of the short-term sliding window and the statistical result of the long-term sliding window, respectively, to obtain a short-term probability value of each target feature in the short-term sliding window and a long-term probability value of each target feature in the long-term sliding window.

[0098] In one embodiment, before the processor implements the preset KSS sleepiness level assessment rule, obtains the probability value of at least one of the target features in the probability value set, and compares the probability value of each of the target features with the probability value threshold corresponding to each of the target features to obtain the sleepiness level of the person to be assessed, it is also used to implement: Based on the probability value set, obtaining a long-term probability value of the presence feature of the face; When the long-term probability value of the face existence feature is less than the preset face existence probability threshold, the image to be detected is reacquired, and the target feature probability is detected, so that when the long-term probability value of the face existence feature is greater than or equal to the face existence threshold, based on the preset KSS sleepiness level assessment rule, the probability value of at least one target feature is obtained in the probability value set, and the probability value of each target feature is compared with the probability value threshold corresponding to each target feature to obtain the sleepiness level of the person to be assessed.

[0099] In one embodiment, the processor, when implementing a preset KSS sleepiness level assessment rule, obtains a probability value of at least one of the target features in the probability value set, and compares the probability value of each of the target features with the probability value threshold corresponding to each of the target features to obtain the sleepiness level of the person to be assessed, is used to implement: Based on the KSS sleepiness level assessment rule, in the probability value set, obtain the long-term probability value of the yawning feature, the long-term probability value of the left eye existing and the left eye closed, the long-term probability value of the right eye existing and the right eye closed, the long-term probability value of the left eye existing and the left eye open, the long-term probability value of the right eye existing and the right eye open, the long-term probability value of the face leaning to the left feature, the long-term probability value of the face leaning to the right feature, the long-term probability value of the head lowering feature, the long-term probability value of the head raising feature, the long-term probability value of the eye rubbing feature, the long-term probability value of the left eye existing feature, the short-term probability value of the left eye closed feature, the long-term probability value of the right eye existing feature, and the short-term probability value of the right eye closed feature; The long-term probability value of the yawning feature, the long-term probability value of the left eye existing and the left eye closed, the long-term probability value of the right eye existing and the right eye closed, the long-term probability value of the left eye existing and the left eye open, and the long-term probability value of the right eye existing and the right eye open are respectively compared with a preset first yawning probability threshold, a preset first left eye closed probability threshold, a preset first right eye closed probability threshold, a preset first left eye open probability value, and a preset first right eye open probability value to obtain a first comparison result; Compare the long-term probability value of the left-leaning face feature, the long-term probability value of the right-leaning face feature, the long-term probability value of the head-lowering feature, the long-term probability value of the head-raising feature, and the long-term probability value of the eye-rubbing feature with a preset first left-leaning probability threshold, a preset first right-leaning probability threshold, a preset first head-lowering probability threshold, a preset head-raising probability threshold, and a preset first eye-rubbing probability threshold, respectively, to obtain a second comparison result; Comparing the long-term probability value of the left eye presence feature, the short-term probability value of the left eye closed feature, the long-term probability value of the right eye presence feature, and the short-term probability value of the right eye closed feature with a preset left eye presence probability threshold, a preset second left eye closed probability threshold, a preset right eye presence probability threshold, and a preset second right eye closed probability threshold, respectively, to obtain a third comparison result; The sleepiness level of the person to be evaluated is determined based on the first comparison result, the second comparison result, and the third comparison result.

[0100] In one embodiment, the processor, when implementing a preset KSS sleepiness level assessment rule, obtains a probability value of at least one of the target features in the probability value set, and compares the probability value of each of the target features with the probability value threshold corresponding to each of the target features to obtain the sleepiness level of the person to be assessed, is further used to implement: Based on the KSS sleepiness level assessment rule, the long-term probability value of the left eye being present and the left eye being closed, the long-term probability value of the right eye being present and the right eye being closed, and the long-term probability value of the yawning feature are respectively compared with a preset third left eye closed probability threshold, a preset third right eye closed probability threshold, and a preset third yawning probability threshold; If the left eye exists and the long-term probability value of the left eye closed is greater than the third left eye closed probability threshold, or the right eye exists and the long-term probability value of the right eye closed is greater than the third right eye closed probability threshold, or the long-term probability value of the yawning feature is greater than the third yawning probability threshold, then the short-term probability value of the left eye closed feature, the short-term probability value of the right eye closed feature, the short-term probability value of the yawning feature, and the short-term probability value of the eye rubbing feature are obtained from the probability value set; comparing the short-term probability value of the left eye closed feature, the short-term probability value of the right eye closed feature, the short-term probability value of the yawning feature, and the short-term probability value of the eye rubbing feature with a preset fourth left eye closed probability threshold, a preset fourth right eye closed probability threshold, a preset fourth yawning probability threshold, and a preset second eye rubbing probability threshold, respectively, to obtain a fourth comparison result; Based on the fourth comparison result, the sleepiness level of the person to be evaluated is determined.

[0101] In one embodiment, the processor, when implementing a preset KSS sleepiness level assessment rule, obtains a probability value of at least one of the target features in the probability value set, and compares the probability value of each of the target features with the probability value threshold corresponding to each of the target features to obtain the sleepiness level of the person to be assessed, is further used to implement: Based on the KSS sleepiness level assessment rule, the long-term probability value of the left eye being present and the left eye being closed, the long-term probability value of the right eye being present and the right eye being closed, and the long-term probability value of the yawning feature are compared with the preset fifth left eye closed probability threshold, the preset fifth right eye closed probability threshold, and the preset fifth yawning probability threshold, respectively: If the left eye exists and the long-term probability value of the left eye closed is greater than the fifth left eye closed probability threshold, or the right eye exists and the long-term probability value of the right eye closed is greater than the fifth right eye closed probability threshold, or the long-term probability value of the yawning feature is greater than the fifth yawning probability threshold, then the short-term probability value of the left eye closed feature, the short-term probability value of the right eye closed feature, the short-term probability value of the yawning feature, and the short-term probability value of the eye rubbing feature are obtained from the probability value set; comparing the short-term probability value of the left eye closed feature, the short-term probability value of the right eye closed feature, the short-term probability value of the yawning feature, and the short-term probability value of the eye rubbing feature with a preset sixth left eye closed probability threshold, a preset sixth right eye closed probability threshold, a preset sixth yawning probability threshold, and a preset third eye rubbing probability threshold, respectively, to obtain a fifth comparison result; Based on the fifth comparison result, the sleepiness level of the person to be evaluated is determined.

[0102] In one embodiment, the processor, when implementing a preset KSS sleepiness level assessment rule, obtains a probability value of at least one of the target features in the probability value set, and compares the probability value of each of the target features with the probability value threshold corresponding to each of the target features to obtain the sleepiness level of the person to be assessed, is further used to implement: Based on the KSS sleepiness level assessment rule, the long-term probability value of the left eye being present and the left eye being closed, the long-term probability value of the right eye being present and the right eye being closed, and the long-term probability value of the yawning feature are compared with the seventh preset left eye closed probability threshold, the seventh preset right eye closed probability threshold, and the seventh preset yawning probability threshold, respectively: If the left eye exists and the long-term probability value of the left eye closed is greater than the seventh left eye closed probability threshold, or the right eye exists and the long-term probability value of the right eye closed is greater than the seventh right eye closed probability threshold, or the long-term probability value of the yawning feature is greater than the seventh yawning probability threshold, then the short-term probability value of the left eye closed feature, the short-term probability value of the right eye closed feature, the short-term probability value of the yawning feature, and the short-term probability value of the eye rubbing feature are obtained from the probability value set; comparing the short-term probability value of the left eye closed feature, the short-term probability value of the right eye closed feature, the short-term probability value of the yawning feature, and the short-term probability value of the eye rubbing feature with a preset eighth left eye closed probability threshold, a preset eighth right eye closed probability threshold, a preset eighth yawning probability threshold, and a preset fourth eye rubbing probability threshold, respectively, to obtain a sixth comparison result; Based on the sixth comparison result, the sleepiness level of the person to be evaluated is determined.

[0103] In one embodiment, the processor, when implementing a preset KSS sleepiness level assessment rule, obtains a probability value of at least one of the target features in the probability value set, and compares the probability value of each of the target features with the probability value threshold corresponding to each of the target features to obtain the sleepiness level of the person to be assessed, is further used to implement: Based on the KSS sleepiness level assessment rule, a short-term probability value of a left-leaning face feature, a short-term probability value of a right-leaning face feature, a short-term probability value of a head-down feature, and a short-term probability value of a head-up feature are obtained; The long-term probability value of the eye rubbing feature, the long-term probability value of the yawning feature, the long-term probability value of the left eye existing and the left eye closed, the long-term probability value of the right eye existing and the right eye closed, the long-term probability value of the left face feature, the long-term probability value of the right face feature, the long-term probability value of the head lowering feature, the long-term probability value of the head raising feature, the short-term probability value of the left eye closed feature, the short-term probability value of the right eye closed feature, the short-term probability value of the left face feature, the short-term probability value of the right face feature, the short-term probability value of the head lowering feature and the short-term probability value of the head raising feature are calculated. The short-term probability value of the head feature is respectively compared with the preset sixth eye rubbing probability threshold, the preset ninth yawning probability threshold, the preset ninth left eye closing threshold, the preset ninth right eye closing threshold, the preset second left probability threshold, the preset second right probability threshold, the preset second head lowering probability threshold, the preset second head raising probability threshold, the preset tenth left eye closing probability threshold, the preset tenth right eye closing probability threshold, the preset third left probability threshold, the preset third right probability threshold, the preset third head lowering probability threshold, and the preset third head raising probability threshold to obtain a seventh comparison result; Based on the seventh comparison result, the sleepiness level of the person to be evaluated is determined.

[0104] A computer-readable storage medium is also provided in an embodiment of the present application. The computer-readable storage medium stores a computer program. The computer program includes program instructions. The processor executes the program instructions to implement any KSS sleepiness level assessment method provided in the embodiments of the present application.

[0105] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.

[0106] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A KSS sleepiness level assessment method, characterized in that: include: Acquire continuous facial images of the person to be evaluated as images to be detected; Detect each frame of the image to be detected to obtain face detection information, eye detection information, expression detection information and face angle detection information of each frame of the image to be detected; Based on a preset threshold, respectively, sliding window statistics are performed on the target features present in the face detection information, the eye detection information, the expression detection information, and the face angle detection information to obtain a probability value of each of the target features, and generate a probability value set; Based on the preset KSS sleepiness level assessment rule, the probability value of at least one of the target features is obtained in the probability value set, and the probability value of each of the target features is compared with the probability value threshold corresponding to each of the target features to obtain the sleepiness level of the person to be assessed.

2. The KSS sleepiness level assessment method according to claim 1, characterized in that: The preset thresholds include a left eye closed threshold and a right eye closed threshold. Based on the preset thresholds, sliding window statistics are performed on the target features present in the face detection information, the eye detection information, the expression detection information, and the face angle detection information to obtain the probability value of each of the target features, and before generating a probability value set, the method further includes: Based on the human eye detection information, acquiring left eye aspect ratio data and right eye aspect ratio data of the person to be evaluated; Performing frequency distribution analysis on the left eye aspect ratio data and the right eye aspect ratio data respectively to obtain a first target aspect ratio interval corresponding to the left eye and a second target aspect ratio interval corresponding to the right eye; Based on a preset proportionality coefficient and a first median value of the first target aspect ratio interval and a second median value of the second target aspect ratio interval, respectively obtaining a first target eye closing threshold value and a second target eye closing threshold value; When the first target eye closing threshold is within the preset eye closing threshold range, the left eye closing threshold is updated to the first target eye closing threshold, and when the second target eye closing threshold is within the eye closing threshold range, the right eye closing threshold is updated to the second target eye closing threshold.

3. The KSS sleepiness level assessment method according to claim 1, characterized in that: The target features include face presence feature, yawning feature, face leaning to the left feature, face leaning to the right feature, head lowering feature, head raising feature, left eye presence feature, left eye closed feature, left eye open feature, right eye presence feature, right eye closed feature, right eye open feature and eye rubbing feature.

4. The KSS sleepiness level assessment method according to claim 3, characterized in that: The method of performing sliding window statistics on target features in the face detection information, the eye detection information, the expression detection information, and the face angle detection information based on a preset threshold to obtain a probability value of each target feature includes: According to the number of frames of the image to be detected, the face detection information, the eye detection information, the expression detection information and the face angle detection information of each image to be detected are added to a preset short-term sliding window and a long-term sliding window respectively; Based on the short-term sliding window, the long-term sliding window and the preset threshold, judging and counting whether each of the target features exists in the face detection information, the eye detection information, the expression detection information and the face angle detection information, respectively, to obtain the statistical results of the short-term sliding window and the statistical results of the long-term sliding window; Probability calculation is performed based on the statistical result of the short-term sliding window and the statistical result of the long-term sliding window, respectively, to obtain a short-term probability value of each target feature in the short-term sliding window and a long-term probability value of each target feature in the long-term sliding window.

5. The KSS sleepiness level assessment method according to claim 1, characterized in that: The method further includes: obtaining the probability value of at least one target feature in the probability value set based on the preset KSS sleepiness level assessment rule, and comparing the probability value of each target feature with the probability value threshold corresponding to each target feature to obtain the sleepiness level of the person to be assessed. Based on the probability value set, obtaining a long-term probability value of the presence feature of the face; When the long-term probability value of the face existence feature is less than the preset face existence probability threshold, the image to be detected is reacquired, and the target feature probability is detected, so that when the long-term probability value of the face existence feature is greater than or equal to the face existence threshold, based on the preset KSS sleepiness level assessment rule, the probability value of at least one target feature is obtained in the probability value set, and the probability value of each target feature is compared with the probability value threshold corresponding to each target feature to obtain the sleepiness level of the person to be assessed.

6. The KSS sleepiness level assessment method according to claim 1, characterized in that: The method of obtaining the probability value of at least one target feature in the probability value set based on the preset KSS sleepiness level assessment rule, and comparing the probability value of each target feature with the probability value threshold corresponding to each target feature to obtain the sleepiness level of the person to be assessed includes: Based on the KSS sleepiness level assessment rule, in the probability value set, obtain the long-term probability value of the yawning feature, the long-term probability value of the left eye existing and the left eye closed, the long-term probability value of the right eye existing and the right eye closed, the long-term probability value of the left eye existing and the left eye open, the long-term probability value of the right eye existing and the right eye open, the long-term probability value of the face leaning to the left feature, the long-term probability value of the face leaning to the right feature, the long-term probability value of the head lowering feature, the long-term probability value of the head raising feature, the long-term probability value of the eye rubbing feature, the long-term probability value of the left eye existing feature, the short-term probability value of the left eye closed feature, the long-term probability value of the right eye existing feature, and the short-term probability value of the right eye closed feature; The long-term probability value of the yawning feature, the long-term probability value of the left eye existing and the left eye closed, the long-term probability value of the right eye existing and the right eye closed, the long-term probability value of the left eye existing and the left eye open, and the long-term probability value of the right eye existing and the right eye open are respectively compared with a preset first yawning probability threshold, a preset first left eye closed probability threshold, a preset first right eye closed probability threshold, a preset first left eye open probability value, and a preset first right eye open probability value to obtain a first comparison result; Compare the long-term probability value of the left-leaning face feature, the long-term probability value of the right-leaning face feature, the long-term probability value of the head-lowering feature, the long-term probability value of the head-raising feature, and the long-term probability value of the eye-rubbing feature with a preset first left-leaning probability threshold, a preset first right-leaning probability threshold, a preset first head-lowering probability threshold, a preset head-raising probability threshold, and a preset first eye-rubbing probability threshold, respectively, to obtain a second comparison result; Comparing the long-term probability value of the left eye presence feature, the short-term probability value of the left eye closed feature, the long-term probability value of the right eye presence feature, and the short-term probability value of the right eye closed feature with a preset left eye presence probability threshold, a preset second left eye closed probability threshold, a preset right eye presence probability threshold, and a preset second right eye closed probability threshold, respectively, to obtain a third comparison result; The sleepiness level of the person to be evaluated is determined based on the first comparison result, the second comparison result, and the third comparison result.

7. The KSS sleepiness level assessment method according to claim 6, characterized in that: The method of obtaining the probability value of at least one target feature in the probability value set based on the preset KSS sleepiness level assessment rule, and comparing the probability value of each target feature with the probability value threshold corresponding to each target feature to obtain the sleepiness level of the person to be assessed, further includes: Based on the KSS sleepiness level assessment rule, the long-term probability value of the left eye being present and the left eye being closed, the long-term probability value of the right eye being present and the right eye being closed, and the long-term probability value of the yawning feature are respectively compared with a preset third left eye closed probability threshold, a preset third right eye closed probability threshold, and a preset third yawning probability threshold; If the left eye exists and the long-term probability value of the left eye closed is greater than the third left eye closed probability threshold, or the right eye exists and the long-term probability value of the right eye closed is greater than the third right eye closed probability threshold, or the long-term probability value of the yawning feature is greater than the third yawning probability threshold, then the short-term probability value of the left eye closed feature, the short-term probability value of the right eye closed feature, the short-term probability value of the yawning feature, and the short-term probability value of the eye rubbing feature are obtained from the probability value set; comparing the short-term probability value of the left eye closed feature, the short-term probability value of the right eye closed feature, the short-term probability value of the yawning feature, and the short-term probability value of the eye rubbing feature with a preset fourth left eye closed probability threshold, a preset fourth right eye closed probability threshold, a preset fourth yawning probability threshold, and a preset second eye rubbing probability threshold, respectively, to obtain a fourth comparison result; Based on the fourth comparison result, the sleepiness level of the person to be evaluated is determined.

8. The KSS sleepiness level assessment method according to claim 6, characterized in that: The method of obtaining the probability value of at least one target feature in the probability value set based on the preset KSS sleepiness level assessment rule, and comparing the probability value of each target feature with the probability value threshold corresponding to each target feature to obtain the sleepiness level of the person to be assessed, further includes: Based on the KSS sleepiness level assessment rule, the long-term probability value of the left eye being present and the left eye being closed, the long-term probability value of the right eye being present and the right eye being closed, and the long-term probability value of the yawning feature are compared with the preset fifth left eye closed probability threshold, the preset fifth right eye closed probability threshold, and the preset fifth yawning probability threshold, respectively: If the left eye exists and the long-term probability value of the left eye closed is greater than the fifth left eye closed probability threshold, or the right eye exists and the long-term probability value of the right eye closed is greater than the fifth right eye closed probability threshold, or the long-term probability value of the yawning feature is greater than the fifth yawning probability threshold, then the short-term probability value of the left eye closed feature, the short-term probability value of the right eye closed feature, the short-term probability value of the yawning feature, and the short-term probability value of the eye rubbing feature are obtained from the probability value set; comparing the short-term probability value of the left eye closed feature, the short-term probability value of the right eye closed feature, the short-term probability value of the yawning feature, and the short-term probability value of the eye rubbing feature with a preset sixth left eye closed probability threshold, a preset sixth right eye closed probability threshold, a preset sixth yawning probability threshold, and a preset third eye rubbing probability threshold, respectively, to obtain a fifth comparison result; Based on the fifth comparison result, the sleepiness level of the person to be evaluated is determined.

9. The KSS sleepiness level assessment method according to claim 6, characterized in that: The method of obtaining the probability value of at least one target feature in the probability value set based on the preset KSS sleepiness level assessment rule, and comparing the probability value of each target feature with the probability value threshold corresponding to each target feature to obtain the sleepiness level of the person to be assessed, further includes: Based on the KSS sleepiness level assessment rule, the long-term probability value of the left eye being present and the left eye being closed, the long-term probability value of the right eye being present and the right eye being closed, and the long-term probability value of the yawning feature are compared with the seventh preset left eye closed probability threshold, the seventh preset right eye closed probability threshold, and the seventh preset yawning probability threshold, respectively: If the left eye exists and the long-term probability value of the left eye closed is greater than the seventh left eye closed probability threshold, or the right eye exists and the long-term probability value of the right eye closed is greater than the seventh right eye closed probability threshold, or the long-term probability value of the yawning feature is greater than the seventh yawning probability threshold, then the short-term probability value of the left eye closed feature, the short-term probability value of the right eye closed feature, the short-term probability value of the yawning feature, and the short-term probability value of the eye rubbing feature are obtained from the probability value set; comparing the short-term probability value of the left eye closed feature, the short-term probability value of the right eye closed feature, the short-term probability value of the yawning feature, and the short-term probability value of the eye rubbing feature with a preset eighth left eye closed probability threshold, a preset eighth right eye closed probability threshold, a preset eighth yawning probability threshold, and a preset fourth eye rubbing probability threshold, respectively, to obtain a sixth comparison result; Based on the sixth comparison result, the sleepiness level of the person to be evaluated is determined.

10. The KSS sleepiness level assessment method according to claim 6, characterized in that: The method of obtaining the probability value of at least one target feature in the probability value set based on the preset KSS sleepiness level assessment rule, and comparing the probability value of each target feature with the probability value threshold corresponding to each target feature to obtain the sleepiness level of the person to be assessed, further includes: Based on the KSS sleepiness level assessment rule, a short-term probability value of a left-leaning face feature, a short-term probability value of a right-leaning face feature, a short-term probability value of a head-down feature, and a short-term probability value of a head-up feature are obtained; The long-term probability value of the eye rubbing feature, the long-term probability value of the yawning feature, the long-term probability value of the left eye existing and the left eye closed, the long-term probability value of the right eye existing and the right eye closed, the long-term probability value of the left face feature, the long-term probability value of the right face feature, the long-term probability value of the head lowering feature, the long-term probability value of the head raising feature, the short-term probability value of the left eye closed feature, the short-term probability value of the right eye closed feature, the short-term probability value of the left face feature, the short-term probability value of the right face feature, the short-term probability value of the head lowering feature and the short-term probability value of the head raising feature are calculated. The short-term probability value of the head feature is respectively compared with the preset sixth eye rubbing probability threshold, the preset ninth yawning probability threshold, the preset ninth left eye closing threshold, the preset ninth right eye closing threshold, the preset second left probability threshold, the preset second right probability threshold, the preset second head lowering probability threshold, the preset second head raising probability threshold, the preset tenth left eye closing probability threshold, the preset tenth right eye closing probability threshold, the preset third left probability threshold, the preset third right probability threshold, the preset third head lowering probability threshold, and the preset third head raising probability threshold to obtain a seventh comparison result; Based on the seventh comparison result, the sleepiness level of the person to be evaluated is determined.

Images