A lightweight yolov8-pose fatigue driving detection method

Abstract

The application provides a fatigue driving detection method of a lightweight YOLOv8-pose, the method constructs a lightweight YOLOv8-pose model by using collected multi-pose and multi-view face data sets, and introduces a Ghost convolution into a model backbone network to reduce model parameter quantity and unnecessary convolution calculation; different size features extracted by a Slim-neck fusion model backbone network are introduced to speed up network prediction calculation; a shielding perception attention module SEAM is added to a model neck part to emphasize a face area in an image and weaken a background, so that key point positioning effect is improved; a GNSC-Head structure is used in a detection head part of the model, shared convolution is used to optimize a BN layer of convolution into a more stable GN layer, so that parameter space and calculation resources of the model are saved; the method constructs a fatigue decision model and evaluates output results of the model to determine whether a driver is in a fatigue state; the method is beneficial to reduce calculation complexity and improve fatigue detection precision.

Description

Technical Field

[0001] This invention relates to the field of fatigue driving detection technology, and in particular to a lightweight fatigue driving detection method using YOLOv8-pose. Background Technology

[0002] In recent years, with the development of the social economy and the gradual improvement of people's living standards, the per capita ownership of cars has been increasing. However, along with the rapid growth in the number of cars and the mileage of roads, traffic accidents have also shown a certain degree of increase, with approximately 18% of these accidents caused by driver fatigue. Research has found that if drivers are given reminders and warnings when they are fatigued, 90% of similar traffic accidents can be avoided. Therefore, researching an efficient method for detecting driver fatigue is of great significance for traffic safety.

[0003] Currently, research on fatigue driving detection is mainly divided into contact detection and non-contact detection. Contact detection involves wearing sensors to acquire and study the driver's physiological information, typically assessing fatigue levels based on electroencephalogram (EEG) and electrocardiogram (ECG) signals. While this method boasts high accuracy, it requires complex hardware, hindering widespread application. Non-contact detection primarily employs two methods. The first is fatigue driving detection based on vehicle trajectory. Although it doesn't require direct contact with the driver, it's susceptible to road conditions and weather, resulting in lower accuracy. The second method uses image acquisition devices to capture facial information and uses computers to determine the driver's fatigue state. Compared to physiological signal acquisition devices and vehicle trajectory acquisition devices, surveillance cameras are inexpensive, convenient, and do not interfere with the driver's normal driving, making this type of research more promising for application.

[0004] However, existing deep learning-based fatigue detection algorithms have some shortcomings. Some methods detect fatigue using only a single feature, failing to consider the various fatigue states of drivers, resulting in poor robustness. While some methods achieve high detection accuracy, they require a large number of model parameters and computational resources, making them difficult to deploy on in-vehicle edge devices. Although some methods utilize lightweight models, the excessive number of detection models necessitates separate training for each model, preventing end-to-end optimization and introducing latency in data transmission between models, ultimately degrading the real-time performance of fatigue detection. Summary of the Invention

[0005] This invention proposes a lightweight YOLOv8-pose fatigue driving detection method, which helps to reduce computational complexity and improve fatigue detection accuracy.

[0006] The present invention adopts the following technical solution.

[0007] A lightweight YOLOv8-pose fatigue driving detection method is disclosed. This method constructs a lightweight YOLOv8-pose model using a multi-pose, multi-view face dataset. Ghost convolutions are introduced into the model's backbone network to reduce the number of model parameters and unnecessary convolutional computations. A Slim-neck fusion feature model extracted from the backbone network of different sizes is introduced to accelerate network prediction computation. An occlusion awareness attention module (SEAM) is added to the model's neck part to emphasize the face region in the image and weaken the background, thereby improving keypoint localization. Simultaneously, a GNSC-Head structure is adopted in the model's detection head, which uses shared convolutions to optimize the BN layers of the convolutions into more stable GN layers, saving model parameter space and computational resources. The method constructs a fatigue decision model and evaluates the model's output to determine whether the driver is fatigued.

[0008] The method includes the following steps;

[0009] Step S1: Collect face datasets with multiple poses and views, and convert the annotation files to generate corresponding YOLO annotation files;

[0010] Step S2: Build and train a lightweight YOLOv8-pose model, use the trained model to perform face detection on the input image, and locate facial key points after recognizing the face. Finally, classify and regress the detected head part to obtain the key point coordinates of the driver's eyes and mouth.

[0011] Step S3: Construct a fatigue decision model to provide fatigue judgment based on the features of the eyes and mouth. The PERCLOS evaluation criterion is used to determine whether the eyes are open or closed, and the driver is in a fatigued state by detecting the number of times the driver's eyes close within a certain period of time. The same principle method is used for the evaluation criterion of mouth features, that is, to determine whether the mouth is open or closed by measuring the coordinate information of the mouth, and to determine whether the driver is in a fatigued state by detecting the number of times the driver yawns within a certain period of time.

[0012] Step S4: Visualize and issue early warnings based on the driver fatigue behavior judgment results obtained from the identification and analysis in Steps S2 and S3.

[0013] In step S1, the AFLW dataset is introduced as a large-scale face dataset containing multiple poses and perspectives. If the number of closed-eye images in this dataset is too low to meet the requirements, the CEW closed-eye dataset is introduced as a supplement. When the format of the annotation files in the above dataset is not the annotation format of the YOLO model, the annotation files are converted. The method is to use Python to process the data and generate the corresponding YOLO annotation files. In step S1, the above dataset is randomly sampled in a ratio of 6:2:2 and divided into training set, validation set and test set.

[0014] In step S2, the lightweight YOLOv8-pose introduces lightweight convolution GhostConv in the backbone network to generate a large number of Ghost feature maps that can extract the required information from the original features at low cost using inexpensive linear transformations, thereby reducing the number of model parameters and computational cost.

[0015] In step S2, a fast detection module C3 is introduced. By performing lightweight operations on the C3 module, two identical GhostConv modules are combined and connected to form a Ghost-Bottleneck structure. The recombined C3 module is used as the C3Ghost module to reduce the complexity of the model and the computational load of the facial landmark model.

[0016] In step S2, the lightweight YOLOv8-pose uses a Slim-neck network structure as an enhanced feature fusion network, introduces a lightweight convolution GSConv, and uses dense convolutional computation to maximize the preservation of the hidden connections between each channel to accelerate the model's prediction computation. It also uses residual connections to form a GS-bottleneck structure to further enhance the network's ability to process features. Finally, it uses a one-time aggregation method to form a VOV-GSCSP module, which allows gradients at different locations to be cross-mixed to enhance the network's gradient performance and learning ability.

[0017] In step S2, the lightweight YOLOv8-pose incorporates occlusion-aware attention SEAM to emphasize the face region in the image and weaken the background while achieving multi-scale face detection, thereby improving the facial landmark localization effect.

[0018] In step S2, the lightweight YOLOv8-pose retains the decoupled structure with the GNSC-Head detection head and uses group-normalized shared convolutions to perform lightweight improvements on the network.

[0019] In step S2, a lightweight YOLOv8-pose keypoint detection model is built using the PyTorch deep learning framework. The number of training rounds is set to 300, the input image size is set to 640×640, the batch size is 64, the training weights are saved every 10 rounds, and the mAP is calculated using the validation set to evaluate the model performance. Finally, the model with the largest mAP is selected as the final model.

[0020] In step S3, the eye aspect ratio (EAR) is calculated based on the eye keypoint information output by the lightweight YOLOv8-pose. The EAR threshold is set with reference to the P80 standard in the PERCLOS standard, where P80 indicates that the eyelid covering more than 80% of the pupil area is considered as eye closure. The formula is as follows:

[0021]

[0022]

[0023] Where x i The x-coordinate of the key points of the eye, y i Let f represent the ordinate. e To determine the driver's eye fatigue level, t e It is the number of frames with eyes closed within the detection time, T e It is the total number of frames during the detection time;

[0024] In step S3, the mouth aspect ratio (MAR) is calculated based on the mouth keypoint information output by the lightweight YOLOv8-pose. The formula is as follows:

[0025]

[0026]

[0027] Where x i The x-coordinate of the key points of the mouth, y i Let f represent the ordinate. m To assess driver's oral fatigue level, t m T represents the number of mouth-opening frames within the detection time. m This indicates the total number of frames within the detection time.

[0028] In step S3, the algorithm evaluates the model's output, comprehensively assesses various indicators to determine whether the driver is fatigued, and issues a visual warning. The trigger criteria for the visual warning are as follows:

[0029] Under normal conditions, the time it takes for one eye to close is 0.1-0.15 seconds, while under fatigue conditions, the time it takes for one eye to close is greater than 0.5 seconds. That is, if the frequency of closing the eyes is greater than 0.5 times per unit time, the driver is judged to be in a state of fatigue.

[0030] The duration of a typical human yawn is 3-5 seconds. In the algorithm in step S3, the unit detection time is selected as 30 seconds. The number of yawns within 30 seconds cannot exceed two. That is, when the yawning frequency per unit time is greater than 0.4, the driver is judged to be in a state of fatigue.

[0031] Compared with existing technologies, this invention offers the following advantages: It provides a lightweight YOLOv8-pose fatigue driving detection method. This method utilizes a lightweight YOLOv8-pose model to achieve face detection and facial key point localization, and then uses a fatigue decision module to determine the driver's mental state. The lightweight YOLOv8-pose model uses Ghost convolution and GS convolution to significantly reduce the number of network parameters and improve the model's detection rate. Simultaneously, it employs an occlusion-aware attention mechanism and a GNSC-Head structure, maintaining high detection accuracy while being lightweight. The fatigue decision module uses a multi-feature fatigue judgment method to more effectively identify the driver's state, providing strong support for the deployment of vehicle edge devices. Attached Figure Description

[0032] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0033] Appendix Figure 1 This is a flowchart illustrating the method implementation of an embodiment of the present invention;

[0034] Appendix Figure 2 This is a structural diagram of the key point detection model in an embodiment of the present invention;

[0035] Appendix Figure 3 This is a structural diagram of the SEAM module in an embodiment of the present invention;

[0036] Appendix Figure 4 This is a structural diagram of the GNSC-Head module in an embodiment of the present invention;

[0037] Appendix Figure 5 This is a distribution diagram of the location of key facial points in an embodiment of the present invention;

[0038] Appendix Figure 6 This is a visual schematic diagram of the fatigue detection system in an embodiment of the present invention. Detailed Implementation

[0039] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0040] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0041] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0042] As shown in the figure, a lightweight YOLOv8-pose fatigue driving detection method is proposed. This method constructs a lightweight YOLOv8-pose model using a multi-pose, multi-view face dataset. Ghost convolutions are introduced into the model's backbone network to reduce the number of model parameters and unnecessary convolutional computations. A Slim-neck fusion feature model extracted from the backbone network of different sizes is introduced to accelerate network prediction computation. An occlusion awareness attention module (SEAM) is added to the model's neck part to emphasize the face region in the image and weaken the background, thereby improving key point localization. Simultaneously, a GNSC-Head structure is adopted in the model's detection head part, which uses shared convolutions to optimize the BN layers of the convolutions into more stable GN layers, saving model parameter space and computational resources. The method constructs a fatigue decision model and evaluates the model's output to determine whether the driver is fatigued.

[0043] The method includes the following steps;

[0044] Step S1: Collect face datasets with multiple poses and views, and convert the annotation files to generate corresponding YOLO annotation files;

[0045] Step S2: Build and train a lightweight YOLOv8-pose model, use the trained model to perform face detection on the input image, and locate facial key points after recognizing the face. Finally, classify and regress the detected head part to obtain the key point coordinates of the driver's eyes and mouth.

[0046] Step S3: Construct a fatigue decision model to provide fatigue judgment based on the features of the eyes and mouth. The PERCLOS evaluation criterion is used to determine whether the eyes are open or closed, and the driver is in a fatigued state by detecting the number of times the driver's eyes close within a certain period of time. The same principle method is used for the evaluation criterion of mouth features, that is, to determine whether the mouth is open or closed by measuring the coordinate information of the mouth, and to determine whether the driver is in a fatigued state by detecting the number of times the driver yawns within a certain period of time.

[0047] Step S4: Visualize and issue early warnings based on the driver fatigue behavior judgment results obtained from the identification and analysis in Steps S2 and S3.

[0048] In step S1, the AFLW dataset is introduced as a large-scale face dataset containing multiple poses and perspectives. If the number of closed-eye images in this dataset is too low to meet the requirements, the CEW closed-eye dataset is introduced as a supplement. When the format of the annotation files in the above dataset is not the annotation format of the YOLO model, the annotation files are converted. The method is to use Python to process the data and generate the corresponding YOLO annotation files. In step S1, the above dataset is randomly sampled in a ratio of 6:2:2 and divided into training set, validation set and test set.

[0049] In step S2, the lightweight YOLOv8-pose introduces lightweight convolution GhostConv in the backbone network to generate a large number of Ghost feature maps that can extract the required information from the original features at low cost using inexpensive linear transformations, thereby reducing the number of model parameters and computational cost.

[0050] In step S2, a fast detection module C3 is introduced. By performing lightweight operations on the C3 module, two identical GhostConv modules are combined and connected to form a Ghost-Bottleneck structure. The recombined C3 module is used as the C3Ghost module to reduce the complexity of the model and the computational load of the facial landmark model.

[0051] In step S2, the lightweight YOLOv8-pose uses a Slim-neck network structure as an enhanced feature fusion network, introduces a lightweight convolution GSConv, and uses dense convolutional computation to maximize the preservation of the hidden connections between each channel to accelerate the model's prediction computation. It also uses residual connections to form a GS-bottleneck structure to further enhance the network's ability to process features. Finally, it uses a one-time aggregation method to form a VOV-GSCSP module, which allows gradients at different locations to be cross-mixed to enhance the network's gradient performance and learning ability.

[0052] In step S2, the lightweight YOLOv8-pose incorporates occlusion-aware attention SEAM to emphasize the face region in the image and weaken the background while achieving multi-scale face detection, thereby improving the facial landmark localization effect.

[0053] In step S2, the lightweight YOLOv8-pose retains the decoupled structure with the GNSC-Head detection head and uses group-normalized shared convolutions to perform lightweight improvements on the network.

[0054] In step S2, a lightweight YOLOv8-pose keypoint detection model is built using the PyTorch deep learning framework. The number of training rounds is set to 300, the input image size is set to 640×640, the batch size is 64, the training weights are saved every 10 rounds, and the mAP is calculated using the validation set to evaluate the model performance. Finally, the model with the largest mAP is selected as the final model.

[0055] In step S3, the eye aspect ratio (EAR) is calculated based on the eye keypoint information output by the lightweight YOLOv8-pose. The EAR threshold is set with reference to the P80 standard in the PERCLOS standard, where P80 indicates that the eyelid covering more than 80% of the pupil area is considered as eye closure. The formula is as follows:

[0056]

[0057]

[0058] Where x i The x-coordinate of the key points of the eye, y i Let f represent the ordinate. e To determine the driver's eye fatigue level, t e It is the number of frames with eyes closed within the detection time, T e It is the total number of frames during the detection time;

[0059] In step S3, the mouth aspect ratio (MAR) is calculated based on the mouth keypoint information output by the lightweight YOLOv8-pose. The formula is as follows:

[0060]

[0061]

[0062] Where x i The x-coordinate of the key points of the mouth, y i Let f represent the ordinate. m To assess driver's oral fatigue level, t m T represents the number of mouth-opening frames within the detection time. mThis indicates the total number of frames within the detection time.

[0063] In step S3, the algorithm evaluates the model's output, comprehensively assesses various indicators to determine whether the driver is fatigued, and issues a visual warning. The trigger criteria for the visual warning are as follows:

[0064] Under normal conditions, the time it takes for one eye to close is 0.1-0.15 seconds, while under fatigue conditions, the time it takes for one eye to close is greater than 0.5 seconds. That is, if the frequency of closing the eyes is greater than 0.5 times per unit time, the driver is judged to be in a state of fatigue.

[0065] The duration of a typical human yawn is 3-5 seconds. In the algorithm in step S3, the unit detection time is selected as 30 seconds. The number of yawns within 30 seconds cannot exceed two. That is, when the yawning frequency per unit time is greater than 0.4, the driver is judged to be in a state of fatigue.

[0066] Example:

[0067] In this example, AFLW is a large-scale face database containing multiple poses and perspectives, including photos affected by various factors such as pose, expression, lighting, and ethnicity. This database contains approximately 25,000 manually annotated face images and is suitable for face recognition, facial landmark detection, and other fields. Since the number of images with closed eyes in the dataset is too small, the CEW closed-eye dataset is introduced as a supplement. It contains 2423 photos of test subjects with both eyes open and closed. The differences in the photos reflect individual differences among test subjects and various environmental changes, such as lighting, blur, and occlusion, effectively improving the robustness of model training.

[0068] In this example, the keypoint detection model constructed for fatigue detection is a lightweight YOLOv8-pose model, and its model structure is as follows: Figure 2 As shown.

[0069] Specifically, the backbone network is used for image feature extraction and consists of Ghost convolutions and C3Ghost modules. In deep convolutional neural networks, the output feature maps of intermediate layers often contain rich and even redundant feature maps, with some feature maps having similar feature information, thus consuming a large amount of memory and FLOPs. Ghost convolutions generate many Ghost feature maps that can extract the required information from the original features at a very low cost through inexpensive linear transformations, effectively reducing the number of model parameters and computational cost. In the original YOLOv8n-pose backbone network, the C2f module sacrificed some speed to obtain more rich gradient flow information. However, in the fatigue driving detection system, since the surrounding scene is relatively simple and there is no need to identify small targets, the C2f module is optimized into the C3 module, and lightweight operations are performed on the C3 module, introducing the Ghost-Bottleneck structure from the Ghost-net network. In this structure, the first Ghost convolution is used to compress the number of channels, reduce the number of parameters, and reduce the impact of high-frequency noise. The second Ghost convolution is used to restore the number of channels. Then, residual connections are used to add original features to compensate for the information loss caused by channel compression. This bottleneck structure helps reduce the number of parameters and information loss. Meanwhile, to avoid affecting the feature extraction capability of the backbone network, the remaining convolutional operations in the C3 module remain unchanged. The improved C3 module is named the C3Ghost module. As a lightweight network, the C3Ghost module can effectively reduce the number of network parameters and computational cost while maintaining the original output feature map size and channel size, further reducing model complexity and computational cost for facial landmark models.

[0070] The role of the feature fusion network is to enhance the extraction of features from the backbone network and to laterally connect features extracted at different stages, achieving the fusion of high-level semantic features and low-level detailed features, thereby generating richer feature representations. To further achieve model lightweighting without compromising feature extraction capabilities, this invention introduces GSConv from the Slim-neck and improves the YOLOv8-pose Neck using the VoV-GSCSP module, further optimizing the model's parameter count and computational complexity. GSConv maximizes the preservation of implicit connections between channels through dense convolutional computation, avoiding semantic information loss caused by spatial compression and channel expansion of the feature map, thus accelerating the model's prediction computation. Furthermore, residual connections form the GS-bottleneck structure, further enhancing the network's feature processing capabilities. Finally, a one-time aggregation method is used to construct the VOV-GSCSP module, allowing gradients at different locations to be cross-mixed, enhancing the network's gradient performance and learning ability, ensuring model accuracy while maintaining lightweight design.

[0071] Simultaneously, an Occlusion Awareness Attention Module (SEAM) is added to the feature fusion network. This module effectively avoids the problems of low facial recognition rate and inaccurate localization caused by drivers wearing sunglasses or masks while driving. The overall architecture of SEAM is as follows: Figure 3 As shown, the left side shows the SEAM architecture, and the right side shows the structure of DcovN (a channel and spatial hybrid module). The first part of DcovN is a depthwise convolution with residual connections. While depthwise convolution can learn the importance of different channels and reduce the number of parameters, it ignores the information relationships between channels. To compensate for this loss, pointwise convolution is introduced in the second part to enhance the representation and generalization capabilities of the DcovN module. After the input feature map passes through the DcovN module, it goes through an average pooling layer to reduce the spatial size of the feature map while retaining important feature information. Then, two fully connected networks are used to fuse the information of each channel, enabling the network to strengthen the connections between all channels. The output learned by the fully connected layers is then processed by an exponential function, expanding the value range from [0,1] to [1,e]. Since exponential normalization provides a monotonic mapping relationship, the output is more tolerant of positional errors. Finally, the output of the SEAM module is multiplied by the original features, enabling the model to effectively handle occluded parts of the face.

[0072] The YOLOv8-pose detection head employs a decoupled structure, extracting target location, category information, and keypoint coordinates separately. These are then learned through different network branches and finally fused. Compared to traditional coupled heads, decoupled heads can better handle semantic information at different scales and levels of detail, improving the model's generalization ability and robustness. However, this also significantly increases the number of parameters; the detection head alone accounts for one-third of the entire model's parameters. Therefore, to improve the model's detection speed while maintaining stable accuracy, this invention uses the GNSC-Head detection head, retaining the original decoupled structure, and improving the network using group-normalized shared convolutions. The improved detection head network structure is shown below. Figure 4 .

[0073] GNSC-Head saves parameter space and computational resources by introducing shared convolutions at multiple locations, enabling its deployment in space-constrained in-vehicle systems. To compensate for the decrease in feature extraction capability caused by shared convolutions, the batch normalization (BN) layers in the convolutional layers are replaced with GN layers. Compared to BN, GN divides the feature map into several groups and normalizes each group. It is independent of batch size, thus maintaining good performance even with small batches of high-precision images. Furthermore, in detection tasks, the input to the detection head comes from Regions of Interest (ROIs), which are sampled from the same images and do not satisfy the independent and identically distributed (IOD) assumption. The non-IOD weakens the mean and variance distribution of the BN layer; therefore, using GN in the detection head yields better results than using BN.

[0074] In classification tasks, regardless of the target size, the detection layers all target the same object. Therefore, shared convolutions can better adapt to different stages of the classification task, thereby improving the model's generalization ability on targets of different scales. In bounding box regression tasks, since the features extracted by shared convolutions are the same for targets of all scales, they cannot effectively distinguish targets of different scales. To address this, a scale layer is added at the end of the network. By introducing a learnable scaling factor, the model can better extract features on targets of different scales, thus effectively adjusting the scale of the input. In keypoint regression tasks, the number of channels in shared convolutions needs to be divisible by the number of groups to ensure that each group has the same number of channels. However, the total number of facial keypoints in this invention cannot be divisible by the number of groups, and keypoint regression requires precise localization of keypoints in the target image, which is a relatively fine-grained task. Therefore, the original detection head network structure is chosen for this task to ensure the overall accuracy of the model.

[0075] In this example, step S3 specifically involves: constructing a fatigue decision model, providing fatigue judgments based on eye and mouth features, using the PERCLOS evaluation criterion to determine eye opening and closing, and judging whether the driver is fatigued by detecting the number of times the driver's eyes close within a certain period. The evaluation criterion for mouth features is similar: judging mouth opening and closing based on mouth coordinate information, and judging whether the driver is fatigued by detecting the number of times the driver yawns within a certain period.

[0076] Specifically, an image is input using a lightweight YOLOv8n-pose network model. After detecting the facial region in the image, facial key points are extracted and their positions are plotted on the image. Traditional facial key points consist of 68 points, such as... Figure 5As shown, in actual testing, it is possible to determine whether a driver is fatigued simply by examining the driver's eyes and mouth. Therefore, this invention selects only 18 key facial points for fatigue assessment, namely 12 key points (37-48) for the left and right eyes, and 6 key points (49, 51, 53, 55, 57, 59) for the mouth.

[0077] The Euclidean distance of each key eye point is obtained using its x and y coordinates. The eye aspect ratio (EAR) is then calculated. The EAR threshold is based on PERCLOS, an internationally recognized standard for assessing fatigue, which defines the proportion of time the eyes are closed within a given period. PERCLOS standards include P70, P80, and EM, representing eye closure when the eyelid covers more than 70%, 80%, and 50% of the pupil area, respectively. P80 is considered the most sensitive standard for fatigue assessment. In this paper, the eye condition evaluation index references the P80 standard from PERCLOS, setting the EAR threshold at 0.2. When the EAR is less than 0.2, the eyes are considered closed. The formula is as follows:

[0078]

[0079]

[0080] By defining f e To determine the driver's eye fatigue level, t e It is the number of frames with eyes closed within the detection time, T e This is the total number of frames during the detection time. Under normal conditions, the monocular closure time is typically 0.1-0.15 seconds, while under fatigue conditions, the monocular closure time is typically greater than 0.5 seconds. Therefore, f... e The parameter threshold is set to 0.5, meaning that when the frequency of closing the eyes is greater than 0.5 per unit time, the driver is judged to be in a state of fatigue.

[0081] Similarly, the evaluation index for mouth condition is similar to that for eye condition. It uses the x and y coordinates of key mouth points to obtain the Euclidean distance between each point, calculates the mouth's aspect ratio (MAR), and defines f as... m The formula for assessing driver's oral fatigue level is as follows:

[0082]

[0083]

[0084] Among them, t m T represents the number of mouth-opening frames within the detection time. mThis represents the total number of frames within the detection time. The typical duration of a human yawn is 3-5 seconds. In this algorithm, the unit detection time is chosen to be 30 seconds, and the number of yawns within 30 seconds cannot exceed two. Therefore, this paper will use f... m The parameter threshold is set to 0.4, meaning that when the yawning frequency per unit time is greater than 0.4, the driver is judged to be in a state of fatigue.

[0085] S4: Provide a visual warning based on the driver fatigue behavior judgment results obtained from the S2 and S3 identification and analysis.

[0086] Specifically, to test the generalization ability and effectiveness of the lightweight YOLOv8-pose model in a driving environment, this paper uses the recognition rate of the YawnDD (YAWNING DETECTION DATASET) driving detection video dataset as the evaluation metric. The video data was collected from the driver's seat of a car, including forward-facing and oblique-side video captures. The captured video content includes normal driving, talking, and yawning due to fatigue. Following the method described above, EAR and MAR were calculated to detect the number of times the driver closed their eyes and yawning behavior. Figure 6 As shown, the lightweight YOLOv8-pose model can accurately detect and track drivers' blinking and yawning behaviors during driving, and effectively identify and statistically analyze these behaviors to determine driving status. Even when female drivers are wearing glasses or male drivers are driving in low-light conditions, the model can still identify driver fatigue.

[0087] In summary, the lightweight YOLOv8-pose fatigue driving detection method of this invention can effectively improve the feature extraction capability and detection accuracy of the model algorithm for key points, enhance the lightweight performance of the model algorithm, and greatly improve the real-time performance of the system.

[0088] Many specific details have been set forth in the foregoing description to provide a thorough understanding of the present invention. However, the above description is merely a preferred embodiment of the present invention, and the present invention can be implemented in many other ways different from those described herein. Therefore, the present invention is not limited to the specific embodiments disclosed above. Furthermore, any person skilled in the art can make many possible variations and modifications to the technical solutions of the present invention, or modify them into equivalent embodiments, using the methods and techniques disclosed above, without departing from the scope of the present invention. Any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention, without departing from the content of the present invention, shall still fall within the protection scope of the present invention.

Claims

1. A lightweight YOLOv8-pose fatigue driving detection method, characterized in that: The method constructs a lightweight YOLOv8-pose model using a multi-pose, multi-view face dataset. Ghost convolutions are introduced into the model's backbone network to reduce the number of model parameters and unnecessary convolutional computations. A Slim-neck fusion feature model extracted from the backbone network of different sizes is introduced to accelerate network prediction computation. An occlusion awareness attention module (SEAM) is added to the model's neck part to emphasize the face region in the image and weaken the background, thereby improving keypoint localization. Simultaneously, a GNSC-Head structure is adopted in the model's detection head part, which uses shared convolutions to optimize the BN layers of the convolutions into more stable GN layers, saving model parameter space and computational resources. The method constructs a fatigue decision model and evaluates the model's output to determine whether the driver is fatigued. The method Includes the following steps; Step S1: Collect face datasets with multiple poses and views, and convert the annotation files to generate corresponding YOLO annotation files; Step S2: Build and train a lightweight YOLOv8-pose model, use the trained model to perform face detection on the input image, and locate facial key points after recognizing the face. Finally, classify and regress the detected head part to obtain the key point coordinates of the driver's eyes and mouth. Step S3: Construct a fatigue decision model to provide fatigue judgment based on the features of the eyes and mouth. The PERCLOS evaluation criterion is used to determine whether the eyes are open or closed, and the driver is in a fatigued state by detecting the number of times the driver's eyes close within a certain period of time. The same principle method is used for the evaluation criterion of mouth features, that is, to determine whether the mouth is open or closed by measuring the coordinate information of the mouth, and to determine whether the driver is in a fatigued state by detecting the number of times the driver yawns within a certain period of time. Step S4: Visualize and issue early warnings based on the driver fatigue behavior judgment results obtained from the identification and analysis in Steps S2 and S3; In step S2, the lightweight YOLOv8-pose introduces lightweight convolution GhostConv in the backbone network to generate a large number of Ghost feature maps that can extract the required information from the original features at low cost using inexpensive linear transformations, thereby reducing the number of model parameters and computational cost. In step S2, a fast detection module C3 is introduced. By performing lightweight operations on the C3 module, two identical GhostConv modules are combined and connected to form a Ghost-Bottleneck structure. The recombined C3 module is used as the C3Ghost module to reduce the complexity of the model and the computational cost of the face key point model. In step S2, the lightweight YOLOv8-pose retains the decoupled structure with the GNSC-Head detection head and uses group-normalized shared convolutions to improve the lightweight nature of the network.

2. The fatigue driving detection method of lightweight YOLOv8-pose according to claim 1, characterized in that: In step S1, the AFLW dataset is introduced as a large-scale face dataset containing multiple poses and perspectives. If the number of closed-eye images in this dataset is too low to meet the requirements, the CEW closed-eye dataset is introduced as a supplement. When the format of the annotation files in the above dataset is not the annotation format of the YOLO model, the annotation files are converted. The method is to use Python to process the data and generate the corresponding YOLO annotation files. In step S1, the above dataset is randomly sampled in a ratio of 6:2:2 and divided into training set, validation set and test set.

3. The fatigue driving detection method of lightweight YOLOv8-pose according to claim 2, characterized in that: In step S2, the lightweight YOLOv8-pose uses a Slim-neck network structure as an enhanced feature fusion network, introduces a lightweight convolution GSConv, and uses dense convolutional computation to maximize the preservation of the hidden connections between each channel to accelerate the model's prediction computation. It also uses residual connections to form a GS-bottleneck structure to further enhance the network's ability to process features. Finally, it uses a one-time aggregation method to form a VOV-GSCSP module, which allows gradients at different locations to be cross-mixed to enhance the network's gradient performance and learning ability.

4. The fatigue driving detection method of lightweight YOLOv8-pose according to claim 1, characterized in that: In step S2, the lightweight YOLOv8-pose incorporates occlusion-aware attention SEAM to emphasize the face region in the image and weaken the background while achieving multi-scale face detection, thereby improving the facial landmark localization effect.

5. The fatigue driving detection method of lightweight YOLOv8-pose according to claim 3, characterized in that: In step S2, a lightweight YOLOv8-pose keypoint detection model is built using the PyTorch deep learning framework. The number of training rounds is set to 300, the input image size is set to 640×640, the batch size is 64, the training weights are saved every 10 rounds, and the mAP is calculated using the validation set to evaluate the model performance. Finally, the model with the largest mAP is selected as the final model.

6. The fatigue driving detection method of lightweight YOLOv8-pose according to claim 1, characterized in that: In step S3, the eye aspect ratio (EAR) is calculated based on the eye keypoint information output by the lightweight YOLOv8-pose. The EAR threshold is set with reference to the P80 standard in the PERCLOS standard, where P80 indicates that the eyelid covering more than 80% of the pupil area is considered as eye closure. The formula is as follows: Where x i The x-coordinate of the key points of the eye, y i Let f represent the ordinate. e To determine the driver's eye fatigue level, t e It is the number of frames with eyes closed within the detection time, T e It is the total number of frames during the detection time; In step S3, the mouth aspect ratio (MAR) is calculated based on the mouth key point information output by the lightweight YOLOv8-pose; the formula is as follows: Where x i The x-coordinate of the key points of the mouth, y i Let f represent the ordinate. m To assess driver's oral fatigue level, t m T represents the number of mouth-opening frames within the detection time. m This indicates the total number of frames within the detection time.

7. The fatigue driving detection method of lightweight YOLOv8-pose according to claim 6, characterized in that: In step S3, the algorithm evaluates the model's output, comprehensively assesses various indicators to determine whether the driver is fatigued, and issues a visual warning. The trigger criteria for the visual warning are as follows: Under normal conditions, the time it takes for one eye to close is 0.1-0.15 seconds. However, under fatigue conditions, the time it takes for one eye to close is greater than 0.5 seconds. That is, if the frequency of eye closing is greater than 0.5 per unit time, the driver is judged to be in a fatigued state. The duration of a yawn in humans is usually 3-5 seconds. In the algorithm in step S3, the unit detection time is selected as 30 seconds. The number of yawns within 30 seconds cannot exceed two. That is, if the frequency of yawning is greater than 0.4 per unit time, the driver is judged to be in a fatigued state.

Images