Improved yolov11 night driver fatigue detection method

Abstract

This invention relates to the field of computer vision technology, and in particular to a driver fatigue detection method based on an improved YOLOv11 neural network, comprising the following steps: creating a blink image training dataset; annotating the blink images in the dataset using Roboflow; building a training model based on the dataset using the YOLOv11 algorithm; inputting the annotated blink images into the training model to train the model; after the training model matures, building a discriminant model based on the training model; and connecting the discriminant model to a monitoring camera in the driver's cab for real-time monitoring and early warning. This method shows good recognition results for driver fatigue and effectively reduces the occurrence of accidents.

Description

Technical Field

[0001] This invention relates to the field of fatigue driving detection technology, and in particular to a lightweight fatigue driving detection method for YOLOv11-n. Background Technology

[0002] Driver fatigue is one of the core risk factors causing road traffic accidents. According to a 2023 report by the National Highway Traffic Safety Administration (NHTSA), 10%-20% of road traffic accidents worldwide are attributed to fatigued driving. In the United States, there were 633 fatal accidents involving fatigued driving throughout the year, accounting for 1.5% of all traffic fatalities. Even though this figure decreased by 9.6% compared to 2022, accidents caused by fatigued driving still have the characteristics of "high fatality rate and high destructive power"—drivers' cognitive functions decline significantly when fatigued, manifested as reduced visual search efficiency, delayed situational perception, and braking reaction time that can be extended by more than 30%, posing a serious threat to driving safety. Currently, high-income countries have passed legislation to limit the continuous driving time of commercial vehicle drivers, but fatigue detection solutions at the technical level still have significant shortcomings.

[0003] Existing fatigue driving detection technologies are mainly divided into three categories: detection based on physiological signals (such as skin conductance and heart rate), detection based on driving behavior (such as vehicle speed fluctuations and emergency braking frequency), and detection based on computer vision (such as eyelid opening and closing and blinking frequency). However, all three types of technologies have limitations in single-application scenarios. 1. Physiological signal detection: Drivers need to wear special devices (such as wristbands or chest patches), which have problems such as high invasiveness, poor wearing comfort, and susceptibility to electromagnetic interference from vehicles, making it difficult to meet the non-invasive needs of daily driving; 2. Driving behavior detection: Relies on vehicle parameters such as speed and acceleration, and is easily affected by road conditions (such as congestion and curves), resulting in a high misjudgment rate. For example, it is difficult to distinguish between sudden braking under normal road conditions and sudden braking caused by fatigue, and cannot be used as a reliable basis for judgment alone. 3. Computer vision detection: As the current mainstream direction, the low contrast and high noise imaging environment in nighttime driving scenarios poses three core bottlenecks to traditional vision models: ① The region of interest (ROI) of the eye has a high degree of overlap with the background noise features, making it difficult to extract key texture information (such as eyelid edges and pupil contours); ② Micro-targets such as eyelids and pupils have insufficient feature representation in conventional detection heads, making it easy to miss detections; ③ The computing resources of in-vehicle edge devices are limited, and traditional deep learning models (such as YOLOv11 basic version) have a large number of parameters and slow inference speed, making it difficult to balance high precision and real-time performance.

[0004] To address the aforementioned pain points, existing technologies mostly focus on single-dimensional optimization (such as only improving low-light imaging quality or only compressing model size), failing to form a collaborative solution encompassing "low-light feature enhancement - lightweight architecture - small target fusion - multi-dimensional auxiliary verification." Therefore, this invention proposes a nighttime driver fatigue detection method based on a lightweight improved YOLOv11-n. By innovatively improving the YOLOv11-n backbone, neck, and head networks in a layered manner, and combining multi-dimensional fatigue judgment criteria, it achieves fatigue detection in nighttime scenarios with "high detection accuracy, low computational cost, and strong robustness," filling the gap in nighttime fatigue detection technology adapted to vehicle edge devices. Summary of the Invention

[0005] This invention proposes a lightweight YOLOv11-n fatigue driving detection method, which helps to reduce computational complexity and improve fatigue detection accuracy.

[0006] The present invention adopts the following technical solution.

[0007] A lightweight YOLOv11-n fatigue driving detection method is proposed. This method constructs a lightweight YOLOv11-n model using multi-pose, multi-view human eye datasets. A Condconv2D module is introduced into the model's backbone network to dynamically adjust convolutional kernel weights, enhancing the representation of eye texture features under low-light conditions. The neck network architecture is reconstructed using Ghostconv, compressing redundant features to improve inference speed. A SwinTransformer cross-attention mechanism is embedded in the head feature layer to construct a multi-scale eye feature association model, improving the feature fusion efficiency of small targets in occluded scenes. The method determines whether the driver is fatigued by constructing a fatigue decision model and evaluating its output.

[0008] The method includes the following steps: Step S1: Create a multi-pose, multi-view face dataset, and convert the annotation files to generate the corresponding YOLO annotation files; Step S2: Construct and train a lightweight YOLOv11-n model, use the trained model to perform face detection on the input image, and perform facial key point localization after recognizing the face. Finally, perform classification and regression on the detected head part to obtain the coordinates of the driver's eye key points. Step S3: Construct a fatigue decision model, make fatigue judgments based on eye features, use the PERCLOS evaluation criterion to determine eye opening and closing, and determine whether the driver is fatigued by detecting the number of times the driver's eyes close within a period of time. Step S4: Visualize and issue early warnings based on the driver fatigue behavior judgment results obtained from the identification and analysis in Steps S2 and S3.

[0009] In step S1, the National Tsing Hua University-DDD dataset from Taiwan is introduced as a large-scale face dataset containing multiple poses and perspectives. However, the quality of some images in this dataset does not meet the requirements, so a face image dataset collected by an infrared camera is introduced as a supplement. When the format of the annotation files in the above dataset is not the annotation format of the YOLO model, the annotation files are converted. The method is to use YOLO annotation software to process the data and generate the corresponding YOLO annotation files. In step S1, the above dataset is randomly sampled in a ratio of 6:2:2 and divided into training set, validation set and test set.

[0010] In step S2, the lightweight YOLOv11-n introduces conditional parameterized convolution CondConv2D in the backbone network, which improves the feature extraction accuracy of complex small targets with lower computational cost. In step S2, CondConv2D calculates the convolution kernel parameters by input, so that each sample can have a dedicated convolution sum, breaking the limitation of static convolution and improving the capacity of the model.

[0011] In step S2, in the lightweight YOLOv11-n, the third convolution in the backbone part is replaced with Ghostconv. By generating sparse features, the computational complexity is reduced, thereby effectively alleviating the additional computational cost brought by the backbone layer and preserving the representational ability of key features.

[0012] In step S2, the lightweight YOLOv11-n improves the head part of the original YOLOv11-n by adding a SwingTransformer module after the fourth C3k2. By introducing a hierarchical structure and a self-attention mechanism based on window offset, the feature extraction capability can be effectively improved and the computational complexity of the model can be reduced.

[0013] In step S2, a lightweight YOLOv11-n keypoint detection model is built using the PyTorch deep learning framework. The number of training rounds is set to 300, the input image size is set to 640×640, the batch size is 64, the training weights are saved every 10 rounds, and the mAP is calculated using the validation set to evaluate the model performance. Finally, the model with the largest mAP is selected as the final model.

[0014] In step S3, the eye aspect ratio EAR is calculated based on the eye key point information output by the lightweight YOLOv11-n. The EAR threshold setting is based on the P80 standard in the PERCLOS standard. P80 means that the eyelid covering more than 80% of the pupil area is considered as eye closure.

[0015] In step S3, the algorithm evaluates the model's output, comprehensively assesses various indicators to determine whether the driver is fatigued, and issues a visual warning. The trigger criteria for the visual warning are as follows: Under normal conditions, the blinking time of a single eye is 0.1-0.15 seconds. However, under fatigue conditions, the blinking time of a single eye will be greater than 0.5 seconds. If the blinking frequency is greater than 0.5 per unit time, it can be judged that the driver is in a fatigued state.

[0016] Compared with existing technologies, this invention offers the following advantages: It provides a lightweight YOLOv11-n fatigue driving detection method. This method utilizes a lightweight YOLOv11-n model to achieve face detection and facial key point localization, and then uses a fatigue decision module to determine the driver's mental state. The lightweight YOLOv11-n model introduces a Condconv2D module into the backbone network to dynamically adjust convolutional kernel weights, enhancing the representation ability of eye texture features under low-light conditions. It uses Ghostconv to reconstruct the neck network architecture, compressing redundant features to improve inference speed. Furthermore, it embeds a SwinTransformer cross-attention mechanism in the head feature layer to construct a multi-scale eye feature association model, improving the feature fusion efficiency of small targets in occluded scenes and providing strong support for the deployment of vehicle edge devices. Attached Figure Description

[0017] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0018] Appendix Figure 1 This is a flowchart illustrating the implementation of the method according to an embodiment of the present invention; Appendix Figure 2 This is a structural diagram of the key point detection model in an embodiment of the present invention; Appendix Figure 3 This is a visual schematic diagram of the fatigue detection system in an embodiment of the present invention. Detailed Implementation

[0019] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0020] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0021] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0022] like Figure 1 As shown, a lightweight YOLOv11-n fatigue driving detection method is proposed. This method constructs a lightweight YOLOv11-n model using collected multi-pose, multi-view human eye datasets. A Condconv2D module is introduced into the model's backbone network to dynamically adjust convolutional kernel weights, enhancing the representation ability of eye texture features under low-light conditions. The neck network architecture is reconstructed using Ghostconv, compressing redundant features to improve inference speed. A SwinTransformer cross-attention mechanism is embedded in the head feature layer to construct a multi-scale eye feature association model, improving the feature fusion efficiency of small targets in occluded scenes. The method determines whether the driver is fatigued by constructing a fatigue decision model and evaluating its output.

[0023] The method includes the following steps: Step S1: Collect face datasets with multiple poses and views, and convert the annotation files to generate corresponding YOLO annotation files; Step S2: Construct and train a lightweight YOLOv11-n model, use the trained model to perform face detection on the input image, and perform facial key point localization after recognizing the face. Finally, perform classification and regression on the detected head part to obtain the coordinates of the driver's eye key points. Step S3: Construct a fatigue decision model, make fatigue judgments based on eye features, use the PERCLOS evaluation criterion to determine eye opening and closing, and determine whether the driver is fatigued by detecting the number of times the driver's eyes close within a period of time. Step S4: Visualize and issue early warnings based on the driver fatigue behavior judgment results obtained from the identification and analysis in Steps S2 and S3.

[0024] In step S1, the National Tsing Hua University-DDD dataset from Taiwan is introduced as a large-scale face dataset containing multiple poses and perspectives. However, the quality of some images in this dataset does not meet the requirements, so a face image dataset collected by an infrared camera is introduced as a supplement. When the format of the annotation files in the above dataset is not the annotation format of the YOLO model, the annotation files are converted. The method is to use the official YOLO annotation software to process the data and generate the corresponding YOLO annotation files. In step S1, the above dataset is randomly sampled in a ratio of 6:2:2 and divided into training set, validation set and test set.

[0025] In step S2, the lightweight YOLOv11-n introduces conditional parameterized convolution CondConv2D in the backbone network, which improves the feature extraction accuracy of complex small targets with lower computational cost. In step S2, CondConv2D calculates the convolution kernel parameters by input, so that each sample can have a dedicated convolution sum, breaking the limitation of static convolution and improving the capacity of the model.

[0026] In step S2, in the lightweight YOLOv11-n, the third convolution in the backbone part is replaced with Ghostconv. By generating sparse features, the computational complexity is reduced, thereby effectively alleviating the additional computational cost brought by the backbone layer and preserving the representational ability of key features.

[0027] In step S2, the lightweight YOLOv11-n improves the head part of the original YOLOv11-n by adding a SwingTransformer module after the fourth C3k2. By introducing a hierarchical structure and a self-attention mechanism based on window offset, the feature extraction capability can be effectively improved and the computational complexity of the model can be reduced.

[0028] In step S2, a lightweight YOLOv11-n keypoint detection model is built using the PyTorch deep learning framework. The number of training rounds is set to 300, the input image size is set to 640×640, the batch size is 64, the training weights are saved every 10 rounds, and the mAP is calculated using the validation set to evaluate the model performance. Finally, the model with the largest mAP is selected as the final model.

[0029] In step S3, the eye aspect ratio EAR is calculated based on the eye keypoint information output by the lightweight YOLOv11-n. The EAR threshold is set with reference to the P80 standard in the PERCLOS standard. P80 means that if the area of ​​the eyelid covering the pupil exceeds 80%, it is considered that the eye is closed. The formula is as follows: Among them, p1, p2, ..., p6 and p1′ p 2′ , ..., p 6′ These are six feature points around the left and right eyes, respectively. EAR L and EAR R These are the EAR values ​​for the left and right eyes, respectively. Finally, the EAR value is obtained by averaging the EAR values ​​for the left and right eyes.

[0030] In step S3, the algorithm evaluates the model's output, comprehensively assesses various indicators to determine whether the driver is fatigued, and issues a visual warning. The trigger criteria for the visual warning are as follows: Under normal conditions, the blinking time of a single eye is 0.1-0.15 seconds, while under fatigue conditions, the blinking time of a single eye will be greater than 0.5 seconds. If the blinking frequency is greater than 0.5 per unit time, it can be judged that the driver is in a state of fatigue.

[0031] Example 1: In this example, the National Tsing Hua University-DDD dataset from Taiwan contains videos of normal driving and fatigued driving, covering both daytime and nighttime scenarios. This dataset collected multimodal data (including multi-angle facial videos and thermal infrared images) in a driving simulator environment, recording the real fatigue states of multiple subjects during prolonged simulated driving. Its core value lies in providing detailed behavioral annotations (such as alertness, fatigue level, yawning / nodding behavior), and by fusing visual, thermal imaging, and underlying physiological signals, it overcomes the limitations of single sensors, establishing a comprehensive benchmark platform for developing and evaluating more robust and accurate fatigue detection algorithms. Because the images in the dataset are not dark enough to fully simulate nighttime driving environments, a supplementary dataset of facial images collected using infrared cameras was introduced. This dataset includes photos of four test subjects with their eyes open and closed. The differences in the photos reflect individual differences among test subjects and various environmental changes, such as lighting, blurriness, and occlusion, effectively improving the robustness of model training.

[0032] In this example, the keypoint detection model constructed for fatigue detection is a lightweight YOLOv11-n model, and its model structure is as follows: Figure 2 As shown.

[0033] Specifically, the backbone network is used for image feature extraction. The second and fourth convolutional layers in the original backbone network are replaced with Condconv2D and Ghostconv modules, respectively. Under the YOLOv11n framework, the original model extracts input image features through basic convolutional modules. However, in dynamic low-light environments, the original model is not flexible enough in extracting features from complex and variable targets. The capacity improvement of conventional convolutions relies on increasing the kernel size and the number of channels, further increasing the overall computation of the network. Condconv2D, on the other hand, only requires calculating weighted convolutional kernels on the input samples using multiple experts before performing convolution calculations. Crucially, each convolutional kernel only needs to be calculated once and applied to different locations. This means that increasing the amount of expert data achieves the goal of increasing network capacity, while the code only incurs a small inference time, with each additional parameter requiring only one multiplication and addition. Condconv2D has the same capacity as a linear mixture of expert formulas with n experts, but because it only needs to calculate one expensive convolution, its computational efficiency is very high. The network structure of Condconv2D is as follows: Figure 2 As shown. The properties of Condconv can be described by the following formula: σ((α1W1+…+α n W n )*x)=σ(α1·(W1*x)+…+α n (W n *x))

[0034] This formula demonstrates the properties of CondConv2D and connects them to previous work on conditional computation and expert mixing. The routing function for each example is crucial to the performance of CondConv2D: if the learned routing function is constant for all examples, a CondConv2D layer has the same capacity as a static convolutional layer. GhostConv is a convolutional technique in convolutional neural networks that generates more feature maps through low-cost linear transformations, aiming to reduce computational burden and improve resource utilization. The network structure of GhostConv is as follows: Figure 2As shown, the GhostConv module generates features in two steps: 1) Primary feature generation: Standard convolution extracts primary features containing key information, serving as the core representation of the input data. 2) Ghost feature generation: Based on the primary features, additional Ghost features are generated using simple linear transformations (e.g., depthwise convolution). These features supplement the complete feature representation with relatively low computational cost. Finally, the main features and Ghost features are concatenated along the channel dimension to produce the final output feature map, achieving representational capabilities comparable to standard convolution while significantly reducing computational cost. It is worth noting that the first convolutional layer in the network is still a standard convolution, not GhostConv. This is because while GhostConv is efficient, it may lose local details in high-resolution inputs, making it less suitable for capturing initial features at the beginning of the network. More specifically, after obtaining the image, GhostConv first uses a convolution to generate m original feature maps, i.e., primary features. The generating formula can be described as: Y′=X·f′ In the formula, These are input features. The convolution kernel used is denoted by f′, where k×k represents the kernel size of the convolution kernel f′. The bias term is omitted here, and all hyperparameters are the same as those in ordinary convolution, ensuring that the spatial sizes h′ and w′ of the output features remain consistent. Next, the feature maps y′ of each channel of Y′ are... i Using linear transformation Φ i，j Obtain the ghost feature map y ij The linear transformation formula is:

[0035] Finally, the intrinsic feature map obtained in the first step is concatenated with the Ghost feature map generated through linear operations to obtain the final output feature map. The introduction of GhostConv significantly reduces redundant computations in the feature fusion process, improving model performance and inference speed. Compared with traditional convolution, Ghostconv generates a large number of feature maps with only a small amount of computation, achieving better performance with the same computing resources. This improvement enhances image quality in low-light environments and enables fatigue detection algorithms to maintain high accuracy while achieving faster processing speeds, ensuring the real-time performance of the model.

[0036] The SwinTransformer module was introduced in LES-YOLO, improving the head portion of the original YOLOv11-n. The SwinTransformer module is added after the fourth C3k2. Its displacement window self-attention mechanism overcomes the scale and resolution differences faced by traditional Transformers in visual tasks, enabling the model to extract driver fatigue features more accurately in low-light environments. Simultaneously, SwinTransformer can be seamlessly integrated into the YOLOv11 head network, maintaining YOLOv11's fast detection advantage. This combination not only improves detection performance but also reduces the model's computational complexity. SwinTransformer is a visual model based on the Transformer architecture. By introducing a hierarchical structure and a window-offset-based self-attention mechanism, it effectively enhances feature extraction capabilities. Its architecture diagram is shown below. Figure 2 As shown, SwinTransformer mainly consists of Patch Partitioning, Linear Embedding, Swin Transformer Blocks, Window-based Multi-Head Self-Attention (W-MSA), and Shifted Window Multi-Head Self-Attention (SW-MSA). The calculation formulas for MSA and W-MSA can be described as follows: Ω(MSA)=4hwC 2 +2(hw) 2 C Ω(W-MSA)=4hwC 2 +2M 2 hwC Where h represents the feature map height, w represents the feature map width, C represents the feature map depth, and M represents the size of each window.

[0037] In this example, step S3 specifically involves: constructing a fatigue decision model, providing fatigue judgment based on eye features, using the p80 algorithm in the PERCLOS evaluation criterion to determine whether the driver's eyes are open or closed, and judging whether the driver is fatigued by detecting the number of times the driver's eyes close within a certain period of time.

[0038] Specifically, an image is input using a lightweight YOLOv11-n network model. After detecting the face region in the image, facial key points are extracted and their locations are plotted on the image. Traditional facial key points consist of 68 points, such as... Figure 2As shown, in actual testing, only the driver's eyes are needed to determine driver fatigue. Therefore, this invention selects only 12 key facial points for fatigue assessment, specifically 12 key points (37-48) for the left and right eyes. The Euclidean distance of each point is obtained using the x and y coordinates of the key eye points, and the aspect ratio (EAR) of the eye is calculated. The EAR threshold is referenced to PERCLOS, an internationally recognized fatigue assessment standard that refers to the proportion of time the eyes are closed within a certain period. PERCLOS criteria include P70, P80, and EM, which represent eye closure when the eyelid covers more than 70%, 80%, and 50% of the pupil area, respectively. Among these, P80 is considered the most sensitive standard for fatigue assessment.

[0039] The eye condition evaluation index in this article refers to the P80 standard in the PERCLOS standard, which sets the EAR threshold at 0.2. When the EAR is less than 0.2, the eye is considered closed. The formula is as follows: Among them, p1, p2, ..., p6 and p 1′ p 2′ , ..., p 6′ These are six feature points around the left and right eyes, respectively. EAR L and EAR R These are the EAR values ​​for the left and right eyes, respectively. Finally, the EAR value is obtained by averaging the EAR values ​​for the left and right eyes.

[0040] Specifically, to test the generalization ability and effectiveness of the lightweight YOLOv11-n model in a driving environment, this paper uses the recognition rate of a self-made hybrid dataset based on driving detection videos from National Tsing Hua University, Taiwan (DDD), as the evaluation metric. The video data was collected from the driver's seat of a car, including forward-facing and oblique-facing video captures. The captured video content includes normal driving, talking, and yawning due to fatigue. Following the method described above, the EAR (Earnings Average Expectation) was calculated to detect the number of times the driver closed their eyes. Figure 3 As shown, the lightweight YOLOv11-n model can accurately detect and track driver blinks during driving and perform effective identification statistics and driving state judgment. Even when the driver is wearing glasses or the driving environment is dimly lit, the model can still identify the driver's fatigue state.

[0041] In summary, the lightweight YOLOv11-n fatigue driving detection method of the present invention can effectively improve the feature extraction capability and detection accuracy of the model algorithm for key points, improve the lightweight performance of the model algorithm, and greatly improve the real-time performance of the system.

[0042] Many specific details have been set forth in the foregoing description to provide a thorough understanding of the present invention. However, the above description is merely a preferred embodiment of the present invention, and the present invention can be implemented in many other ways different from those described herein. Therefore, the present invention is not limited to the specific embodiments disclosed above. Furthermore, any person skilled in the art can make many possible variations and modifications to the technical solutions of the present invention, or modify them into equivalent embodiments, using the methods and techniques disclosed above, without departing from the scope of the present invention. Any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention, without departing from the content of the present invention, shall still fall within the protection scope of the present invention.

Claims

1. A fatigue driving detection method based on an improved YOLOv11-n, characterized in that... The method constructs a lightweight YOLOv11-n model using collected multi-pose and multi-view human eye datasets. A Condconv2D module is introduced into the model's backbone network to dynamically adjust convolutional kernel weights, enhancing the representation of eye texture features under low-light conditions. Ghostconv is used to reconstruct the neck network architecture, compressing redundant features to improve inference speed. A Swin Transformer cross-attention mechanism is embedded in the head feature layer to construct a multi-scale eye feature association model, improving the feature fusion efficiency of small targets in occluded scenes. The method also constructs a fatigue decision model and evaluates its output to determine whether the driver is fatigued.

2. The fatigue driving detection method for lightweight YOLOv11-n according to claim 1, characterized in that... The method includes the following steps; Step S1: Create a multi-pose, multi-view face dataset, and convert the annotation files to generate the corresponding YOLO annotation files; Step S2: Construct and train a lightweight YOLOv11-n model, use the trained model to perform face detection on the input image, and perform facial key point localization after recognizing the face. Finally, perform classification and regression on the detected head part to obtain the coordinates of the driver's eye key points. Step S3: Construct a fatigue decision model, make fatigue judgment based on eye features, use the PERCLOS evaluation criterion to determine eye opening and closing, and determine whether the driver is in a fatigued state by detecting the number of times the driver's eyes close within a period of time. Step S4: Visualize and issue early warnings based on the driver fatigue behavior judgment results obtained from the identification and analysis in Steps S2 and S3.

3. The fatigue driving detection method for lightweight YOLOv11-n according to claim 2, characterized in that... In step S1, the NTHU-DDD dataset is introduced as a large-scale face dataset containing multiple poses and perspectives. However, the quality of some images in this dataset does not meet the requirements, so a face image dataset collected by an infrared camera is introduced as a supplement. When the format of the annotation files in the above dataset is not the annotation format of the YOLO model, the annotation files are converted. The method is to use the official YOLO annotation software to process the data and generate the corresponding YOLO annotation files. In step S1, the above dataset is randomly sampled in a ratio of 6:2:2 and divided into training set, validation set and test set.

4. The fatigue driving detection method for lightweight YOLOv11-n according to claim 2, characterized in that... In step S2, the lightweight YOLOv11-n introduces conditional parameterized convolution CondConv2D in the backbone network, which improves the feature extraction accuracy of complex small targets with lower computational cost. In step S2, CondConv2D calculates the convolution kernel parameters by input, so that each sample can have a dedicated convolution sum, breaking the limitation of static convolution and improving the capacity of the model.

5. The fatigue driving detection method for lightweight YOLOv11-n according to claim 4, characterized in that... In step S2, in the lightweight YOLOv11-n, the third convolution in the backbone part is replaced with Ghostconv. By generating sparse features, the computational complexity is reduced, thereby effectively alleviating the additional computational cost brought by the backbone layer and preserving the representational ability of key features.

6. The fatigue driving detection method for lightweight YOLOv11-n according to claim 4, characterized in that... In step S2, the lightweight YOLOv11-n improves the head part of the original YOLOv11-n by adding a Swing Transformer module after the fourth C3k2. By introducing a hierarchical structure and a self-attention mechanism based on window offset, the feature extraction capability can be effectively improved and the computational complexity of the model can be reduced.

7. The fatigue driving detection method for lightweight YOLOv11-n according to claim 5, characterized in that... In step S2, a lightweight YOLOv11-n keypoint detection model is built using the PyTorch deep learning framework. The number of training rounds is set to 300, the input image size is set to 640×640, the batch size is 64, the training weights are saved every 10 rounds, and the mAP is calculated using the validation set to evaluate the model performance. Finally, the model with the largest mAP is selected as the final model.

8. The fatigue driving detection method for lightweight YOLOv11-n according to claim 2, characterized in that... In step S3, based on the eye keypoint information output by the lightweight YOLOv11-n, the eye aspect ratio EAR is calculated. The EAR threshold is set with reference to the P80 standard in the PERCLOS standard. P80 means that if the area of ​​the eyelid covering the pupil exceeds 80%, it is considered that the eye is closed. The formula is as follows: Among them, p1, p2, ..., p6 and p 1′ ,p 2′ ,…,p 6′ These are six feature points around the left and right eyes, respectively. EAR L and EAR R These are the EAR values ​​for the left and right eyes, respectively. Finally, the EAR value is obtained by averaging the EAR values ​​for the left and right eyes.

9. The fatigue driving detection method for lightweight YOLOv11-n according to claim 8, characterized in that... In step S3, the algorithm evaluates the model's output, comprehensively assesses various indicators to determine whether the driver is fatigued, and issues a visual warning. The triggering criteria for the visual warning are as follows: Under normal conditions, the blinking time of a single eye is 0.1-0.15 seconds. However, under fatigue conditions, the blinking time of a single eye will be greater than 0.5 seconds. If the blinking frequency is greater than 0.5 per unit time, it can be judged that the driver is in a fatigued state.

Images