A lightweight driver distraction detection method

Abstract

This invention provides a lightweight method for detecting distracted driving behavior, belonging to the field of computer vision technology. It solves the technical problems of complex model structure, large number of parameters, and the imbalance between model lightweighting and detection accuracy in distracted driving detection algorithms. The technical solution includes the following steps: S1: Collect distracted driving behavior data images to construct a distracted driving behavior dataset; S2: Construct a lightweight distracted driving detection network, ASU-YOLO, based on the YOLOv8n model; S3: Train the constructed ASU-YOLO network model using the created distracted driving behavior dataset to obtain a lightweight distracted driving detection model; S4: Detect distracted driving behavior using the trained model. The beneficial effects of this invention are: it improves the detection accuracy of the model while reducing the number of parameters and computational load, achieving model lightweighting and meeting the needs of real-time detection.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a lightweight method for detecting driver distraction. Background Technology

[0002] Among the many factors leading to traffic accidents, distracted driving occupies a significant position. Drivers' lack of concentration due to distraction is a major cause of traffic accidents, especially rear-end collisions. According to the International Organization for Standardization (ISO), distracted driving refers to a phenomenon where a driver's attention is directed towards activities unrelated to normal driving, resulting in a decline in driving ability. Common examples of distracted driving include using a mobile phone while driving, drinking water, smoking, and other behaviors that demonstrate a lack of concentration.

[0003] Methods for detecting driver distraction can be categorized into three types: those based on the driver's physiological signals, such as electroencephalograms (EEGs) and electrocardiograms (ECGs); those based on vehicle information, such as vehicle speed, steering wheel angle, and acceleration; and those based on driver behavioral characteristics. Physiological signal-based methods generally require drivers to wear related equipment, which can be inconvenient. Vehicle information-based methods may suffer from low accuracy and significant bias due to differences in driving habits among drivers.

[0004] Detection methods based on driver behavior characteristics have become the mainstream detection approach due to their non-contact and more intuitive features. These methods typically use cameras to acquire driver behavior data and then use deep learning-based target detection to determine the driver's distracted driving state. Summary of the Invention

[0005] While current methods for detecting distracted driving behavior based on driver behavior features have made significant progress on high-performance platforms, most algorithms suffer from complex models and large parameter counts. Given the real-world requirements of distracted driving detection, mobile devices are better suited for driving scenario detection tasks. Therefore, further research is needed to balance model lightweighting and detection accuracy on resource-constrained embedded platforms. This invention proposes a lightweight method for detecting distracted driver behavior that improves detection accuracy while reducing the number of parameters and computational load, achieving model lightweighting and further meeting the demands of real-time detection.

[0006] To achieve the above-mentioned objectives, the present invention employs the following technical solution: a lightweight method for detecting distracted driving behavior, comprising the following steps:

[0007] S1: Collect images of distracted driving behavior data and construct a distracted driving behavior dataset;

[0008] S2: Based on the YOLOv8n model, a lightweight distracted driving detection network, ASU-YOLO, is constructed. A lightweight AKDM downsampling module is built in the backbone network. A collaborative attention mechanism, SCSA, is added to the neck network, and a C2f-SCSA module is constructed. The Unified-IoU boundary loss function is introduced to replace the original loss function. The specific steps are as follows:

[0009] (1) Constructing the AKDM downsampling module

[0010] Downsampling in the YOLOv8n network is achieved using ordinary convolutional layers, typically with a stride of 2 and a kernel size of 3x3. This operation reduces the size of the feature map, inevitably resulting in the loss of some spatial information. Furthermore, as downsampling occurs, the resolution of the feature map decreases, blurring the target's location information and affecting the model's ability to capture target details. To address this issue, a novel lightweight downsampling module, AKDM (Adaptive Kernel Down Module), was designed to reduce the model's parameter count and computational cost. This module primarily consists of average pooling, max pooling, AKConv, and 1x1 convolutions.

[0011] (2) Constructing the C2f-SCSA module

[0012] The Bottleneck module in YOLOv8n's C2f module introduces nonlinear transformations during dimensionality reduction and expansion to improve feature representation. However, in some cases, it fails to adequately focus on key features, thus reducing the model's detection accuracy. To address this issue, the Bottleneck structure in the C2f module is replaced with a Collaborative Attention Module (SCSA), and the improved network structure is named C2f-SCSA. The SCSA collaborative attention module mainly consists of a shareable multi-semantic spatial attention module (SMSA) and a progressive channel self-attention module (PCSA), effectively combining the advantages of channel and spatial attention to fully utilize multi-semantic information and extract rich semantic features from multi-scale spatial information.

[0013] (3) Introduce the Unified-IoU loss function to replace the original loss function.

[0014] To address the issue of varying image quality across object detection datasets, the original loss function CIou is replaced with the Unified-IoU loss function. The Unified-IoU loss function dynamically shifts the model's focus from low-quality predicted boxes to high-quality predicted boxes in a novel way, thereby enhancing the model's detection performance and achieving a balance between training speed and performance.

[0015] S3: Train the constructed ASU-YOLO network model using the created distracted driving behavior dataset to obtain a lightweight distracted driving detection model.

[0016] S4: Detect distracted driving behavior using a trained model. Real-time data collection of driver distracted driving behavior is fed into the distracted driving behavior detection model for identification, yielding the distracted driving detection results.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0018] 1. The lightweight downsampling module AKDM designed in this invention combines variable kernel convolution with pooling operations, enabling dynamic adaptive adjustment to capture features of distracted driving targets and improve the model's feature extraction efficiency. It reduces information loss caused by ordinary convolutional downsampling modules and the reduced resolution of feature maps, increasing the model's feature representation capability. Simultaneously, the AKDM module is a lightweight downsampling module, improving the model's feature extraction capability while reducing the number of parameters and computational cost, thus improving computational efficiency. 2. The C2f-SCSA module constructed in this invention replaces the Bottleneck structure in the C2f module with the Collaborative Attention Module (SCSA), addressing the problem that the model cannot fully focus on key features in some situations, thereby improving feature representation capability. The C2f-SCSA module effectively combines the advantages of channel and spatial attention, fully utilizing multi-semantic information to extract rich semantic features from multi-scale spatial information, improving the model's detection accuracy.

[0019] 3. This invention improves the loss function by introducing the Unified-IoU loss function to replace the original CIoU loss function. Unified-IoU achieves the purpose of assigning different weights to prediction boxes of different qualities by enlarging or shrinking the predicted boxes and ground truth boxes. At the same time, it uses a novel method to dynamically transfer the model's attention from low-quality prediction boxes to high-quality prediction boxes, thereby improving the model's high-quality detection performance and enhancing the model's performance in detecting distracted driving behavior. Attached Figure Description

[0020] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0021] Figure 1 This is a flowchart of the overall method in this invention.

[0022] Figure 2 This is a network structure diagram of the ASU-YOLO algorithm in this invention.

[0023] Figure 3This is a structural diagram of the AKDM downsampling module in this invention.

[0024] Figure 4 This is a structural diagram of the C2f-SCSA module in this invention.

[0025] Figure 5 This is a structural diagram of the SCSA module in this invention.

[0026] Figure 6 This is a schematic diagram showing the scaling of the predicted bounding box and the ground truth bounding box in this invention.

[0027] Figure 7 This is a training result diagram of the ASU-YOLO network in this invention.

[0028] Figure 8 This is a visualization of the driver distraction behavior detection in this invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. Of course, the specific embodiments described herein are merely illustrative and not intended to limit the invention. Example

[0030] This embodiment provides a lightweight method for detecting driver distraction behavior, and the flowchart of the method is shown below. Figure 1 As shown, the specific steps are as follows:

[0031] S1: The data was cleaned and filtered to obtain distracted driving behavior image data. The labelimg tool was used to label cigarette butts, water cups, and mobile phone images in each collected image, generating corresponding .txt files in YOLO format to construct the distracted driving behavior dataset. This dataset consists of images representing three categories of distracted driving behavior: smoke, drink, and phone, containing a total of 8218 images. The dataset was divided into training and validation sets in an 8:2 ratio.

[0032] S2: Construct a lightweight distracted driving detection model, ASU-YOLO, based on the YOLOv8n model network with improvements. The ASU-YOLO network structure diagram is shown below. Figure 2 As shown. First, the original convolutional downsampling module in the backbone network is replaced with an AKDM module. Second, a collaborative attention mechanism (SCSA) is added to the neck network to construct a C2f-SCSA module and replace the original C2f module. Finally, the Unified-IoU boundary loss function is introduced to replace the original CIoU loss function. The specific steps are as follows:

[0033] Constructing an AKDM downsampling module

[0034] The downsampling module in the YOLOv8n network inevitably loses some spatial information. Furthermore, with downsampling, the resolution of the feature map may decrease, blurring the target's location information and affecting the model's ability to capture target details. To address this issue, a new lightweight downsampling module, AKDM (Adaptive Kernel DownModule), was designed. This module improves the model's feature extraction capabilities while reducing the number of parameters and computational cost. It mainly consists of average pooling, max pooling, a variable kernel convolution AKConv, and a 1x1 convolution. The structure of the AKDM downsampling module is as follows: Figure 3 As shown.

[0035] The AKDM downsampling module first applies average pooling to the input feature map, preserving more original feature information while reducing the feature map size. Then, it splits into two branches. One branch uses AKConv variable kernel convolution, dynamically adjusting the kernel size and shape to achieve finer feature extraction, thus better preserving image details and reducing information loss due to reduced feature map resolution. The other branch uses max pooling and convolution operations to reduce the number of model parameters and improve computational efficiency. Finally, the outputs of the two branches are concatenated to obtain the final downsampled output.

[0036] (2) Constructing the C2f-SCSA module

[0037] The C2f module in the YOLOv8n backbone network is improved and replaced with a C2f-SCSA module, such as... Figure 4 As shown, the C2f-SCSA module consists of both a convolutional and collaborative attention module (SCSA). The C2f-SCSA module comprises two parts: SMSA and PCSA. The network structure diagram of this module is shown below. Figure 5 As shown, SMSA utilizes multi-scale depth-shared one-dimensional convolution to capture multi-semantic space information, enhancing local and global feature representations and effectively integrating global contextual dependencies and multi-semantic space priors. PCSA employs an input-aware sub-attention mechanism to compute similarity and contribution between channels, thereby mitigating semantic differences between different sub-features in SMSA and ensuring robust feature integration across channels.

[0038] The SMSA module first decomposes the given input feature map along the height and width dimensions. Average pooling is performed on each dimension to obtain two new feature maps. and The newly obtained feature map is divided into four independent sub-features of equal size, each containing one-quarter of the original number of channels. Finally, four one-dimensional convolutions with shared depth kernels of sizes 3, 5, 7, and 9 are used to process each sub-feature to extract information at different scales.

[0039] PCSA combines Single-Head Self-Attention (SHSA) with the modulation space prior of Smart-Signal (SMSA) to calculate inter-channel similarity. First, the feature map is downsampled to obtain a smaller feature map. Then, the downsampled feature map is normalized, and a 1×1 depthwise convolution is used to generate a query Q, key K, and value V for calculating inter-channel relationships. The query and key are then dot-productd to calculate the attention matrix, and a scaling factor is applied to prevent numerical overflow. The final attention weights are calculated using a gating mechanism, with Dropout used for regularization. Finally, the attention weights and values ​​are matrix multiplied to obtain a weighted feature map, which is output as the channel-enhanced feature map.

[0040] (3) Introduce the Unified-IoU loss function

[0041] YOLOv8 employs the CIOU loss function. CIoU primarily focuses on the deviation between the predicted bounding box and the ground truth (GT) box, incorporating considerations of center distance and bounding box scale to make object box regression more stable. However, it uses aspect ratio to measure the difference between the two boxes; if the aspect ratios of the predicted and ground truth boxes are the same, the penalty for this term is always 0. In distracted driving object detection where high regression accuracy is required, high-quality object detection necessitates high-quality bounding box regression loss. Therefore, the Unified-IoU loss function dynamically shifts the model's focus from low-quality predicted boxes to high-quality predicted boxes in a novel way, thereby enhancing the model's detection performance.

[0042] The Unified-IoU loss function achieves the goal of assigning different weights to predicted boxes of varying quality by scaling up or down the predicted bounding boxes and ground truth boxes. The advantage of this method is that it avoids redundant computation of the bounding boxes. After obtaining the height, width, and center point coordinates of the bounding boxes, the height and width can be scaled up or down proportionally. A diagram illustrating the scaling of the predicted and ground truth boxes is shown below. Figure 6 As shown.

[0043] By shrinking the bounding box, the IoU loss between the predicted and ground truth boxes can be amplified. This is equivalent to giving more weight to high-quality predicted boxes, thus allowing the model to focus on them. Conversely, enlarging the bounding box increases the weight of low-quality predicted boxes, leading to faster convergence. To balance the trade-off between model attention and convergence rate, a dynamic hyperparameter is used to adjust the bounding box scaling ratio. A dual-attention mechanism is also designed for the bounding box regression (BBR) loss to further optimize weight allocation.

[0044] S3: Train the constructed ASU-YOLO network model to obtain a lightweight distracted driving detection model. The dataset is the distracted driving behavior dataset prepared in S1. After training, save the optimal model. The training effect is... Figure 7 As shown. The experimental configuration was as follows: System: Ubuntu 20.04; GPU: NVIDIA GeForce RTX 3090; CPU: 15 vCPU Intel(R) Xeon(R) Platinum 8358P CPU @ 2.60GHz; Experimental environment: PyTorch 1.11.0, CUDA 11.3, Python 3.8. The training epochs were 200, the batch size was 32, and the initial learning rate was 0.001. Average precision (mAP), model parameters, model computation (GFLOPs), and frames per second (FPS) were used as evaluation metrics for the model.

[0045] To verify the impact of each improved module of the modified YOLOv8n network on detection performance, ablation experiments were conducted on a distracted driving behavior dataset to more intuitively observe the effect of each module. The experimental results are shown in Table 1, where "Ö" indicates that the module was used on the original YOLOv8n model, and the absence of this symbol indicates that it was not used.

[0046] Table 1 Ablation Experiment Results

[0047]

[0048] As shown in Table 1, replacing the original downsampling module in the backbone network with the AKDM module increased accuracy by 3.5%, reduced the number of parameters by 0.34M, and reduced computation by 0.6. The AKDM downsampling model improves detection performance while reducing the number of model parameters, demonstrating its high efficiency. Replacing the C2f module in the neck network with the C2f-SCSA module improved mAP0.5 by 1.5% without increasing the number of parameters or computation, enhancing the model's feature representation ability. Adding the Unified-IoU loss function to replace the original CIoU improved mAP0.5 by 0.4%, further improving the model's detection accuracy. Finally, adding all three improved modules to the baseline model, compared with the original YOLOv8n model, resulted in a 4.3% increase in mAP0.5, and reductions in the number of parameters and computation by 11.2% and 7.3%, respectively. This demonstrates the effectiveness of the proposed improved modules in distracted driving detection, improving detection accuracy while reducing the number of model parameters.

[0049] S4: Detect distracted driving behavior using the trained model. Driver behavior image data is collected and input into the distracted driving detection model for identification, yielding the detection results. The ASU-YOLO algorithm proposed in this embodiment achieves higher detection confidence levels than the original YOLOv8n for all three types of distracted driving behaviors. The detection results are visualized as follows: Figure 8 As shown. Example

[0050] Based on Example 1, the algorithm proposed in this paper is compared with current mainstream object detection algorithms to further demonstrate its effectiveness and superiority. The selected algorithms include the two-stage SSD algorithm and Faster R-CNN, as well as the one-stage YOLO series YOLOv5, YOLOv7, and YOLOv10. In this paper, YOLOv8n, the model with the fewest parameters in the YOLOv8 model, is selected. Therefore, for the sake of experimental fairness, YOLOv5s, YOLOv7-tiny, and YOLOv10n are selected for comparison with the YOLO series algorithms. The comparison results are shown in Table 2.

[0051] Table 2 Comparison of experimental results

[0052]

[0053] As shown in Table 2, among the various algorithms, the two-stage SSD algorithm has relatively low detection accuracy and a large number of parameters and computational cost, which affects its training and inference time. In contrast, the one-stage YOLO series algorithms perform better. YOLOv8n has better detection performance than YOLOv5s, YOLOv7-tinny, and YOLOv10n, and has the fewest parameters and computational cost. The ASU-YOLO algorithm proposed in this embodiment has the highest mAP of 95.3% compared to other mainstream algorithms, with the fewest model parameters and computational cost, and the highest detection frame rate (FPS), proving that the proposed algorithm achieves high detection accuracy and real-time performance while maintaining lightweight design.

[0054] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A lightweight method for detecting driver distraction behavior, characterized in that, Includes the following steps: S1: Collect images of distracted driving behavior data and construct a distracted driving behavior dataset; S2: Based on the YOLOv8n model, a lightweight distracted driving detection network ASU-YOLO is constructed. A lightweight AKDM downsampling module is constructed in the backbone network. The collaborative attention mechanism SCSA is added to the neck network and a C2f-SCSA module is constructed. The Unified-IoU boundary loss function is introduced to replace the original loss function. Step S2 includes the following steps: (1) Constructing the AKDM downsampling module This module mainly consists of average pooling, max pooling, AKConv, and 1x1 convolution. The AKDM downsampling module first performs average pooling on the input feature map, then divides it into two branches. One branch uses AKConv variable kernel convolution, and the other branch uses max pooling and convolution operations. Finally, the outputs of the two branches are concatenated to obtain the final downsampling output. (2) Constructing the C2f-SCSA module The Bottleneck structure in the C2f module is replaced with the Collaborative Attention Module (SCSA), and the improved network structure is named C2f-SCSA. The SCSA Collaborative Attention Module consists of a shareable multi-semantic spatial attention (SMSA) and a progressive channel self-attention (PCSA) module, which extracts rich semantic features from multi-scale spatial information. (3) Introduce the Unified-IoU loss function to replace the original loss function. The original loss function CIou is replaced by the Unified-IoU loss function. The Unified-IoU loss function achieves a balance in training speed by dynamically shifting the model's focus from low-quality prediction boxes to high-quality prediction boxes. S3: Train the constructed ASU-YOLO network model using the created distracted driving behavior dataset to obtain a lightweight distracted driving detection model; S4: Detect distracted driving behavior using a trained model, collect driver distracted driving behavior data in real time and input it into the distracted driving behavior detection model for identification, and obtain the distracted driving detection result.

2. The lightweight driver distraction behavior detection method according to claim 1, characterized in that, In step S1, the data is cleaned and filtered to obtain distracted driving behavior image data. The labelimg tool is used to label cigarette butts, water cups, and mobile phone images in each collected data image, generating corresponding .txt files in YOLO format to construct a distracted driving behavior dataset. This distracted driving behavior dataset consists of three types of distracted behaviors: smoke, drink, and phone, and contains a total of 8218 images.

3. The lightweight driver distraction behavior detection method according to claim 1, characterized in that, In step S3, the ASU-YOLO network model is trained. The dataset is the distracted driving behavior dataset prepared in step S1. The experimental environment is configured, and the training parameters are set to determine the model evaluation metrics as mean accuracy (mAP), model parameters, model computation (GFLOPs), and frames per second (FPS). After the model training is completed, the optimal model is saved to obtain a lightweight distracted driving detection model weight file.

4. The lightweight driver distraction behavior detection method according to claim 3, characterized in that, In step S4, the trained distracted driving detection model weight file is loaded into the YOLOv8n network. The driver's driving behavior image video is acquired by calling the camera, and the video is analyzed and detected frame by frame. The target in the image is detected and identified. When any one of the three types of targets, such as a water cup, a cigarette butt, or a mobile phone, is detected in the image, it is considered distracted driving behavior, and the distracted driving detection result is obtained.

Images