Lightweight driver distraction behavior detection method

Abstract

The invention provides a lightweight driver distraction behavior detection method, and belongs to the technical field of computer vision. The technical problems that a distraction driving detection algorithm model is complex in structure and large in parameter quantity, and model light weight and detection precision are unbalanced are solved. According to the technical scheme, the method comprises the following steps: S1, collecting distraction driving behavior data pictures, and constructing a distraction driving behavior data set; s2, a lightweight distracted driving detection network ASU-YOLO is constructed on the basis of the YOLOv8n model; s3, training the constructed ASU-YOLO network model by using the manufactured distraction driving behavior data set to obtain a lightweight distraction driving detection model; and S4, detecting distraction driving behaviors through the trained model. The method has the beneficial effects that the detection precision of the model is improved while the parameter quantity and the calculation quantity of the algorithm are reduced, the light weight of the model is realized, and the requirement of real-time detection is met.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a lightweight driver distraction behavior detection method. Background Art

[0002] Among the many factors that lead to traffic accidents, distracted driving behavior occupies an important position. The driver's lack of concentration due to distraction is an important cause of traffic accidents, especially in rear-end collision accidents. According to the International Organization for Standardization ISO, distracted driving refers to a phenomenon in which the driver's attention is directed to activities unrelated to normal driving, resulting in a decrease in driving operation ability. Common distracted driving includes a series of inattentive driving behaviors such as playing with mobile phones, drinking water, smoking, etc. while driving.

[0003] Driver distraction behavior detection methods can be divided into three categories: based on the driver's physiological signals such as EEG and ECG, based on vehicle information such as vehicle speed, steering wheel angle and acceleration, and based on the driver's behavioral characteristics. Detection methods based on physiological signals basically require the wearing of relevant instruments and equipment, which will bring inconvenience to the driver. The detection method based on vehicle information may cause the monitoring accuracy to be too low and the deviation to be too large due to factors such as different driving habits of different drivers.

[0004] The detection method based on driver behavior characteristics has become the mainstream detection method due to its non-contact and more intuitive characteristics. This detection method usually uses a camera to obtain the driver's driving behavior data and judges the driver's distracted driving status through deep learning target detection methods. Summary of the invention

[0005] The current distraction detection methods based on driver behavior characteristics have made good progress on high-performance platforms, but most algorithm models have complex structures and large number of parameters. According to the actual distracted driving detection work, mobile terminal devices are more suitable for performing driving scene detection tasks. Therefore, on resource-limited embedded platforms, the balance between model lightweight and detection accuracy needs to be further studied; the present invention proposes a lightweight driver distraction detection method, which reduces the number of algorithm parameters and the amount of calculation while improving the detection accuracy of the model, achieving model lightweightness, and further meeting the needs of real-time detection.

[0006] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is specifically: a lightweight distracted driving behavior detection method, comprising the following steps:

[0007] S1: Collect distracted driving behavior data images and build a distracted driving behavior dataset;

[0008] S2: Based on the YOLOv8n model, a lightweight distracted driving detection network ASU-YOLO is built. A lightweight AKDM downsampling module is built in the backbone network. The collaborative attention mechanism SCSA is added to the neck network and a C2f-SCSA module is built. The Unified-IoU boundary loss function is introduced to replace the original loss function. The specific steps are as follows:

[0009] (1) Constructing the AKDM downsampling module

[0010] In the YOLOv8n network, ordinary convolution layers are used to implement downsampling. Usually, ordinary convolution layers with a stride of 2 and a convolution kernel size of 3×3 are used. This operation will reduce the size of the feature map and inevitably lose some spatial information. And with downsampling, the resolution of the feature map may decrease, and the location information of the target will become blurred, thus affecting the model's ability to capture target details. To address this problem, a new lightweight downsampling module AKDM (Adaptive KernelDown Module) is designed to reduce the number of model parameters and computation. The module mainly consists of average pooling, maximum pooling, AKConv, and 1×1 convolution.

[0011] (2) Building the C2f-SCSA module

[0012] The Bottleneck module in C2f of YOLOv8n introduces nonlinear transformations in the process of dimensionality reduction and dimensionality increase to improve the expressiveness of features. In some cases, it cannot fully focus on key features, thereby reducing the detection accuracy of the model. To address this problem, the Bottleneck structure in the C2f module is replaced with a collaborative attention module SCSA, and the improved network structure is named C2f-SCSA; the SCSA collaborative attention module is mainly composed of a shareable multi-semantic spatial attention SMSA and a progressive channel self-attention PCSA, which effectively combines the advantages of channel and spatial attention, fully utilizes multi-semantic information, and extracts rich semantic features from multi-scale spatial information.

[0013] (3) Introduce the Unified-IoU loss function to replace the original loss function

[0014] In view of the uneven quality of each image in the target detection dataset, the Unified-IoU loss function is used to replace the original loss function CIou. The Unified-IoU loss function dynamically shifts the model's attention from low-quality prediction boxes to high-quality prediction boxes in a novel way to enhance the detection performance of the model and achieve a balance in training speed.

[0015] S3: Use the prepared distracted driving behavior dataset to train the constructed ASU-YOLO network model to obtain a lightweight distracted driving detection model.

[0016] S4: Detect distracted driving behavior through the trained model. Collect the driver's distracted driving behavior in real time and input it into the distracted driving behavior detection model for identification to obtain distracted driving detection results.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] 1. The lightweight downsampling module AKDM designed by the present invention uses a combination of variable kernel convolution and pooling operations, which can dynamically and adaptively adjust to capture distracted driving target features and improve the feature extraction efficiency of the model; reduce the information loss caused by the ordinary convolution downsampling module and the information loss caused by the reduced resolution of the feature map, and increase the feature expression ability of the model. At the same time, the AKDM module is a lightweight downsampling module, which reduces the number of parameters and calculations of the model while improving the feature extraction ability of the model, thereby improving the computational efficiency of the model. 2. The C2f-SCSA module constructed by the present invention replaces the Bottleneck structure in the C2f module with the collaborative attention module SCSA, which solves the problem that the model cannot fully focus on key features in some cases, thereby improving the feature expression ability. The C2f-SCSA module can effectively combine the advantages of channel and spatial attention, make full use of multi-semantic information, extract rich semantic features from multi-scale spatial information, and improve the detection accuracy of the model.

[0019] 3. The present invention improves the loss function and introduces the Unified-IoU loss function to replace the original CIoU loss function. Unified-IoU achieves the purpose of assigning different weights to prediction frames of different qualities by enlarging or reducing the prediction frame and the GT frame. At the same time, a novel method is used to dynamically shift the model's attention from low-quality prediction frames to high-quality prediction frames, thereby improving the model's high-quality detection performance and enhancing the model's detection performance for distracted driving behavior. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0021] Figure 1 The figure is an overall flow chart of the method in the present invention.

[0022] Figure 2 This is a network structure diagram of the ASU-YOLO algorithm in the present invention.

[0023] Figure 3This is a structural diagram of the AKDM downsampling module in the present invention.

[0024] Figure 4 This is a structural diagram of the C2f-SCSA module in the present invention.

[0025] Figure 5 This is a structural diagram of the SCSA module in the present invention.

[0026] Figure 6 It is a schematic diagram of the scaled prediction box and the true box in the present invention.

[0027] Figure 7 This is the ASU-YOLO network training effect diagram in the present invention.

[0028] Figure 8 This is a visualization diagram of driver distraction behavior detection in the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. Of course, the specific embodiments described here are only used to explain the present invention and are not used to limit the present invention.

[0030] Example 1

[0031] This embodiment provides a lightweight driver distraction behavior detection method, and the flowchart of the invented method is as follows: Figure 1 As shown, the specific steps are as follows:

[0032] S1: Through the public data set SFDDD and online image collection, the data is cleaned and filtered to obtain distracted driving behavior image data. Use the labelimg annotation tool to annotate the cigarette butts, water cups, and mobile phone images in each collected data image, generate a .txt file corresponding to the yolo format, and construct a distracted driving behavior data set. The distracted driving behavior data consists of three types of distracted behavior images: smoke, drink, and phone, and contains a total of 8218 images. The data set is divided into a training set and a validation set in a ratio of 8:2.

[0033] S2: Build a lightweight distracted driving detection model ASU-YOLO, which is improved based on the YOLOv8n model network. The ASU-YOLO network structure is shown in the figure below: Figure 2 As shown. First, replace the original convolutional downsampling module with the AKDM module in the backbone network. Secondly, add the collaborative attention mechanism SCSA to the neck network to build the C2f-SCSA module and replace the original C2f module. Finally, introduce the Unified-IoU boundary loss function to replace the original CIoU loss function. The specific steps are as follows:

[0034] (1) Constructing the AKDM downsampling module

[0035] The downsampling module in the YOLOv8n network will inevitably lose some spatial information, and with downsampling, the resolution of the feature map may decrease, and the location information of the target will become blurred, thus affecting the model's ability to capture target details. To address this problem, a new lightweight downsampling module AKDM (Adaptive Kernel DownModule) is designed to improve the model's feature extraction capability while reducing the number of model parameters and computation. The module is mainly composed of average pooling, maximum pooling, variable kernel convolution AKConv, and 1×1 convolution. The structure of the AKDM downsampling module is as follows: Figure 3 shown.

[0036] The AKDM downsampling module first performs average pooling on the input feature map to retain more original feature information and reduce the size of the feature map. Then it is divided into two branches. One branch uses AKConv variable kernel convolution to dynamically adjust the size and shape of the convolution kernel to achieve more refined feature extraction, thereby better retaining image detail information and reducing information loss caused by reduced resolution of the feature map. The other branch uses maximum pooling and convolution operations to reduce the number of model parameters and improve the model's computational efficiency. Finally, the output results of the two branches are spliced ​​to obtain the final downsampling output result.

[0037] (2) Building the C2f-SCSA module

[0038] Improve the C2f module of the YOLOv8n backbone network and change it to the C2f-SCSA module, such as Figure 4 As shown in Figure 2, the C2f-SCSA module consists of a convolutional and collaborative attention module SCSA. The C2f-SCSA module consists of two parts: SMSA and PCSA. The network structure of this module is shown in Figure 2. Figure 5 As shown in the figure. SMSA uses multi-scale deep shared one-dimensional convolution to capture multi-semantic spatial information, enhances local and global feature representations, and effectively integrates global contextual dependencies and multi-semantic spatial priors. PCSA uses an input-aware sub-attention mechanism to calculate the similarity and contribution between channels, thereby alleviating the semantic differences between different sub-features in SMSA and ensuring robust feature integration across channels.

[0039] The SMSA module first decomposes the given input feature map X∈R along the height and width dimensions B×C×H×W , average pooling is performed on each dimension to obtain two new feature maps X H ∈R B×C×W and X w ∈R B×C×ΗThe new feature map is divided into four independent sub-features of the same size, and the number of features contained in each part is 1 / 4 of the original number of channels. Finally, four deep shared convolution kernels with sizes of 3, 5, 7, and 9 are used to process the features of each part to extract information of different scales.

[0040] PCSA combines single-headed self-attention (SHSA) with the modulation space prior of SMSA to calculate the similarity between channels. First, the feature map is downsampled to obtain a smaller feature map, and then the downsampled feature map is normalized. A 1×1 deep convolution is used to generate the query Q, key K, and value V for calculating the relationship between channels. The query and the key are dot-producted to calculate the attention matrix, and a scaling factor is applied to prevent numerical overflow. The final attention weights are calculated through a gating mechanism, where Dropout is used for regularization. Finally, the attention weights are matrix multiplied with the values ​​to obtain the weighted features, which are output as channel-enhanced feature maps.

[0041] (3) Introducing the Unified-IoU loss function

[0042] YOLOv8 uses the CIoU loss function. CIoU focuses on the deviation between the predicted box and the ground truth (GT) box. It adds considerations of the center distance and bounding box scale to make the target box regression more stable. However, it uses the aspect ratio to measure the difference between the two boxes. If the aspect ratio of the predicted box and the ground truth box is the same, the penalty for this item is always 0. In some distracted driving target detections that have high requirements for regression accuracy, high-quality target detection requires high-quality bounding box regression losses. Therefore, the Unified-IoU loss function is a novel way to dynamically shift the model's attention from low-quality predicted boxes to high-quality predicted boxes to enhance the model's detection performance.

[0043] The Unified-IoU loss function achieves the purpose of assigning different weights to prediction boxes of different qualities by enlarging or reducing the prediction box and the GT box. The advantage of this method is that it does not require redundant calculations of the bounding box. After obtaining the height and width of the bounding box and the coordinates of the center point, the height and width can be enlarged or reduced in a certain proportion. The schematic diagram of the scaling of the prediction box and the true box is shown in Figure 6 shown.

[0044] By shrinking the bounding box, the IoU loss between the predicted box and the true box can be amplified, which is equivalent to giving more weight to the high-quality predicted box, thereby achieving the effect of allowing the model to focus on the high-quality predicted box. By enlarging the bounding box, the weight of the low-quality predicted box can also be increased to achieve faster convergence. In order to balance the contradiction between model attention and convergence rate, a dynamic hyperparameter is used to adjust the scaling ratio of the bounding box. At the same time, a dual attention mechanism is designed for the bounding box regression (BBR) loss to further optimize the weight distribution.

[0045] S3: Train the constructed ASU-YOLO network model to obtain a lightweight distracted driving detection model. The data set is the distracted driving behavior data set prepared in S1. After the training, save the optimal model. The training effect is Figure 7 As shown. The experimental configuration is system: Ubuntu20.04; GPU: NVIDIA GeForce RTX 3090; CPU: 15vCPU Intel(R)Xeon(R)Platinum 8358P CPU@2.60GHz; the experimental environment is PyTorch 1.11.0, CUDA 11.3, Python3.8. The training epochs value is 200, the batch_size value is 32, and the initial learning rate is 0.001. The average precision mAP, model parameters Parameters, model calculation GFLOPs, and frames per second FPS are used as the evaluation indicators of the model.

[0046] In order to verify the impact of each improved module of the improved YOLOv8n network on the detection performance, an ablation experiment was conducted on the distracted driving behavior dataset to more intuitively observe the impact of each module. The experimental results are shown in Table 1, where "√" represents the use of the module on the original YOLOv8n model, and the absence of this symbol represents that it is not used.

[0047] Table 1 Ablation experiment results

[0048]

[0049] From the experimental results in Table 1, it can be seen that replacing the original downsampling module of the backbone network with the AKDM module increases the accuracy by 3.5%, reduces the number of parameters by 0.34M, and reduces the amount of calculation by 0.6. The AKDM downsampling model can reduce the number of model parameters while improving the model detection performance, which shows the high efficiency of the module. The C2f-SCSA module replaces the C2f module of the neck network. Under the premise that the number of parameters and the amount of calculation do not increase, its mAP0.5 increases by 1.5%, enhancing the feature expression ability of the model. Adding the Unified-IoU loss function to replace the original CIoU increases its mAP0.5 value by 0.4%, further improving the accuracy of model detection. Finally, the three improved modules are added to the baseline model. Compared with the original YOLOv8n model, the mAP0.5 increases by 4.3%, and the number of parameters and the amount of calculation are reduced by 11.2% and 7.3%, respectively. This proves the effectiveness of the proposed improved module in distracted driving detection, and improves the detection accuracy of the model while reducing the number of model parameters.

[0050] S4: Detect distracted driving behavior through the trained model. Collect the driver's driving behavior image data and input it into the distracted driving detection model for recognition to obtain distracted driving detection results. The ASU-YOLO algorithm proposed in this embodiment has higher detection confidence in the three distracting behaviors than the original YOLOv8n. The detection results are visualized as follows: Figure 8 shown.

[0051] Example 2

[0052] On the basis of Example 1, the algorithm proposed in this paper is compared with the current mainstream target detection algorithm to further prove the effectiveness and superiority of this algorithm. The selected algorithms include the two-stage SSD algorithm and Faster R-CNN, as well as the one-stage YOLO series of YOLOv5, YOLOv7, and YOLOv10. In this paper, YOLOv8n, the model with the smallest number of parameters in the YOLOv8 model, is selected. Therefore, in order to be fair in the experiment, YOLOv5s, YOLOv7-tiny and YOLOv10n are selected to compare the YOLO series algorithms. The comparative experimental results are shown in Table 2.

[0053] Table 2 Comparative experimental results

[0054]

[0055] As can be seen from Table 2, among the algorithms, the detection accuracy of the two-stage algorithm SSD algorithm is relatively low, and the number of parameters and calculations are large, which will affect its training and reasoning time. In contrast, the one-stage algorithm YOLO series algorithm performs better. Compared with YOLOv5s, YOLOv7-tinny, and YOLOv10n, YOLOv8n has better detection performance, and the number of parameters and calculations are the smallest. Compared with several other mainstream algorithms, the ASU-YOLO algorithm proposed in this embodiment has the highest mAP0.5 value of 95.3%, the smallest model parameter number and calculation amount, and the highest detection frame rate FPS, which proves that the algorithm proposed in this article has high detection accuracy and real-time performance while meeting lightweight requirements.

[0056] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A lightweight driver distraction behavior detection method, characterized in that: The following steps are involved: S1: Collect distracted driving behavior data images and build a distracted driving behavior dataset; S2: Based on the YOLOv8n model, a lightweight distracted driving detection network ASU-YOLO is built. A lightweight AKDM downsampling module is built in the backbone network. The collaborative attention mechanism SCSA is added to the neck network and a C2f-SCSA module is built. The Unified-IoU boundary loss function is introduced to replace the original loss function. S3: Use the prepared distracted driving behavior dataset to train the constructed ASU-YOLO network model to obtain a lightweight distracted driving detection model; S4: Detect distracted driving behavior through the trained model, collect the driver's distracted driving behavior in real time and input it into the distracted driving behavior detection model for identification, and obtain distracted driving detection results.

2. A lightweight driver distraction behavior detection method according to claim 1, characterized in that: The step S2 comprises the following steps: (1) Constructing the AKDM downsampling module In the YOLOv8n network, ordinary convolution layers are used to implement downsampling. Ordinary convolution layers with a step size of 2 and a convolution kernel size of 3×3 are used. This operation will reduce the size of the feature map and inevitably lose some spatial information. With downsampling, the resolution of the feature map may decrease, and the location information of the target will become blurred. A new lightweight downsampling module AKDM is designed to reduce the number of parameters and calculations of the model. The module is mainly composed of average pooling, maximum pooling, AKConv and 1×1 convolution. (2) Building the C2f-SCSA module The Bottleneck structure in the C2f module is replaced by the collaborative attention module SCSA, and the improved network structure is named C2f-SCSA. The SCSA collaborative attention module consists of a shareable multi-semantic spatial attention SMSA and a progressive channel self-attention PCSA, which extracts rich semantic features from multi-scale spatial information; (3) Introduce the Unified-IoU loss function to replace the original loss function The Unified-IoU loss function is used to replace the original loss function CIou. The UIoU loss function achieves a balance in training speed by dynamically shifting the model's focus from low-quality prediction boxes to high-quality prediction boxes.

3. A lightweight driver distraction behavior detection method according to claim 1, characterized in that: In step S1, the image data of driver driving behavior is searched through the public data set SFDDD and the Internet, and the data is cleaned and filtered to obtain distracted driving behavior image data. The labeling tool labelimg is used to label the cigarette butts, water cups, and mobile phone images in each collected data image, and a .txt file corresponding to the yolo format is generated to construct a distracted driving behavior data set. The distracted driving behavior data set consists of three types of distracting behaviors: smoke, drink, and phone, and contains a total of 8218 images.

4. A lightweight driver distraction behavior detection method according to claim 1, characterized in that: In the step S3, the ASU-YOLO network model is trained. The data set is the distracted driving behavior data set prepared in step S1. The experimental environment is configured, and the training parameters are set to determine the model evaluation indicators as the average precision mAP, model parameters Parameters, model calculation amount GFLOPs, and frames per second FPS. After the model training is completed, the optimal model is saved to obtain a lightweight distracted driving detection model weight file.

5. A lightweight driver distraction behavior detection method according to claim 1, characterized in that: In step S4, the trained distracted driving detection model weight file is loaded into the YOLOv8n network, and the driving behavior image video of the driver is obtained by calling the camera, and the video is analyzed and detected frame by frame, and the target in the image is detected and identified. When any one of the three types of targets, namely, a water cup, a cigarette butt, and a mobile phone, is detected in the image, it means that the driver is in distracted driving behavior, and a distracted driving detection result is obtained.

Images