Distraction driving behavior detection method based on single-stage algorithm

Abstract

The invention discloses a distraction driving behavior detection method based on a single-stage algorithm, and the method comprises the steps: employing a self-made distraction driving behavior data set based on an improved YOLOv8 algorithm; a weighted bidirectional feature pyramid network comprising a small target detection layer and a novel connection structure is designed on the basis of YOLOv8, and an efficient multi-scale EMA attention mechanism module is introduced into a backbone network to enhance learning and extraction of key information. In addition, according to the provided novel C2fODEMA structure, dynamic convolution and an EMA attention mechanism are integrated into C2f, and the situations of missing detection and false detection of small targets are remarkably reduced. The system provides a more efficient and more accurate solution for the distraction driving behavior detection field, has good generalization performance, and provides powerful support for road safety and traffic supervision.

Description

Technical Field

[0001] The present invention relates to the field of machine vision and target detection, and in particular to a distracted driving behavior detection method based on improved YOLOv8. Background Art

[0002] Distracted driving is one of the main causes of traffic accidents. Distracted driving behaviors include using mobile phones, drinking water and smoking, which distract the driver's attention and increase the risk of traffic accidents. Therefore, developing an effective distracted driving behavior detection method is of great significance to improving traffic safety. There are mainly the following methods for distracted driving: detection methods based on driver physiological signals, detection methods based on deep learning, and detection methods based on vehicle driving data.

[0003] At present, computer vision-based target detection technology has broad application prospects in the field of driving behavior monitoring. The mainstream single-stage algorithms in the existing technology are YOLOv3, YOLOv5, YOLOv7, YOLOv8, SSD and the two-stage algorithm Faster R-CNN. YOLOv8 has become the leading single-stage target detection algorithm with its excellent performance, multi-scale adaptability and efficient detection speed. Since its inception, the YOLO (You Only Look Once) target detection algorithm has evolved to the current YOLOv8 version after continuous iterative development. In order to meet the requirements of different inference speeds, models of different scales are generated by compound scaling methods, such as v8n, v8s and v8l. It has also derived a number of excellent algorithms based on its architecture. Summary of the invention

[0004] The technical problems to be solved by the present invention are: the current algorithm has low detection accuracy in actual environments, and the detection method based on deep learning fails to fully consider the limitations of the actual vehicle-mounted host hardware equipment on the model size and the number of model parameters.

[0005] The present invention is specifically implemented by the following technical solution: a distracted driving behavior detection method based on improved YOLOv8, specifically comprising the following steps:

[0006] Step 1: Create a self-made driver distraction behavior dataset, use data enhancement methods to pre-process the images, and divide them into training sets, validation sets, and test sets in proportion;

[0007] Step 2: Construct an improved YOLOv8 network feature extraction module, and use the EMA attention mechanism before the feature extraction module SPPF;

[0008] Step 3: Build an improved YOLOv8 network feature fusion module, add a small target detection layer to the feature fusion module, build a weighted bidirectional feature pyramid network with a small target detection layer and a new connection structure, and design a C2f_ODEMA module that integrates the EMA attention mechanism and dynamic convolution, conduct an embedding position comparison experiment, and finally select the 26th layer to replace the C2f structure in the small target feature fusion;

[0009] Step 4: Build an improved YOLOv8 network detection head module and add a small target detection head to the detection part;

[0010] Step 5: Input the above training set into the improved YOLOv8 network for training, and evaluate the model improvement according to the preset evaluation index;

[0011] Step 6: Perform generalization training on a public dataset with more images and object categories than the dataset in this article, and evaluate the improved model based on the preset evaluation indicators.

[0012] The step 1 specifically includes the following contents:

[0013] Labelimg was used to create a YOLO format distracted driving dataset, which includes three categories: cigarettes, water cups, and mobile phones. Taking into account the data source and data diversity, some data came from the Kaggle competition State-Farm. These data are mainly side views taken by fixed cameras with high background repetition rate. Multi-background front views were obtained from sources such as Google to promote the comprehensiveness and robustness of model training; the self-made dataset was enhanced by MOSAIC data enhancement, random cropping, scaling, etc., and finally divided into training set, validation set, and test set in a ratio of 7:1:2.

[0014] The steps 2-4 specifically include the following contents:

[0015] Create attention.py in YOLOv8 and add EMA attention definition; add dynamic convolution ODConv definition in block.py, write ODBottlenck definition and replace the two-dimensional convolution in Bottlenck with ODConv2d. Write C2f_ODEMA definition, replace Bottlenck with the previously defined ODBottlenck, add EMA attention mechanism before ODBottlenck and after Split. Finally, register EMA attention mechanism and C2f_ODEMA module in tasks.py.

[0016] Add BiFPN weighted bidirectional pyramid structure, improve BiFPN feature fusion network, and add a small target detection layer P2 on the basis of the existing structure. At the same time, the bottom-up path is connected to the Conv layer before this layer, replacing the original feature extraction process. The top-down path in BiFPN transmits the semantic information of high-level features; the bottom-up path transmits the position information of low-level features; a path in the same layer directly pointing to the output node from the input node can fuse more features; finally, modify the yaml running configuration file according to the above improvements.

[0017] The step 5 specifically includes the following contents:

[0018] Based on the preset evaluation indicators, the prediction results of the improved YOLOv8 distracted driving behavior detection network model are evaluated; the preset evaluation indicators include: parameter quantity, model size, recall rate Recall, precision P (Precision), mean average precision mAP (mean Average Precision) and other 6 aspects to evaluate the performance of the model. The calculation formulas of P, R and mAP are shown in formulas (1) to (3):

[0019]

[0020] Among them, TP represents the positive samples predicted by the model as positive, FP represents the negative samples predicted by the model as positive, and FN represents the positive samples predicted by the model as negative. n is the number of categories detected.

[0021] The step 6 specifically includes the following contents:

[0022] The generalization experiment generally uses public datasets with more images and object categories than the self-built dataset in this paper, such as VOC2007, VOC2012, COCO datasets, etc. The single-stage algorithm YOLOv3-tiny, YOLOv7-tiny, YOLOv8n, SSD and the two-stage algorithm Faster R-CNN are used for comparative experiments. Under the same training environment, the preset evaluation indicators are compared. The various parameter indicators of the improved model have been greatly improved, which proves that the improved model has better generalization performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate test examples consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0024] Figure 1 The present invention is a flowchart of a distracted driving behavior detection method based on a single-stage algorithm according to an embodiment of the present invention.

[0025] Figure 2This is an image of a distracted driving dataset according to an embodiment of the present invention.

[0026] Figure 3 This is an annotated image of the Labelimg dataset of an embodiment of the present invention.

[0027] Figure 4 A schematic diagram of the framework of the improved YOLOv8n network structure according to an embodiment of the present invention.

[0028] Figure 5 Schematic diagram of the BiFPN feature fusion network before and after improvement of an embodiment of the present invention.

[0029] Figure 6 Schematic diagram of the EMA attention module structure of an embodiment of the present invention.

[0030] Figure 7 Schematic diagram of the ODConv dynamic convolution structure of an embodiment of the present invention.

[0031] Figure 8 Schematic diagram of the C2f_ODEMA module structure of an embodiment of the present invention.

[0032] Fig. 9 This is a comparison chart of YOLOv8n tests before and after the improvement of an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments. Each example is provided by way of explanation of the present application and does not limit the present application. In fact, it will be clear to those skilled in the art that modifications and variations may be made in the present application without departing from the scope or spirit of the present application. For example, a feature shown or described as a part of an embodiment may be used in another embodiment to produce yet another embodiment. Therefore, it is desired that the present application includes such modifications and variations within the scope of the appended claims and their equivalents.

[0034] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below through specific implementations and in conjunction with the accompanying drawings.

[0035] like Figure 1 As shown, the present invention is specifically implemented by the following technical solution: a distracted driving behavior detection method based on improved YOLOv8, specifically comprising the following steps:

[0036] Step 1: Create a self-made driver distraction behavior dataset, use data enhancement methods to pre-process the images, and divide them into training sets, validation sets, and test sets in proportion;

[0037] Step 2: Construct an improved YOLOv8 network feature extraction module, and use the EMA attention mechanism before the feature extraction module SPPF;

[0038] Step 3: Build an improved YOLOv8 network feature fusion module, add a small target detection layer to the feature fusion module, build a weighted bidirectional feature pyramid network with a small target detection layer and a new connection structure, and design a C2f_ODEMA module that integrates the EMA attention mechanism and dynamic convolution, conduct an embedding position comparison experiment, and finally select the 26th layer to replace the C2f structure in the small target feature fusion;

[0039] Step 4: Build an improved YOLOv8 network detection head module and add a small target detection head to the detection part;

[0040] Step 5: Input the above training set into the improved YOLOv8 network for training, and evaluate the model improvement according to the preset evaluation index;

[0041] Step 6: Model performance verification: Perform generalization training, ablation experiments, and comparative tests on a public dataset with more images and object categories than the dataset in this article, and evaluate the improved model based on preset evaluation indicators.

[0042] The step 1 specifically includes the following contents:

[0043] Use Labelimg to create a YOLO format distracted driving dataset, which contains 3 categories: cigarettes, water cups, and mobile phones. Considering the data source and data diversity, some data comes from the Kaggle competition State-Farm. These data are mainly side views taken by fixed cameras with high background repetition rate. Multi-background front views are obtained from sources such as Google to promote the comprehensiveness and robustness of model training. The self-made dataset is enhanced by MOSAIC data enhancement, random cropping, scaling, etc., and finally divided into training set, validation set, and test set in a ratio of 7:1:2.

[0044] The steps 2-4 specifically include the following contents:

[0045] Create attention.py in YOLOv8 and add EMA attention definition; add dynamic convolution ODConv definition in block.py, write ODBottlenck definition and replace the two-dimensional convolution in Bottlenck with ODConv2d. Write C2f_ODEMA definition, replace Bottlenck with the previously defined ODBottlenck, add EMA attention mechanism before ODBottlenck and after Split. Finally, register EMA attention mechanism and C2f_ODEMA module in tasks.py.

[0046] Through the improved BiFPN feature fusion network, a small target detection layer P2 is added on the basis of the existing structure. At the same time, the Conv before this layer is connected to the bottom-up path, replacing the original feature extraction process. This improvement has significantly improved the performance of the feature fusion network in small target detection, thereby improving the accuracy and efficiency of detection. The structure before and after the improvement is as follows Figure 5 As shown in the figure, the blue arrows represent the top-down pathways, which transmit the semantic information of high-level features; the yellow arrows represent the bottom-up pathways, which transmit the location information of low-level features; the green part is a pathway from the input node to the output node in the same layer, which can fuse more features. Finally, modify the yaml to improve the model configuration file according to the above improvements.

[0047] The step 5 specifically includes the following contents:

[0048] Based on the preset evaluation indicators, the prediction results of the improved YOLOv8 distracted driving behavior detection network model are evaluated; the preset evaluation indicators include: parameter quantity (Parameters), model size (volume), recall rate R (Recall), precision P (Precision), mean average precision mAP (mean Average Precision) and other 6 aspects to evaluate the performance of the model. The calculation formulas of P, R and mAP are shown in formulas (1) to (3):

[0049]

[0050] Among them, TP represents the positive samples predicted by the model as positive, FP represents the negative samples predicted by the model as positive, and FN represents the positive samples predicted by the model as negative. n is the number of categories detected.

[0051] The step 6 specifically includes the following contents:

[0052] In the same specific example, the influence of C2f_ODEMA at different positions on the improved model through the YOLOv8n network model is compared with YOLOv8n. In the table, the lightweight PBE-YOLOv8n with improved feature fusion network and EMA attention is used as the improved benchmark. Backbone1 indicates that this module is applied in the 6th layer of the improved model, Backbone1 indicates that this module is applied in the 8th layer of the improved model, Neck1 indicates that this module is applied in the 16th layer, Neck2 indicates that this module is applied in the 19th layer, Neck3 indicates that this module is used in the 23rd layer, Neck4 indicates that this module is applied in the 26th layer, Neck5 indicates that this module is applied in the 29th layer, and Neck6 indicates that this module is applied in the 32nd layer. By analyzing the test results in the table, only Neck4 has the best effect in all experiments, and the evaluation indicators all exceed the PBE-YOLOv8n benchmark. Therefore, the present invention determines to apply the C2f_ODEMA module in the 23rd layer of the improved model. See the specific embedding position. Figure 4 Schematic diagram of the improved YOLOv8n network structure.

[0053]

[0054] In the same specific example, the YOLOv8n network model is used to conduct comparative experiments with the mainstream single-stage algorithms YOLOv3-tiny, YOLOv7-tiny, YOLOv8n, SSD and two-stage algorithm Faster R-CNN in the prior art. The SSD algorithm uses VGG as the backbone, and Faster R-CNN uses RestNet50 as the backbone. No pre-trained weights are used during the training process.

[0055]

[0056] From the results in the table, we can see that the improved YOLOv8n has the highest average accuracy, with a model size of 4.7M, which is easy to deploy on the vehicle terminal and can be used for real-time detection on the vehicle terminal. The mAP of the improved YOLOv8n is as high as 91%, and the other parameter indicators are also significantly better than other single-stage and double-stage algorithms.

[0057] The self-built dataset has a large proportion of small objects, so the generalization experiment uses the VisDrone2019 public dataset, which contains 10 object categories, a total of 10,209 images, and 457,066 instance labels. This includes 6,471 training sets, 548 validation sets, and 3,190 test sets (including 1,580 unlabeled images). The experiment uses all training sets and validation sets to participate in the training process, and uses 1,610 labeled test sets for testing. Compared with the self-built dataset in this article, there are more images and object categories. The experimental results are shown in the table. From the analysis, it can be seen that the various parameter indicators of the improved model have been greatly improved, proving that the improved model has good generalization performance.

[0058]

[0059] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A distracted driving behavior detection method based on improved YOLOv8, characterized in that: The specific steps include: Step 1: Create a self-made driver distraction behavior dataset, use data enhancement methods to pre-process the images, and divide them into training sets, validation sets, and test sets in proportion; Step 2: Construct an improved YOLOv8 network feature extraction module, and use the EMA attention mechanism before the feature extraction module SPPF; Step 3: Build an improved YOLOv8 network feature fusion module, add a small target detection layer to the feature fusion module, build a weighted bidirectional feature pyramid network with a small target detection layer and a new connection structure, and design a C2f_ODEMA module that integrates the EMA attention mechanism and dynamic convolution, conduct an embedding position comparison experiment, and finally select the 26th layer to replace the C2f structure in the small target feature fusion; Step 4: Build an improved YOLOv8 network detection head module and add a small target detection head to the detection part; Step 5: Input the above training set into the improved YOLOv8 network for training, and evaluate the model improvement according to the preset evaluation index; Step 6: Perform generalization training, ablation experiments, and comparative tests on a public dataset with more images and object categories than the dataset in this article, and evaluate the improved model based on the preset evaluation indicators.

2. The distracted driving behavior detection method based on improved YOLOv8 according to claim 1, characterized in that: The specific contents of step 1 are as follows: Labelimg was used to create a YOLO format distracted driving dataset, which includes three categories: cigarettes, water cups, and mobile phones. Taking into account the data source and data diversity, some data came from the Kaggle competition State-Farm, which mainly consists of side views taken by fixed cameras with high background repetition rate; multi-background front views were obtained from sources such as Google to promote the comprehensiveness and robustness of model training; the self-made dataset was enhanced by MOSAIC data enhancement, random cropping, scaling, etc., and finally divided into training set, validation set, and test set in a ratio of 7:1:

2.

3. The distracted driving behavior detection method based on improved YOLOv8 according to claim 1, characterized in that: The specific contents of steps 2-4 are as follows: Create attention.py in YOLOv8 and add EMA attention definition; add dynamic convolution ODConv definition in block.py, write ODBottlenck definition to replace the two-dimensional convolution in Bottlenck with ODConv2d; write C2f_ODEMA definition, replace Bottlenck with the previously defined ODBottlenck, add EMA attention mechanism before ODBottlenck and after Split; finally, register EMA attention mechanism and C2f_ODEMA module in tasks.py; Add BiFPN weighted bidirectional pyramid structure and improved BiFPN feature fusion network. A small target detection layer P2 is added on the basis of the existing structure. At the same time, the bottom-up pathway is connected to the Conv layer before this layer, replacing the original feature extraction process. The top-down pathway in BiFPN transmits the semantic information of high-level features; the bottom-up pathway transmits the location information of low-level features; a pathway in the same layer directly points to the output node from the input node, which can fuse more features. Finally, modify the yaml running configuration file according to the above improvements.

4. The distracted driving behavior detection method based on improved YOLOv8 according to claim 1, characterized in that: The specific contents of step 5 are as follows: Based on the preset evaluation index, the prediction results of the improved YOLOv8 distracted driving behavior detection network model are evaluated; The preset evaluation indicators include: parameter quantity, model size, recall rate, precision P (Precision), mean average precision mAP (mean Average Precision) and other six aspects to evaluate the performance of the model; the calculation formulas of P, R and mAP are shown in formulas (1) to (3): Among them, TP represents the positive samples predicted by the model as positive, FP represents the negative samples predicted by the model as positive, FN represents the positive samples predicted by the model as negative, and n is the number of detected categories.

5. The distracted driving behavior detection method based on improved YOLOv8 according to claim 1, characterized in that: The step 6 specifically includes the following contents: Generalization experiments generally use public datasets with more images and object categories than the self-built datasets in this article, such as VOC2007, VOC2012, COCO datasets, etc.; single-stage algorithms YOLOv3-tiny, YOLOv7-tiny, YOLOv8n, SSD and two-stage algorithm Faster R-CNN are used for comparative experiments; under the same training environment, the preset evaluation indicators are compared; the various parameter indicators of the improved model are greatly improved, which proves that the improved model has better generalization performance.

6. The distracted driving behavior detection method based on improved YOLOv8 according to claim 1, characterized in that: Includes the following modules: Use Labelimg to create a YOLO format distracted driving dataset. The self-made dataset is enhanced by MOSAIC data enhancement, random cropping, scaling, etc., and finally divided into training set, validation set and test set according to the proportion. An improved YOLOv8 network is constructed, including a feature extraction module, a feature fusion module and a detection head module; the EMA attention mechanism is used in the feature extraction module; a small target detection layer is added to the feature fusion module, a weighted bidirectional feature pyramid network with a small target detection layer and a new connection structure is constructed, and a C2f_ODEMA module integrating the EMA attention mechanism and dynamic convolution is designed to replace the C2f structure in the small target feature fusion; a small target detection head is added to the detection part; The above training set is input into the improved YOLOv8 network for training, and the model improvement is evaluated according to the preset evaluation indicators; finally, ablation experiments, comparative experiments, and generalization experiments are carried out on the public dataset before and after the improvement of the model.

Images