A driver distraction behavior detection method and system based on improved YOLOv8 model

Abstract

The present invention discloses a driver distraction behavior detection method and system based on an improved YOLOv8 model, belonging to the field of information processing technology. First, based on the State Farm Distracted Driver Detection dataset, normal samples are eliminated to construct a driver distraction behavior dataset, and the driver distraction behavior dataset is preprocessed using Mosaic data enhancement technology. After manual annotation, the driver distraction behavior dataset is divided into a training set and a validation set. Secondly, an improved YOLOv8 model framework is constructed. The CSPLayer_2Conv module in Stage Layer 1, Stage Layer 2, Stage Layer 3, and Stage Layer 4 in the original YOLOv8 network backbone is replaced with a LarK module, a CAFM module is added to the neck part, and the original detection head Decoupled Head is replaced with an RT-DETR Decoder in the head part. Then, the improved YOLOv8 model is trained using the training set, and the model performance is tested using the Driver Distraction Dataset as a test set. Finally, the detection results are evaluated using evaluation indicators. The method and system provided by the present invention effectively resolve the dilemma of balancing accuracy and real-time performance of existing detection methods and reduce the risk of traffic accidents.

Description

Technical Field

[0001] The present invention belongs to the field of information processing technology, and in particular relates to a driver distraction behavior detection method and system based on an improved YOLOv8 model. Background Art

[0002] Currently, computer vision-based driver distraction detection has become a key research area in intelligent transportation systems. However, in reality, methods for detecting driver distraction lack a good balance between accuracy and real-time performance. Vehicle status detection methods are susceptible to interference, physiological signal detection methods are equipment-intensive, and visual detection methods have limitations with depth cameras. While conventional cameras are useful, existing models lack feature extraction and fusion, resulting in low detection accuracy in complex scenarios. Therefore, developing accurate, efficient, and well-balanced detection strategies is of great significance and practical value.

[0003] Existing distraction detection methods primarily rely on local feature extraction, but this approach lacks global feature information capture, potentially leading to unstable performance in complex scenarios. Furthermore, effectively integrating the local feature extraction capabilities of convolutional neural networks with the global feature capture advantages of Transformer models while ensuring the overall real-time and accuracy of the system remains an urgent challenge.

[0004] In response to the above situation, the present invention proposes a driver distraction behavior detection method and system based on the YOLOv8 model, which can effectively improve the existing technology and overcome its shortcomings. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and system for detecting driver distraction behavior based on an improved YOLOv8 model, so as to solve the problems of poor accuracy and real-time performance, over-reliance on complex models and high consumption of computing resources in the existing technology.

[0006] To achieve the above objectives, the present invention provides a method for detecting driver distraction based on an improved YOLOv8 model, comprising the following steps:

[0007] Step 1: Based on the State Farm Distracted Driver Detection dataset, normal samples were removed to construct a distracted driving dataset. The distracted driving dataset was preprocessed using Mosaic data augmentation technology and divided into training and validation sets after manual annotation.

[0008] Step 2: Build the improved YOLOv8 model framework; replace the CSPLayer_2Conv module in Stage Layer 1, Stage Layer 2, Stage Layer 3, and Stage Layer 4 in the original YOLOv8 network Backbone with the LarK module, add the CAFM module to the Neck part, and replace the original detection head DecoupledHead with the RT-DETR Decoder in the Head part;

[0009] Step 3: Use the training set to train the improved YOLOv8 model, and use some images in the Driver Distraction Dataset as the test set to test the model performance;

[0010] Step 4: Use accuracy, precision, recall, F1 value, number of parameters and GFLOPs as evaluation indicators to evaluate the performance of the detection results of step 3.

[0011] Preferably, the LarK module consists of a depthwise separable convolution (DW), a SE Block, and a feedforward network (FFN) module with a GRN unit. Feature extraction is performed through the LarK module. The specific process is as follows: BN is selected and incorporated into the convolution layer; in the DW conv part, based on pixel correlation, an architecture combining a convolution layer parallel to a large kernel and a dilated convolution module is adopted to capture high-quality feature maps, which are converted into equivalent kernel Dilated Re-param Blocks, and default hyperparameter values are given. The default hyperparameter values are equivalent kernel size K = 13, parallel convolution layer size k = (5, 7, 3, 3, 3), and dilation rate r = (1, 2, 3, 4, 5) of the parallel convolution layer.

[0012] Preferably, the CAFM module is used to extract local and global features to capture a wider range of data information. The specific process is as follows:

[0013] The local branch of CAFM is used to extract local features and achieve comprehensive denoising. The calculation expression is as follows:

[0014] F conv =W 3×3×3 (CS(W 1×1 (Y)));

[0015] Among them, F conv is the output of the local branch, W 1×1 represents the convolution kernel of size 1×1, W 3×3×3 represents a convolution kernel of size 3×3×3, CS represents the channel shuffle operation, and Y is the input feature map;

[0016] The self-attention mechanism is used for the global branch of CAFM to capture a wider range of data information; query (Q), key (K) and value (V) are generated through 1×1 convolution and 3×3 depth convolution to obtain three tensors of shape ^H×^W×^C, and then Q is reshaped into Reshape K into Finally, the attention map is calculated by K and Q The calculation expression of the global branch is as follows:

[0017]

[0018] Here, α is a learnable scaling parameter that controls the and The matrix product size, F att Represents the output of the global branch;

[0019] Get the output result F of the CAFM module out , the expression is as follows:

[0020] F out =F att +F conv .

[0021] Preferably, the RT-DETR Decoder adopts a multi-level iterative optimization mechanism to improve the expressiveness of target features and ultimately output the target category information and bounding box position. The specific contents are as follows: The core of RT-DETR lies in the TransformerDecoder module, which iteratively optimizes the target through a multi-layer structure, generates target categories and bounding boxes, and improves detection performance with the help of auxiliary prediction heads; each layer of the Decoder consists of a self-attention mechanism, a cross-attention mechanism, and a feedforward network module, among which the self-attention mechanism allows target queries to exchange information with each other and simulate the global relationship between them; the cross-attention mechanism enables the target query to interact with the output characteristics of the encoder, extract relevant information from multi-scale image features and integrate global context; the feedforward network module further processes the query embedding through nonlinear transformation, thereby enhancing its expressiveness.

[0022] Preferably, the calculation expressions for accuracy, precision, recall and F1 value in step 4 are as follows:

[0023]

[0024] Where Accuracy represents the accuracy, Precision represents the precision, Recall represents the recall, FP represents the number of negative class predictions as positive class, TP represents the number of positive class predictions as positive class, TN represents the number of negative class predictions as negative class, and FN represents the number of positive class predictions as negative class.

[0025] The present invention also provides a system for detecting driver distraction behavior based on an improved YOLOv8 model, comprising a data set construction module, a feature extraction module, a model optimization module, a behavior recognition module, and a performance improvement module.

[0026] Dataset Construction Module: This module creates and preprocesses a driving behavior dataset containing various distracting behaviors (such as texting and phone calls). It then divides the dataset into training and validation sets to meet the needs of model training and evaluation. It also uses a subset of images from the DriverDistractionDataset as a test set to verify model accuracy and generalization.

[0027] Feature extraction module: This module uses an improved YOLOv8 network to collect and extract the driver's status information and behavioral characteristics in real time. Information collection is achieved using on-board equipment and cameras.

[0028] Model Optimization Module: This module introduces the large-kernel convolution Lark (LarK), the convolutional attention hybrid CAFM (Convolutional Attention Flow Mechanism), and the RT-DETR Decoder module into the YOLOv8 network. By optimizing the network structure, the model adaptively adjusts the feature extraction strategy to achieve efficient recognition of driver distraction.

[0029] Behavior Recognition Module: This module is used to adjust and output the recognition results of driver distraction behaviors in real time based on the model optimization results, ensuring the accuracy and real-time performance of the system in identifying driver distraction behaviors.

[0030] Performance Enhancement Module: This module is used to globally optimize the preliminary recognition results based on the large kernel convolution Lark (LarK), the convolutional attention hybrid CAFM (Convolutional Attention Flow Mechanism), and the RT-DETRDecoder module. The optimization process involves evaluating the fitness of the recognition strategy to generate the optimal driver distraction behavior detection strategy.

[0031] Preferably, the behavior recognition module extracts image features based on LarK through an improved YOLOv8 model, calculates the output results of local branches and global branches, and uses RT-DETR Dec oder to accurately identify driver distraction behaviors.

[0032] Therefore, the present invention adopts the above-mentioned driver distraction behavior detection method and system based on the improved YOLOv8 model, and performs identification after data fusion, feature extraction and preprocessing on the collected driving behavior data according to a preset processing flow to perform distraction behavior detection; this method overcomes the shortcomings of existing detection methods in accuracy and real-time performance; at the same time, compared with traditional detection methods, it effectively improves detection accuracy and establishes a distraction behavior recognition model based on the improved YOLOv8 model, effectively overcoming the problems of reliance on complex models and high computing resource consumption in the existing technology, which is crucial for improving road traffic safety and preventing distracted driving accidents, and also provides more safety protection for drivers.

[0033] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a flowchart of a driver distraction behavior detection method based on an improved YOLOv8 model of the present invention;

[0035] Figure 2 This is a structural block diagram of a system used in a method for detecting driver distraction based on an improved YOLOv8 model of the present invention;

[0036] Figure 3 Schematic diagram of the structure of the improved YOLOv8 according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort shall fall within the scope of protection of the present invention.

[0038] See also Figure 1 , a driver distraction behavior detection method based on an improved YOLOv8 model, comprising the following steps:

[0039] Step 1: Based on the State Farm Distracted Driver Detection dataset, normal samples were removed to construct a distracted driving dataset. The dataset was preprocessed using Mosaic data augmentation technology and manually labeled to divide the dataset into training and validation sets. More specifically, the original dataset contains 17,462 training images and 4,961 test images in 10 categories. These images come from 26 subjects of different body shapes and skin colors. Normal driving samples were removed, and 10,760 representative images were selected as the training dataset. The dataset was further divided into a training set (7,531 images) and a validation set (3,229 images) in a 7:3 ratio.

[0040] Step 2: Build an improved YOLOv8 model framework; replace the CSPLayer_2Conv module in Stage Layer 1, Stage Layer 2, Stage Layer 3, and Stage Layer 4 in the original YOLOv8 network Backbone with the LarK module, use the LarK module and the dilated convolution mechanism to improve feature extraction, promote inter-channel communication and spatial aggregation through SE Block, and increase feature depth; add the CAFM module to the Neck part, extract local features and capture broader data information through local branches and global branches respectively; and replace the original detection head DecoupledHead with RT-DETR Decoder in the Head part. Iteratively optimize the target through a multi-layer structure, generate target categories and bounding boxes, and use the auxiliary prediction head to improve detection performance;

[0041] Among them, the LarK module consists of a depthwise separable convolution (DW), SE Block, and a feedforward network (FFN) module with a GRN unit. Feature extraction is performed through the LarK module. The specific process is as follows: BN is selected and incorporated into the convolution layer; in the DW conv part, based on pixel correlation, an architecture combining a convolution layer parallel to a large kernel and a dilated convolution module is adopted to capture high-quality feature maps, which are converted into equivalent kernel Dilated Re-param Blocks and given default hyperparameter values. Among them, the default hyperparameter values are equivalent kernel size K = 13, parallel convolution layer size k = (5, 7, 3, 3, 3) and dilation rate r = (1, 2, 3, 4, 5) of the parallel convolution layer.

[0042] The CAFM module extracts local and global features to capture a wider range of data information. The specific process is as follows:

[0043] The local branch of CAFM is used to extract local features and achieve comprehensive denoising. The calculation expression is as follows:

[0044] Fconv =W 3×3×3 (CS(W 1×1 (Y)));

[0045] Among them, F conv is the output of the local branch, W 1×1 represents the convolution kernel of size 1×1, W 3×3×3 represents a convolution kernel of size 3×3×3, CS represents the channel shuffle operation, and Y is the input feature map;

[0046] The self-attention mechanism is used for the global branch of CAFM to capture a wider range of data information; query (Q), key (K) and value (V) are generated through 1×1 convolution and 3×3 depth convolution, and three shapes are obtained. ^ H× ^ W× ^ C, and then reshape Q into Reshape K into Finally, the attention map is calculated by K and Q The calculation expression of the global branch is as follows:

[0047]

[0048] Here, α is a learnable scaling parameter that controls the and The matrix product size, F att Represents the output of the global branch;

[0049] Get the output result F of the CAFM module out , the expression is as follows:

[0050] F out =F att +F conv .

[0051] The RT-DETR Decoder adopts a multi-level iterative optimization mechanism to continuously improve the expressive ability of target features, and ultimately outputs the target category information and bounding box position. At the same time, by introducing an auxiliary prediction head, the detection effect and accuracy are further enhanced; the specific contents are as follows: The core of RT-DETR lies in the Transformer Decoder module, which iteratively optimizes the target through a multi-layer structure, generates target categories and bounding boxes, and uses auxiliary prediction heads to improve detection performance; each layer of the Decoder consists of a self-attention mechanism, a cross-attention mechanism, and a feedforward network module. Among them, the self-attention mechanism allows target queries to exchange information with each other and simulate the global relationship between them; the cross-attention mechanism enables the target query to interact with the output characteristics of the encoder, extract relevant information from multi-scale image features and integrate global context; the feedforward network module further processes the query embedding through nonlinear transformations, thereby enhancing its expressiveness.

[0052] Step 3: The improved YOLOv8 model was trained using the training set, and the model performance was tested using a subset of images from the Driver Distraction Dataset as a test set. More specifically, the algorithm was implemented on a Windows 10 operating system, an Intel(R) Core(TM) i9-10900K CPU, and an Nvidia GeForce RTX 3090 24G GPU, using the PyTorch 1.10 deep learning framework and a CUDA 11.1 environment. During network training, the input images were scaled to a uniform size (640 pixels × 640 pixels) and normalized. SGD was used as the network optimizer, with a learning rate of 0.01, a momentum factor of 0.937, a weight decay factor of 0.0005, and a batch size of 32, for 50 training rounds. Finally, the driver distraction behavior detection method achieved a precision and recall of 95.85% and 94.12%, respectively, on the test set.

[0053] Step 4: Use accuracy, precision, recall, F1 value, number of parameters, and GFLOPs as evaluation indicators to evaluate the performance of the detection results of step 3. The calculation expressions of accuracy, precision, recall, and F1 value are as follows:

[0054]

[0055] Where Accuracy represents the accuracy, Precision represents the precision, Recall represents the recall, FP represents the number of negative class predictions as positive class, TP represents the number of positive class predictions as positive class, TN represents the number of negative class predictions as negative class, and FN represents the number of positive class predictions as negative class.

[0056] See also Figure 2 A system for detecting driver distracted behavior based on an improved YOLOv8 model includes a dataset construction module, a feature extraction module, a model optimization module, a behavior recognition module, and a performance improvement module.

[0057] Dataset Construction Module: This module creates and preprocesses a driving behavior dataset containing various distracting behaviors (such as texting and phone calls). It then divides the dataset into training and validation sets to meet the needs of model training and evaluation. It also uses a subset of images from the DriverDistractionDataset as a test set to verify model accuracy and generalization.

[0058] Feature extraction module: This module uses an improved YOLOv8 network to collect and extract the driver's status information and behavioral characteristics in real time. Information collection is achieved using on-board equipment and cameras.

[0059] Model Optimization Module: This module introduces the large-kernel convolution Lark (LarK), the convolutional attention hybrid CAFM (Convolutional Attention Flow Mechanism), and the RT-DETR Decoder module into the YOLOv8 network. By optimizing the network structure, the model adaptively adjusts the feature extraction strategy to achieve efficient recognition of driver distraction.

[0060] Behavior Recognition Module: This module adjusts and outputs the driver distraction identification results in real time based on the model optimization results, ensuring the system's accuracy and real-time performance in identifying driver distraction behaviors. Specifically, the behavior recognition module uses an improved YOLOv8 model and LarK to extract image features. Based on the output results of local and global branch calculations, it uses the RT-DETR Decoder to accurately identify driver distraction behaviors.

[0061] Performance Enhancement Module: This module is used to globally optimize the preliminary recognition results based on the large kernel convolution Lark (LarK), the convolutional attention hybrid CAFM (Convolutional Attention Flow Mechanism), and the RT-DETRDecoder module. The optimization process involves evaluating the fitness of the recognition strategy to generate the optimal driver distraction behavior detection strategy.

[0062] Therefore, the present invention adopts the above-mentioned driver distraction behavior detection method and system based on the improved YOLOv8 model, effectively resolving the dilemma of existing detection methods in balancing accuracy and real-time performance, breaking through the limitations of traditional convolutional neural networks in global feature extraction and the bottlenecks of Transformer models in resource usage and inference time. The established distraction behavior recognition model based on the improved YOLOv8 model is able to keenly capture the subtle characteristics of driver distraction behavior in complex driving scenarios, significantly improving detection accuracy while ensuring computational efficiency, laying a solid foundation for the driving safety warning mechanism, and effectively reducing the risk of traffic accidents.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A driver distraction behavior detection method based on an improved YOLOv8 model, characterized in that: The following steps are involved: Step 1: Based on the State Farm Distracted Driver Detection dataset, normal samples were removed to construct a distracted driving dataset. The distracted driving dataset was preprocessed using Mosaic data augmentation technology and divided into training and validation sets after manual annotation. Step 2: Build the improved YOLOv8 model framework; replace the CSPLayer_2Conv module in Stage Layer 1, Stage Layer 2, Stage Layer 3, and Stage Layer 4 in the original YOLOv8 network Backbone with the LarK module, add the CAFM module to the Neck part, and replace the original detection head Decoupled Head with RT-DETR Decoder in the Head part; Step 3: Use the training set to train the improved YOLOv8 model, and use the Driver Distraction Dataset as the test set to test the model performance; Step 4: Use accuracy, precision, recall, F1 value, number of parameters, and GFLOPs as evaluation indicators to evaluate the performance of the detection results in step 3; The LarK module consists of a depthwise separable convolution, a SE Block, and a feedforward network module with a GRN unit. Feature extraction is performed through the LarK module. The specific process is as follows: BN is selected and incorporated into the convolution layer. In the DW conv part, based on pixel correlation, an architecture combining convolution layers in parallel with large kernels and dilated convolution modules is adopted to capture high-quality feature maps and convert them into equivalent kernel Dilated Re-param Blocks. Default hyperparameter values are given, among which the default hyperparameter values are equivalent kernel size K=13, parallel convolution layer sizes k=(5,7,3,3,3), and dilation rates r=(1,2,3,4,5). The CAFM module is used to extract local and global features. The specific process is as follows: The local features are extracted using the local branch of CAFM; the calculation expression is as follows: ； in, is the output of the local branch, Indicates the size The convolution kernel, Indicates the size The convolution kernel, CS represents the channel shuffle operation, and Y is the input feature map; The self-attention mechanism is adopted for the global branch of CAFM; Convolution and The depth convolution generates Q, K and V, resulting in three shapes: Then reshape Q into , reshape K into ; Finally, the attention map is calculated by K and Q ; The calculation expression of the global branch is as follows: ； ； in, is a learnable scaling parameter that controls the and The size of the matrix product of Represents the output of the global branch; Get the output of the CAFM module , the expression is as follows: 。 2. The method for detecting driver distraction based on the improved YOLOv8 model according to claim 1, characterized in that: The RT-DETR Decoder uses a multi-level iterative optimization mechanism to improve the expressiveness of target features and ultimately outputs the target category information and bounding box position. The specific contents are as follows: The core of RT-DETR lies in the Transformer Decoder module, which iteratively optimizes the target through a multi-layer structure to generate the target category and bounding box; each layer of the Decoder consists of a self-attention mechanism, a cross-attention mechanism, and a feedforward network module. Among them, the self-attention mechanism allows target queries to exchange information with each other and simulate the global relationship between them; the cross-attention mechanism enables the target query to interact with the output characteristics of the encoder, extracting relevant information from multi-scale image features and integrating global context; the feedforward network module further processes the query embedding through nonlinear transformations.

3. The method for detecting driver distraction based on the improved YOLOv8 model according to claim 2, characterized in that: The calculation expressions for accuracy, precision, recall, and F1 value in step 4 are as follows: ； ； ； ； Where, Indicates the accuracy, represents the accuracy, represents the recall rate, Indicates the number of negative class predictions as positive class, Indicates that the positive class prediction is the positive class number, Indicates that the negative class prediction is the number of negative classes, Indicates the number of positive class predictions that are negative.

4. A system for use in the driver distraction behavior detection method based on the improved YOLOv8 model according to any one of claims 1 to 3, characterized in that: It includes dataset construction module, feature extraction module, model optimization module, behavior recognition module and performance improvement module; Dataset Construction Module: This module is used to create and preprocess a driving behavior dataset containing various distracting behaviors, divide the dataset into training and validation sets, and introduce the Driver Distraction Dataset as a test set to verify model accuracy and generalization capabilities. Feature extraction module: This module uses an improved YOLOv8 network to collect and extract the driver's status information and behavioral characteristics in real time. Information collection is achieved using on-board equipment and cameras. Model Optimization Module: This module introduces the large-kernel convolution Lark (LarK), the convolutional attention hybrid CAFM (Convolutional Attention Flow Mechanism), and the RT-DETR Decoder module into the YOLOv8 network. By optimizing the network structure, the model adaptively adjusts the feature extraction strategy to achieve efficient recognition of driver distraction. Behavior recognition module: used to adjust and output the recognition results of driver distraction behavior in real time based on model optimization results; Performance improvement module: used to globally optimize the preliminary recognition results based on the large kernel convolution Lark, convolutional attention hybrid CAFM mechanism and RT-DETRDecoder module.

5. The system for detecting driver distraction based on the improved YOLOv8 model according to claim 4, characterized in that: The behavior recognition module uses an improved YOLOv8 model to extract image features based on LarK, calculates the output results of local branches and global branches, and uses the RT-DETR Decoder to accurately identify driver distraction behaviors.

Images