A driving distraction detection model and method
The driving distraction detection model, which incorporates edge image processing and a multi-scale global attention module, solves the problems of high computational cost and numerous parameters in existing models, achieving real-time and accurate driving distraction detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHANGAN UNIV
- Filing Date
- 2025-04-29
- Publication Date
- 2026-05-12
AI Technical Summary
Existing driver distraction detection models require a large amount of computation and parameters when processed in real time on embedded devices, making it difficult to achieve efficient real-time detection.
We use the Sobel operator to obtain the edge image of the driver's pose, and extract the feature vector through four convolutional layers and a multi-scale global attention module. We then combine residual connections to perform feature fusion and finally classify the data through a fully connected layer.
It reduces computational cost and the risk of overfitting, improves the model's feature mining capabilities and real-time detection accuracy, and is suitable for real-time processing in embedded devices.
Smart Images
Figure CN120689845B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of automotive safe driving technology, and specifically relates to a driving distraction detection model and method. Background Technology
[0002] According to the World Health Organization's (WHO) "State of Road Safety 2023" report, while the number of road traffic fatalities has slightly decreased annually, 1.19 million people still died from road traffic in 2021 alone, and road traffic remains a leading cause of death for children and young adults aged 5-29. Data from various countries on traffic accidents caused by drivers shows that the most common and dangerous driving behaviors are speeding, driving under the influence of alcohol or psychotropic drugs, and distracted driving. Statistics indicate that distracted driving accounts for a large proportion of accidents. The U.S. National Highway Traffic Safety Administration (NHTSA) defines distracted driving as a specific type of inattention where a driver's attention is diverted from driving to another activity. Countless injuries and economic losses are caused by distracted driving each year. It has been observed that these accidents are entirely avoidable if driver abnormalities are detected in time and appropriate reminders are given. Therefore, many cars are now equipped with advanced driver assistance systems (ADAS), such as Lane Departure Warning (LDW) and Forward Collision Warning (FCW). A popular approach currently is to monitor driver behavior using RGB images. This non-contact device prevents human interference with the driver. Furthermore, with the development of computer vision technology, video-based methods are becoming increasingly faster, rapidly becoming a viable solution for autonomous driving. This approach can also be applied to other tasks in Intelligent Transportation Systems (ITS), such as fatigue and driver attention detection, driver skeleton detection, and detection of children remaining in the vehicle.
[0003] With the rapid development of deep learning and computer vision, many researchers have dedicated themselves to studying different types of distracted driving detection (DDD). In recent years, deep learning methods have been widely applied in image classification, detection, and segmentation problems. Compared with traditional algorithms, deep learning offers better performance and accuracy. Extensive research has been conducted on deep convolutional methods, resulting in many high-performance models such as LeNet, AlexNet, VGG16 and VGG19, GooLeNet, and ResNet. While these models have achieved good results, their large size and excessive number of parameters make them unsuitable for real-time processing in embedded systems. Summary of the Invention
[0004] The purpose of this invention is to overcome the defects in the existing technology and provide a driving distraction detection model and method.
[0005] A first aspect of the present invention provides a driving distraction detection model, comprising:
[0006] include:
[0007] Image acquisition module: used to acquire RGB images containing the driver's posture;
[0008] Image processing module: Performs image edge processing on the RGB image based on the Sobel operator to obtain the edge image;
[0009] Feature vector extraction module: includes several feature extraction units, each of which performs a convolution operation on the edge image in sequence to obtain feature vectors;
[0010] Feature vector dimensionality reduction output module: used to reduce the dimensionality of the feature vector and output the prediction result.
[0011] A further approach is that each feature extraction unit performs at least normalization and average pooling on the edge image.
[0012] A further embodiment is that the feature extraction unit includes a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, and a fourth feature extraction unit arranged sequentially.
[0013] The first feature extraction unit, the second feature extraction unit, the third feature extraction unit, and the fourth feature extraction unit each include a convolutional layer, a normalization layer, an activation layer, and an average pooling layer arranged sequentially.
[0014] In the first feature extraction unit, the second feature extraction unit, the third feature extraction unit, and the fourth feature extraction unit, the number of convolution kernels in the convolutional layer increases sequentially; the activation function used in the activation layer is the ReLU function.
[0015] A further proposed solution is that the first feature extraction unit has 32 convolutional kernels and a kernel size of 6×6.
[0016] The second feature extraction unit has 64 convolution kernels and a kernel size of 6×6;
[0017] The third feature extraction unit has 128 convolution kernels and a kernel size of 6×6.
[0018] The fourth feature extraction unit has 128 convolution kernels and a kernel size of 6×6.
[0019] A further embodiment is that the model also includes a multi-scale global attention module, which is connected to the feature vector extraction module;
[0020] The multi-scale global attention module includes several different branch convolutional layers, which obtain convolutional feature vectors and multi-scale depth-separable convolutional feature vectors, and perform feature fusion on the convolutional feature vectors and the multi-scale depth-separable convolutional feature vectors to obtain enhanced feature vectors.
[0021] The feature fusion method involves connecting the convolutional feature vector and the multi-scale depth-separable convolutional feature vector using a method similar to residual connection.
[0022] A further embodiment is that the multi-scale global attention module includes a first branch convolutional layer, a second branch convolutional layer, and a third branch convolutional layer. The first branch convolutional layer is a convolution with a kernel size of 1×1. The second branch convolutional layer consists of a depthwise convolution with a kernel size of 3×3 and a point convolution with a kernel size of 1×1. The third branch convolutional layer consists of a depthwise convolution with a kernel size of 5×5 and a point convolution with a kernel size of 1×1.
[0023] Assuming x1' is the feature vector output by the first branch convolutional layer, x2' is the feature vector output by the first branch convolutional layer, and x3' is the feature vector output by the third branch convolutional layer, then the enhanced feature vector x is:
[0024] .
[0025] A further approach is that the feature vector dimensionality reduction output module includes a pooling layer and a fully connected layer, which are sequentially connected to the multi-scale global attention module to perform adaptive max pooling and fully connected operations on the enhanced feature vector, reducing the feature vector dimension from 256 to n, where n is the number of detection categories to be predicted.
[0026] A second aspect of the present invention provides a driving distraction detection method, which applies the above-described driving distraction detection model and includes the following steps:
[0027] S1. Obtain an RGB image containing the driver's posture;
[0028] S2. Process the RGB image using the Sobel operator to obtain the edge image;
[0029] S3. Input the edge images into the feature extraction unit in sequence to obtain feature vectors;
[0030] S4. After performing adaptive max pooling and fully connected operations on the acquired feature information, output the type of distracted driving.
[0031] A further solution is that step S3, obtaining the feature vector, specifically includes:
[0032] The edge image is convolved once using 32 kernels (6×6 kernel size), ReLU activation function, and BatchNorm and average pooling. Then, it is convolved once more using 64 kernels (6×6 kernel size), ReLU activation function, BatchNorm and average pooling. Next, it is convolved once more using 128 kernels (6×6 kernel size), ReLU activation function, BatchNorm and average pooling. Finally, it is convolved once more using 256 kernels (6×6 kernel size), ReLU activation function, BatchNorm and average pooling, to obtain the feature vector.
[0033] A further approach involves feeding the feature vectors obtained in step S3 into a multi-scale global attention module, fusing the feature vectors extracted from the first branch and the feature vectors obtained from the latter two branches through element-wise multiplication and addition, and using the captured driver's hand and body movement features to classify distracted driving behavior.
[0034] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0035] This invention transforms RGB images containing driver poses into edge images, resulting in less background noise and preserving only the task-related driver pose, thus providing strong support for improving image training and inference speed. Using convolutional layers with a kernel size of 6×6 ensures a larger receptive field to capture global information while reducing computational cost and the risk of overfitting.
[0036] This invention adds a multi-scale global attention module after four convolutional layers, which contains two depthwise separable convolutional blocks of different scales. This improves the feature mining capability of the model without increasing the number of parameters. In order to make the model more expressive, the feature vectors are combined in a residual manner to make the model pay more attention to the information related to the driver's actions in the global information. Attached Figure Description
[0037] The following figures are for illustrative purposes only and are not intended to limit the scope of the invention.
[0038] Figure 1 : Overall framework diagram of the detection model of this invention;
[0039] Figure 2 Edge map, histogram of oriented gradients, local binary data Figure 3 Visual comparison of various images;
[0040] Figure 3: Schematic diagram of depthwise convolution and pointwise convolution. Detailed Implementation
[0041] To make the objectives, technical solutions, design methods, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention.
[0042] This invention utilizes a multi-scale, depthwise separable convolutional attention neural network to process edge images. The edge image acquisition method uses the Sobel operator to extract features from RGB images. Edge images retain essential feature information for model classification, such as human posture and actions. By converting images into edge images, unnecessary background information can be filtered out, allowing the network to focus on recognizing different driving actions. For example... Figure 1 As shown, the detection model has four convolutional layers, including convolution, batch normalization, ReLU, and pooling. Batch normalization transforms the data into a normal distribution, helping the neural network learn feature information better and improving its performance and stability. Following this is the normalization operation, including the ReLU activation function and 2×2 average pooling. Average pooling retains more background information; since this invention uses edge images with most background information removed, average pooling is more suitable and yields better results. The four convolutional layers used in this invention all have a 6×6 kernel size. The first layer has 32 filters, the second has 64, the third has 128, and the fourth has 256. Because no data augmentation is performed on the dataset, overfitting is avoided, and therefore, a random dropout layer is not used. After four convolutional layers, the resulting feature maps are fed into a multi-scale global attention module, which includes multi-scale depthwise convolutions (with kernel sizes of 3×3 and 5×5) and pointwise convolutions. The feature vectors are then connected in a residual-like manner, and finally classified through a global average pooling layer and a fully connected layer.
[0043] The entire process of proposing the model in this invention can be described as follows:
[0044]
[0045] in, This represents the feature vector extracted by the i-th feature extraction unit. This refers to the input edge image. This represents a convolution kernel with a size of 6×6. This represents a convolution kernel with a size of 1×1. Representative batch normalization operation, Represents the ReLU activation function operation. Represents average pooling operation. This represents point convolution. and These represent depthwise convolutions with kernel sizes of 3×3 and 5×5, respectively.
[0046]
[0047] and These represent element-wise multiplication and addition operations, respectively. This invention uses a method similar to residual connections to connect the feature information extracted by multi-scale depth-separable convolution with the features extracted by convolution.
[0048]
[0049] in, This represents an adaptive max-pooling operation. Represents a fully connected operation. The label represents the output.
[0050] Example 1
[0051] like Figure 1 As shown, the present invention provides a driving distraction detection model, including:
[0052] Image acquisition module: used to acquire RGB images containing the driver's posture;
[0053] Image processing module: Performs image edge processing on the RGB image based on the Sobel operator to obtain the edge image, such as... Figure 2 As shown, edge maps, histograms of oriented gradients, and local binary data are used to analyze these features. Figure 3 Visual comparison of the images reveals that the edge image contains the least information, retaining only the basic outline of the driver; the HOG image has slightly more detail, but also some background noise; the LBP image has even more detail, retaining almost all features of the original image except for color. Corresponding to these details are the image sizes: the edge image is the smallest, followed by the HOG image, and the LBP image is the largest. Therefore, in this embodiment, the Sobel operator is used to process the RGB images for edge processing, resulting in edge images with less background noise that retain only the driver's pose relevant to the task, providing strong support for improving the training and inference speed of the images.
[0054] The feature vector extraction module includes four feature extraction units. Each feature extraction unit sequentially performs a convolution operation on the edge image to obtain feature vectors. Specifically, the four feature extraction units are: a convolution with 32 kernels (6×6 kernel size, ReLU activation function, and including BatchNorm and average pooling); a convolution with 64 kernels (6×6 kernel size, ReLU activation function, and including BatchNorm and average pooling); a convolution with 128 kernels (6×6 kernel size, ReLU activation function, and including BatchNorm and average pooling); and a convolution with 256 kernels (6×6 kernel size, ReLU activation function, and including BatchNorm and average pooling).
[0055] Multi-scale global attention modules, such as Figure 2 As shown, the system includes multi-scale depthwise convolutions (with kernel sizes of 3×3 and 5×5) and pointwise convolutions, connected to the feature vector extraction module. Specifically, the multi-scale global attention module comprises three branches: the first branch is a convolution with a kernel size of 1×1; the second branch consists of a depthwise convolution with a kernel size of 3×3 and a pointwise convolution with a kernel size of 1×1; and the third branch consists of a depthwise convolution with a kernel size of 5×5 and a pointwise convolution with a kernel size of 1×1. The feature vectors obtained by the feature vector extraction module are input into the multi-scale global attention module, and the three branches respectively obtain the feature vectors and fuse them in the form of element-wise multiplication and addition in the form of residuals.
[0056] Feature vector dimensionality reduction output module: Used to reduce the dimensionality of the feature vector and output the prediction result. In this embodiment, the feature vector dimensionality reduction output module includes a pooling layer and a fully connected layer. The pooling layer and the fully connected layer are sequentially connected to the multi-scale global attention module. They are used to perform adaptive max pooling and fully connected operations on the fused feature vector, reducing the feature vector dimension from 256 to 10, and outputting 10 predicted detection categories, namely normal driving, making a phone call with the right hand, sending a message with the right hand, making a phone call with the left hand, sending a message with the left hand, adjusting the radio, drinking water, reaching behind, applying makeup or touching hair, and chatting with passengers.
[0057] Example 2
[0058] This embodiment provides a method for detecting driver distraction, including the following steps:
[0059] (1) First, the RGB image is processed by the Sobel operator to obtain the edge image, and then the edge image is input into four feature extraction modules in sequence to obtain the feature vector;
[0060] (2) The acquired feature vectors are fed into the multi-scale global attention module to enhance the model’s ability to capture long-distance dependencies and local information, so that the model can better acquire the feature information of the image for classification.
[0061] (3) After the acquired feature information is processed by adaptive max pooling and full connection, the types of distracted driving are output. There are 10 types in total, namely normal driving, making a phone call with the right hand, sending a message with the right hand, making a phone call with the left hand, sending a message with the left hand, adjusting the radio, drinking water, reaching the back of the hand, applying makeup or touching the hair and chatting with passengers.
[0062] Furthermore, the specific steps for inputting the edge image into the feature extraction module to obtain the feature vector include:
[0063] Step (1-1): Process the RGB image using the Sobel operator to obtain the edge image. Then, perform one convolution operation using 32 convolution kernels (6×6 kernel size), ReLU activation function, and BatchNorm and average pooling. Next, perform one convolution operation using 64 convolution kernels (6×6 kernel size), ReLU activation function, BatchNorm and average pooling. Then, perform one convolution operation using 128 convolution kernels (6×6 kernel size), ReLU activation function, BatchNorm and average pooling. Finally, perform one convolution operation using 256 convolution kernels (6×6 kernel size), ReLU activation function, BatchNorm and average pooling to obtain the feature vector. .
[0064] Step (1-2): The feature vector obtained in step (1-1) The data is fed into a multi-scale global attention module, which has three branches. The first branch is a 1×1 convolution, the second branch consists of a 3×3 depthwise convolution and a 1×1 pointwise convolution, and the third branch consists of a 5×5 depthwise convolution and a 1×1 pointwise convolution. The feature vectors extracted from the first branch and the feature vectors obtained from the latter two branches are then fused using element-wise multiplication and addition as residuals. The captured driver hand and body movement features are then used to classify distracted driving behavior.
[0065] Furthermore, the method for feature dimensionality reduction and classification of the feature vectors extracted in step (1-2) is to first perform adaptive max pooling on the extracted feature vectors, and then perform a fully connected operation to reduce the feature vector dimension from 256 to 10, and output the predicted 10 categories.
[0066] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.
Claims
1. A driver distraction detection system, characterized in that, include: Image acquisition module: used to acquire RGB images containing the driver's posture; Image processing module: Performs image edge processing on the RGB image based on the Sobel operator to obtain the edge image; Feature vector extraction module: includes several feature extraction units, each of which sequentially performs a convolution operation on the edge image to obtain feature vectors; Feature vector dimensionality reduction output module: used to reduce the dimensionality of the feature vector and output the prediction result; A multi-scale feature enhancement module, which is connected to the feature vector extraction module; The multi-scale feature enhancement module includes several different branch convolutional layers, which obtain convolutional feature vectors and multi-scale depth-separable convolutional feature vectors, and perform feature fusion on the convolutional feature vectors and the multi-scale depth-separable convolutional feature vectors to obtain enhanced feature vectors. The feature fusion is achieved by using a residual connection method to connect the convolutional feature vector and the multi-scale depthwise separable convolutional feature vector; The multi-scale feature enhancement module includes a first branch convolutional layer, a second branch convolutional layer, and a third branch convolutional layer. The first branch convolutional layer is a convolution with a kernel size of 1×1. The second branch convolutional layer consists of a depthwise convolution with a kernel size of 3×3 and a point convolution with a kernel size of 1×1. The third branch convolutional layer consists of a depthwise convolution with a kernel size of 5×5 and a point convolution with a kernel size of 1×1. Assuming x1' is the feature vector output by the first branch convolutional layer, x2' is the feature vector output by the second branch convolutional layer, and x3' is the feature vector output by the third branch convolutional layer, then the enhanced feature vector x is: ； in, and These represent element-wise multiplication and addition operations, respectively.
2. The driving distraction detection system according to claim 1, characterized in that, Each of the feature extraction units performs at least normalization and average pooling on the edge image.
3. The driving distraction detection system according to claim 1, characterized in that, The feature extraction unit includes a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, and a fourth feature extraction unit arranged sequentially. The first feature extraction unit, the second feature extraction unit, the third feature extraction unit, and the fourth feature extraction unit each include a convolutional layer, a normalization layer, an activation layer, and an average pooling layer arranged sequentially. In the first feature extraction unit, the second feature extraction unit, the third feature extraction unit, and the fourth feature extraction unit, the number of convolution kernels in the convolutional layer increases sequentially; the activation function used in the activation layer is the ReLU function.
4. The driving distraction detection system according to claim 3, characterized in that, The first feature extraction unit has 32 convolution kernels and a kernel size of 6×6; The second feature extraction unit has 64 convolution kernels and a kernel size of 6×6. The third feature extraction unit has 128 convolution kernels and a kernel size of 6×6. The fourth feature extraction unit has 256 convolution kernels and a kernel size of 6×6.
5. A driver distraction detection system according to claim 4, characterized in that, The feature vector dimensionality reduction output module includes a pooling layer and a fully connected layer, which are sequentially connected to the multi-scale feature enhancement module. The pooling layer and the fully connected layer are used to perform adaptive max pooling and fully connected operations on the enhanced feature vector to reduce the feature vector dimension from 256 to n, where n is the number of detection categories to be predicted.
6. A method for detecting driver distraction, characterized in that, The driving distraction detection system according to any one of claims 1-5 includes the following steps: S1. Obtain an RGB image containing the driver's posture; S2. Process the RGB image using the Sobel operator to obtain the edge image; S3. Input the edge images into the feature extraction unit in sequence to obtain feature vectors; S4. After performing adaptive max pooling and fully connected operations on the acquired feature information, output the type of distracted driving.
7. The driving distraction detection method according to claim 6, characterized in that, The specific steps of obtaining the feature vector in step S3 include: The edge image is convolved once using 32 kernels (6×6 kernel size), ReLU activation function, and BatchNorm and average pooling. Then, it is convolved once more using 64 kernels (6×6 kernel size), ReLU activation function, BatchNorm and average pooling. Next, it is convolved once more using 128 kernels (6×6 kernel size), ReLU activation function, BatchNorm and average pooling. Finally, it is convolved once more using 256 kernels (6×6 kernel size), ReLU activation function, BatchNorm and average pooling, to obtain the feature vector.
8. The driving distraction detection method according to claim 7, characterized in that, The feature vector obtained in step S3 is sent to the multi-scale feature enhancement module. The feature vector extracted from the first branch is fused with the feature vectors obtained from the last two branches. The fusion method is as follows: the feature vectors obtained from the last two branches are multiplied element by element to obtain an intermediate feature vector. Then, the intermediate feature vector is added element by element to the feature vector extracted from the first branch. The captured hand and body movement features of the driver are used to classify distracted driving behavior.