Driving distraction detection model and method

Abstract

The invention belongs to the technical field of safe driving of automobiles, and particularly discloses a driving distraction detection model, which comprises an image acquisition module, a data processing module and a data processing module, the image processing module is used for processing the RGB image based on a Sobel operator to obtain an edge image; the feature vector extraction module is used for performing convolution operation on the edge image to obtain a feature vector; the feature vector dimensionality reduction output module is used for reducing the dimensionality of the feature vector and outputting a prediction result; based on the detection model, the invention further discloses a driving distraction detection method, the RGB image containing the driver posture is converted into the edge image, background noise is low, only the driver posture related to a task is reserved, powerful support is provided for picture training and reasoning speed improvement, the convolution layer with the convolution kernel size being 6 * 6 is used, and the driving distraction detection method is more accurate. And the calculation amount and the over-fitting risk are reduced while a larger receptive field is ensured to capture global information.

Description

Technical Field

[0001] The present invention belongs to the technical field of automobile safe driving, and in particular relates to a driving distraction detection model and method. Background Art

[0002] According to the World Health Organization (WHO)'s "Global Status Report on Road Safety 2023," although annual road traffic fatalities have declined slightly, 1.19 million people still died in 2021 alone. Road traffic remains the leading killer of children and young people aged 5-29. Based on statistics from various countries on driver-related traffic accidents, the most common and harmful behaviors are speeding, driving under the influence of alcohol or psychotropic drugs, and distracted driving. Statistics show that distracted driving accounts for more than half of these accidents. The U.S. National Highway Traffic Safety Administration (NHTSA) defines distracted driving as a specific type of inattention in which a driver's attention shifts from the driving task to another activity. Distracted driving causes countless casualties and economic losses each year. It has been observed that these accidents could be completely prevented if driver errors could be detected and appropriately warned. Therefore, many cars are now equipped with advanced driver assistance systems (ADAS), such as lane departure warning (LDW) and forward collision warning (FCW). A popular method currently uses RGB images to monitor driver behavior. This contactless device can prevent human interference with the driver. Furthermore, with the advancement of computer vision technology, video-based methods are processing faster and faster, rapidly becoming a viable solution for autonomous driving. This approach can also be applied to other tasks within Intelligent Traffic Systems (ITS), such as fatigue and driver attention detection, driver skeleton detection, and child in-vehicle detection.

[0003] With the rapid development of deep learning and computer vision, many researchers have devoted themselves to studying various types of distracted driving detection (DDD). In recent years, deep learning methods have been widely applied to image classification, detection, segmentation, and other problems. Compared with traditional algorithms, deep learning offers superior performance and accuracy. Extensive research has been conducted on deep convolutional methods, and many high-performing models have been proposed, such as LeNet, AlexNet, VGG16 and VGG19, GooLeNet, and ResNet. While these models have achieved promising results, they are too large and contain too many parameters, making them unsuitable for real-time processing in embedded systems. Summary of the Invention

[0004] The purpose of the present invention is to overcome the defects in the prior art and provide a driving distraction detection model and method.

[0005] A first aspect of the present invention provides a driving distraction detection model, comprising: include: Image acquisition module: used to acquire RGB images containing driver posture; Image processing module: performing image edge processing on the RGB image based on the Sobel operator to obtain an edge image; Feature vector extraction module: including a plurality of feature extraction units, each of which performs convolution operation on the edge image in turn to obtain a feature vector; Feature vector dimension reduction output module: used to reduce the dimension of the feature vector and output the prediction result.

[0006] A further solution is that each of the feature extraction units respectively completes at least normalization processing and average value pooling for the edge image.

[0007] A further solution is that the feature extraction unit includes a first feature extraction unit, a second feature extraction unit, a third feature extraction unit and a fourth feature extraction unit arranged in sequence; The first feature extraction unit, the second feature extraction unit, the third feature extraction unit and the fourth feature extraction unit each include a convolution layer, a normalization layer, an excitation layer and an average pooling layer arranged in sequence; In the first feature extraction unit, the second feature extraction unit, the third feature extraction unit and the fourth feature extraction unit, the number of convolution kernels of the convolution layer increases successively; the activation function adopted by the excitation layer is the ReLU function.

[0008] A further solution is that the number of convolution kernels of the first feature extraction unit is 32 and the convolution kernel size is 6×6; The number of convolution kernels of the second feature extraction unit is 64 and the convolution kernel size is 6×6; The number of convolution kernels of the third feature extraction unit is 128, and the convolution kernel size is 6×6; The number of convolution kernels of the fourth feature extraction unit is 128 and the size of the convolution kernel is 6×6.

[0009] A further solution is that the model further includes a multi-scale global attention module, and the multi-scale global attention module is connected to the feature vector extraction module; The multi-scale global attention module includes several different branch convolution layers, obtains a convolution feature vector and a multi-scale depth-separable convolution feature vector, performs feature fusion on the convolution feature vector and the multi-scale depth-separable convolution feature vector, and obtains an enhanced feature vector; The feature fusion is to connect the convolution feature vector and the multi-scale depth-separable convolution feature vector using a method similar to residual connection.

[0010] A further solution is that the multi-scale global attention module includes a first branch convolution layer, a first branch convolution layer and a third branch convolution layer, the first branch convolution layer is a convolution with a convolution kernel size of 1×1, the second branch convolution layer is composed of a depth convolution with a convolution kernel size of 3×3 and a point convolution with a convolution kernel size of 1×1, and the third branch convolution layer is composed of a depth convolution with a convolution kernel size of 5×5 and a point convolution with a convolution kernel size of 1×1; Assuming that x1' is the feature vector output by the first branch convolution layer, x2' is the feature vector output by the first branch convolution layer, and x3' is the feature vector output by the third branch convolution layer, the enhanced feature vector x is: .

[0011] A further solution is that the feature vector dimensionality reduction output module includes a pooling layer and a fully connected layer, and the pooling layer and the fully connected layer are connected to the multi-scale global attention module in sequence to perform adaptive maximum pooling and full connection operations on the enhanced feature vector, reducing the feature vector dimension from 256 to n, where n is the number of detection categories to be predicted.

[0012] A second aspect of the present invention provides a method for detecting distracted driving, which uses the above-mentioned distracted driving detection model and includes the following steps: S1, obtaining an RGB image containing the driver's posture; S2. Perform image edge processing on the RGB image using the Sobel operator to obtain an edge image; S3, inputting the edge image into the feature extraction unit in sequence to obtain a feature vector; S4. After performing adaptive maximum pooling and full connection operations on the acquired feature information, the type of distracted driving is output.

[0013] A further solution is that the step S3 of obtaining the feature vector specifically includes: The edge image is convolved once using 32 convolution kernels, a convolution kernel size of 6×6, an activation function of ReLU, and convolution with BatchNorm and average pooling; then a convolution operation is performed once using 64 convolution kernels, a convolution kernel size of 6×6, an activation function of ReLU, and convolution with BatchNorm and average pooling; then a convolution operation is performed once using 128 convolution kernels, a convolution kernel size of 6×6, an activation function of ReLU, and convolution with BatchNorm and average pooling; finally, a convolution operation is performed once using 256 convolution kernels, a convolution kernel size of 6×6, an activation function of ReLU, and convolution with BatchNorm and average pooling to obtain the feature vector.

[0014] A further solution is to feed the feature vector obtained in step S3 into a multi-scale global attention module, perform feature fusion on the feature vector extracted by the first branch and the feature vectors obtained by the last two branches through element-wise multiplication and addition, and use the captured hand and body motion features of the driver to classify distracted driving behavior.

[0015] Compared with the prior art, the present invention has the following beneficial effects: This method converts RGB images containing driver posture into edge images, reducing background noise and retaining only the driver's posture relevant to the task, providing strong support for improving image training and inference speed. Using a convolutional layer with a kernel size of 6×6 ensures a larger receptive field to capture global information while reducing computational overhead and the risk of overfitting.

[0016] The present invention adds a multi-scale global attention module after the four-layer convolution, which contains two depth-wise separable convolution blocks of different scales. This improves the model's feature mining capabilities without adding too many additional parameters. To make the model more expressive, the feature vectors are combined in a residual manner, allowing the model to pay more attention to information related to the driver's actions in the global information. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The following drawings are merely provided for illustrative purposes only and are not intended to limit the scope of the present invention. Figure 1 : The overall framework diagram of the detection model of the present invention; Figure 2 : Edge map, directional gradient histogram, local binary Figure 3 Visual comparison of images; Figure 3 : Schematic diagram of depth-wise convolution and point-wise convolution. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution, design method and advantages of the present invention more clear, the present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0019] The present invention processes edge images based on a multi-scale deep separable convolutional attention neural network. The edge image acquisition method is to use the Sobel operator to extract the RGB image. The edge image can retain the feature information necessary for model classification, such as the posture and action of the person. By converting the image into an edge image, unnecessary background information can be filtered out, allowing the network to focus on identifying different driving actions. Figure 1 As shown, the detection model has four convolutional layers, including convolution, batch normalization, ReLU and pooling. Batch normalization can transform the data into a normal distribution, which helps the neural network to better learn feature information, and also improves the performance and stability of the neural network. It is followed by a normalization operation, including the ReLU activation function and the 2×2 average pooling operation. The average pooling can retain more background information. Since the present invention uses edge images that have removed most of the background information, the average pooling is more suitable and the effect is better. The four-layer convolution adopted in the present invention has a convolution kernel size of 6×6, with 32 filters in the first layer, 64 in the second layer, 128 in the third layer, and 256 in the fourth layer. Because the data set is not augmented, network overfitting will not occur, and no random dropout layer is used. After four layers of convolution, the obtained feature map is fed into a multi-scale global attention module, which includes multi-scale depthwise convolution (with kernel sizes of 3×3 and 5×5, respectively) and point convolution. The feature vectors are then connected in a residual-like manner and finally classified through a global average pooling layer and a fully connected layer.

[0020] The whole process of the model proposed by the present invention can be expressed as:

[0021] in, represents the feature vector extracted by the i-th feature extraction unit, refers to the input edge image, represents a convolution kernel with a size of 6×6. represents a convolution kernel with a size of 1×1, represents the batch normalization operation, Represents the ReLU activation function operation, represents the average pooling operation, represents point convolution, and They represent depthwise convolution with kernel sizes of 3×3 and 5×5, respectively.

[0022]

[0023] and They represent element-wise multiplication and addition operations respectively. The present invention uses a method similar to residual connection to connect the feature information extracted by multi-scale depth-separable convolution and the features extracted by convolution.

[0024]

[0025] in, represents the adaptive maximum pooling operation, represents a fully connected operation, Represents the label of the output.

[0026] Example 1 like Figure 1 As shown, the present invention provides a driving distraction detection model, comprising: Image acquisition module: used to acquire RGB images containing driver posture; Image processing module: perform image edge processing on the RGB image based on the Sobel operator to obtain an edge image, such as Figure 2 As shown, through edge map, directional gradient histogram, local binary Figure 3 From the visual comparison of the three images, it can be seen that the edge image has the least information and only retains the basic outline of the driver; the HOG image has slightly more details and some small amount of background noise; the LBP image has more details and almost retains all the features of the original image except color. Corresponding to these details is the size of the image: the edge image has the smallest size, followed by the HOG image, and the LBP image has the largest size. Therefore, in this embodiment, the Sobel operator is used to perform image edge processing on the RGB image to obtain an edge image with less background noise and only retaining the driver's posture related to the task, which provides strong support for improving the training and inference speed of the image.

[0027] Feature vector extraction module: includes four feature extraction units, each of which performs a convolution operation on the edge image in turn to obtain a feature vector; specifically, the four feature extraction units are 32 convolution kernels, the convolution kernel size is 6×6, the activation function is ReLU, and the convolution includes BatchNorm and average pooling, 64 convolution kernels, the convolution kernel size is 6×6, the activation function is ReLU, and the convolution includes BatchNorm and average pooling, 128 convolution kernels, the convolution kernel size is 6×6, the activation function is ReLU, and the convolution includes BatchNorm and average pooling, and 256 convolution kernels, the convolution kernel size is 6×6, the activation function is ReLU, and the convolution includes BatchNorm and average pooling.

[0028] Multi-scale global attention modules, such as Figure 2 As shown, it contains multi-scale depth convolution (the convolution kernel sizes are 3×3 and 5×5 respectively) and point convolution, which are connected to the feature vector extraction module; specifically, the multi-scale global attention module includes three branches, the first branch is a convolution with a convolution kernel size of 1×1, the second branch consists of a depth convolution with a convolution kernel size of 3×3 and a point convolution with a convolution kernel size of 1×1, and the third branch consists of a depth convolution with a convolution kernel size of 5×5 and a point convolution with a convolution kernel size of 1×1; the feature vector obtained by the feature vector extraction module is input into the multi-scale global attention module, and the three branches respectively obtain feature vectors and fuse them in the form of element-wise multiplication and addition in a residual manner.

[0029] Feature vector dimensionality reduction output module: used to reduce the dimension of the feature vector and output the prediction result. In this embodiment, the feature vector dimensionality reduction output module includes a pooling layer and a fully connected layer, which are sequentially connected to the multi-scale global attention module to perform adaptive maximum pooling and fully connected operations on the fused feature vector, reducing the feature vector dimension from 256 to 10, and outputting 10 predicted detection categories, namely normal driving, making a phone call with the right hand, sending a message with the right hand, making a phone call with the left hand, sending a message with the left hand, adjusting the radio, drinking water, reaching behind the back, applying makeup or touching hair, and chatting with passengers.

[0030] Example 2 This embodiment provides a method for detecting distracted driving, including the following steps: (1) First, the RGB image is processed by the Sobel operator to obtain the edge image, and then the edge image is sequentially input into the four feature extraction modules to obtain the feature vector; (2) The acquired feature vector is fed into the multi-scale global attention module to enhance the model’s ability to capture long-range dependencies and local information, enabling the model to better acquire the feature information of the image for classification; (3) After the acquired feature information is subjected to adaptive maximum pooling and full connection, the types of distracted driving are output, which are 10 in total, namely normal driving, making a phone call with the right hand, sending a message with the right hand, making a phone call with the left hand, sending a message with the left hand, adjusting the radio, drinking water, reaching behind the body, putting on makeup or touching hair, and chatting with passengers.

[0031] Furthermore, the edge image is input into the feature extraction module to obtain the feature vector in the following specific steps: Step (1-1): Process the RGB image with the Sobel operator to obtain the edge image, and then use 32 convolution kernels, a convolution kernel size of 6×6, an activation function of ReLU, and a convolution with BatchNorm and average pooling for one convolution operation, and then use 64 convolution kernels, a convolution kernel size of 6×6, an activation function of ReLU, and a convolution with BatchNorm and average pooling for one convolution operation, and then use 128 convolution kernels, a convolution kernel size of 6×6, an activation function of ReLU, and a convolution with BatchNorm and average pooling for one convolution operation, and finally use 256 convolution kernels, a convolution kernel size of 6×6, an activation function of ReLU, and a convolution with BatchNorm and average pooling for one convolution operation to obtain the feature vector .

[0032] Step (1-2): The feature vector obtained in step (1-1) The result is fed into a multi-scale global attention module, which has three branches. The first branch is a convolution with a kernel size of 1×1. The second branch consists of a depthwise convolution with a kernel size of 3×3 and a pointwise convolution with a kernel size of 1×1. The third branch consists of a depthwise convolution with a kernel size of 5×5 and a pointwise convolution with a kernel size of 1×1. The feature vector extracted by the first branch and the feature vectors obtained by the last two branches are then fused in a residual manner through element-wise multiplication and addition. The captured hand and body motion features of the driver are used to classify distracted driving behavior.

[0033] Furthermore, the method for feature dimensionality reduction and classification of the feature vector extracted in step (1-2) is to first subject the extracted feature vector to adaptive maximum pooling, and then perform a full connection operation to reduce the feature vector dimension from 256 to 10, and output the predicted 10 categories.

[0034] While various embodiments of the present invention have been described above, the above descriptions are intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technological improvements in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A driving distraction detection model, characterized in that: include: Image acquisition module: used to acquire RGB images containing driver posture; Image processing module: performing image edge processing on the RGB image based on the Sobel operator to obtain an edge image; Feature vector extraction module: including a plurality of feature extraction units, each of which performs convolution operation on the edge image in turn to obtain a feature vector; Feature vector dimension reduction output module: used to reduce the dimension of the feature vector and output the prediction result.

2. The driving distraction detection model according to claim 1, characterized in that: Each of the feature extraction units respectively performs at least normalization processing and average value pooling on the edge image.

3. The driving distraction detection model according to claim 1, characterized in that: The feature extraction unit includes a first feature extraction unit, a second feature extraction unit, a third feature extraction unit and a fourth feature extraction unit which are arranged in sequence; The first feature extraction unit, the second feature extraction unit, the third feature extraction unit and the fourth feature extraction unit each include a convolution layer, a normalization layer, an excitation layer and an average pooling layer arranged in sequence; In the first feature extraction unit, the second feature extraction unit, the third feature extraction unit and the fourth feature extraction unit, the number of convolution kernels of the convolution layer increases successively; the activation function adopted by the excitation layer is the ReLU function.

4. The driving distraction detection model according to claim 3, characterized in that: The number of convolution kernels of the first feature extraction unit is 32, and the convolution kernel size is 6×6; The number of convolution kernels of the second feature extraction unit is 64 and the convolution kernel size is 6×6; The number of convolution kernels of the third feature extraction unit is 128, and the convolution kernel size is 6×6; The number of convolution kernels of the fourth feature extraction unit is 128 and the size of the convolution kernel is 6×6.

5. The driving distraction detection model according to claim 1, characterized in that: The model also includes a multi-scale global attention module, which is connected to the feature vector extraction module; The multi-scale global attention module includes several different branch convolution layers, obtains a convolution feature vector and a multi-scale depth-separable convolution feature vector, performs feature fusion on the convolution feature vector and the multi-scale depth-separable convolution feature vector, and obtains an enhanced feature vector; The feature fusion is to connect the convolution feature vector and the multi-scale depth-separable convolution feature vector using a residual connection method.

6. The driving distraction detection model according to claim 5, characterized in that: The multi-scale global attention module includes a first branch convolution layer, a first branch convolution layer and a third branch convolution layer, wherein the first branch convolution layer is a convolution with a convolution kernel size of 1×1, the second branch convolution layer is composed of a depth convolution with a convolution kernel size of 3×3 and a point convolution with a convolution kernel size of 1×1, and the third branch convolution layer is composed of a depth convolution with a convolution kernel size of 5×5 and a point convolution with a convolution kernel size of 1×1; Assuming that x1' is the feature vector output by the first branch convolution layer, x2' is the feature vector output by the first branch convolution layer, and x3' is the feature vector output by the third branch convolution layer, the enhanced feature vector x is: 。 7. The driving distraction detection model according to claim 5, characterized in that: The feature vector dimensionality reduction output module includes a pooling layer and a fully connected layer, which are connected to the multi-scale global attention module in sequence to perform adaptive maximum pooling and full connection operations on the enhanced feature vector, reducing the feature vector dimension from 256 to n, where n is the number of detection categories to be predicted.

8. A method for detecting distracted driving, characterized in that: Applying the driving distraction detection model described in any one of claims 1 to 7 comprises the following steps: S1, obtaining an RGB image containing the driver's posture; S2. Perform image edge processing on the RGB image using the Sobel operator to obtain an edge image; S3, inputting the edge image into the feature extraction unit in sequence to obtain a feature vector; S4. After performing adaptive maximum pooling and full connection operations on the acquired feature information, the type of distracted driving is output.

9. The method for detecting distracted driving according to claim 8, wherein: The step S3 of obtaining the feature vector specifically includes: The edge image is convolved once using 32 convolution kernels, a convolution kernel size of 6×6, an activation function of ReLU, and convolution with BatchNorm and average pooling; then a convolution operation is performed once using 64 convolution kernels, a convolution kernel size of 6×6, an activation function of ReLU, and convolution with BatchNorm and average pooling; then a convolution operation is performed once using 128 convolution kernels, a convolution kernel size of 6×6, an activation function of ReLU, and convolution with BatchNorm and average pooling; finally, a convolution operation is performed once using 256 convolution kernels, a convolution kernel size of 6×6, an activation function of ReLU, and convolution with BatchNorm and average pooling to obtain the feature vector.

10. The method for detecting distracted driving according to claim 9, wherein: The feature vector obtained in step S3 is fed into the multi-scale global attention module. The feature vector extracted by the first branch and the feature vectors obtained by the last two branches are fused by element-wise multiplication and addition. The captured hand and body motion features of the driver are used to classify distracted driving behavior.

Images