Driving distraction identification method, device, equipment and storage medium

Abstract

The present application relates to the fields of computer vision and deep learning technology, and discloses a method, apparatus, device, and storage medium for identifying distracted driving, which are used to address the technical problem that traditional distracted driving identification solutions have difficulty balancing the timeliness and accuracy of identification. The distracted driving identification method includes: inputting the original driver image data into the input layer of a preset distracted driving identification model for preprocessing, and extracting features from the candidate driver image data through a depthwise separable convolutional layer to obtain an initial driving behavior feature map; inputting the initial driving behavior feature map into a multi-spectral attention layer for spectral segmentation, and performing an attention fusion operation on the feature maps of each spectral range to obtain a fused feature map; inputting the fused feature map into a pooling layer and a feature flattening layer for processing to obtain a feature vector of the driving behavior to be identified; and inputting the feature vector of the driving behavior to be identified into a classifier for driving distracted behavior identification to obtain a distracted driving identification result.

Description

Technical Field

[0001] The present application relates to the fields of computer vision and deep learning technology, and in particular to a method, apparatus, device, and storage medium for identifying distracted driving. Background Art

[0002] With the rapid development of intelligent transportation systems and autonomous driving technologies, driving safety has become a major concern in society. Distracted driving behaviors, such as using a mobile phone, eating, or talking to passengers, significantly increase the risk of traffic accidents. Therefore, developing efficient and accurate methods to identify distracted driving is of great practical significance.

[0003] In related technologies, traditional distracted driving identification solutions, due to considerations of the operation and resource usage of on-board equipment, are usually based on feature extraction algorithms such as Histogram of Oriented Gradients (HOG), Scale-Invariant Feature Transform (SIFT), and shallow neural network models for feature extraction. These solutions then classify and identify distracted driving behaviors based on classifiers (such as support vector machines and decision trees), which can, to a certain extent, identify distracted driving.

[0004] However, the feature extraction method of traditional driving distraction recognition scheme has high computational complexity when processing high-dimensional data, which makes it difficult to meet the needs of real-time recognition. Moreover, due to the limitations of the feature extraction method, it is difficult to fully capture the subtle characteristics of the driver's distracted behavior, resulting in low accuracy of driving distraction recognition. Summary of the Invention

[0005] The present application provides a method, apparatus, device and storage medium for identifying distracted driving, which are used to solve the technical problem that traditional distracted driving identification solutions have difficulty in balancing the timeliness and accuracy of identification.

[0006] A first aspect of the present application provides a method for identifying distracted driving, comprising: inputting original driver image data into a preset distracted driving identification model, performing preprocessing through an input layer, and obtaining candidate driver image data;

[0007] The candidate driver image data is subjected to feature extraction through a depthwise separable convolutional layer to obtain an initial driving behavior feature map;

[0008] The initial driving behavior feature map is input into the multi-spectral attention layer for spectrum division, and the feature maps of each spectrum range are subjected to attention fusion operation to obtain the fused feature map;

[0009] The fused feature map is input into the pooling layer and the feature flattening layer for processing to obtain the feature vector of the driving behavior to be identified;

[0010] The feature vector of the driving behavior to be identified is input into the classifier to perform distracted driving behavior identification and obtain the distracted driving identification result.

[0011] A second aspect of the present application provides a driving distraction recognition device, comprising: a preprocessing unit, configured to input original driver image data into a preset driving distraction recognition model, perform preprocessing through an input layer, and obtain candidate driver image data;

[0012] A feature extraction unit is used to extract features from the candidate driver image data through a depthwise separable convolutional layer to obtain an initial driving behavior feature map;

[0013] The frequency division attention unit is used to input the initial driving behavior feature map into the multi-spectral attention layer for spectrum division, and perform attention fusion operation on the feature maps of each spectrum range to obtain a fused feature map;

[0014] The feature processing unit is used to input the fused feature map into the pooling layer and the feature flattening layer for processing to obtain the feature vector of the driving behavior to be identified;

[0015] The recognition unit is used to input the feature vector of the driving behavior to be recognized into the classifier to recognize the distracted driving behavior and obtain the distracted driving recognition result.

[0016] A third aspect of the present application provides a driving distraction recognition device, comprising: a memory and at least one processor, wherein instructions are stored in the memory; and at least one processor calls the instructions in the memory so that the driving distraction recognition device executes the above-mentioned driving distraction recognition method.

[0017] A fourth aspect of the present application provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, the computer executes the above-mentioned driving distraction identification method.

[0018] The technical solution provided in this application significantly improves the performance of distracted driving detection by introducing a driving distraction recognition model that combines deep separable convolution and multi-spectral attention mechanisms. In particular, it achieves a good balance between efficient real-time and high-precision detection. This addresses the problem that traditional driving distraction recognition solutions rely on manually designed local feature information such as edges and corners, but are deficient in capturing complex driver behavior characteristics, especially subtle changes in driving behavior. Shallow neural networks find it difficult to deeply understand the deep semantic information in image data. This application significantly reduces computational complexity and parameter count through deep separable convolution, while effectively retaining feature extraction capabilities, thereby meeting the dual requirements of real-time and accuracy. The multi-spectral attention mechanism can adaptively weight the different spectral features of the input image data, thereby enhancing the model's sensitivity to key driving behavior features and overcoming the traditional method's lack of ability to capture instantaneous behavior, thereby effectively capturing these subtle but crucial feature changes and greatly improving the accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a schematic diagram of the first embodiment of the method for identifying distracted driving in this application;

[0020] Figure 2 This is a schematic diagram of a second embodiment of the method for identifying distracted driving in this application;

[0021] Figure 3 A schematic diagram of the network architecture of the distracted driving recognition model for this application;

[0022] Figure 4 This is a schematic diagram of a third embodiment of the method for identifying distracted driving in this application;

[0023] Figure 5 A schematic diagram of an embodiment of a distracted driving identification device in this application;

[0024] Figure 6 This is a schematic diagram of another embodiment of the driving distraction identification device in this application;

[0025] Figure 7 This is a schematic diagram of an embodiment of a distracted driving identification device in this application. DETAILED DESCRIPTION

[0026] The present application provides a method, apparatus, device and storage medium for identifying distracted driving, which are used to ensure the accuracy and real-time performance of distracted driving identification.

[0027] In the specification and claims of this application and the accompanying drawings, the terms "first," "second," "third," "fourth," and so on (if any) are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that shown or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, product, or apparatus.

[0028] It should be noted that the image data, videos and the privacy information that may be involved used in this application are implemented with the authorization of the identified user and / or in compliance with relevant legal provisions.

[0029] For ease of understanding, the specific process of this application is described below. Figure 1 , an embodiment of the method for identifying distracted driving in this application includes:

[0030] 101. Input the original driver image data into a preset driving distraction recognition model and perform preprocessing through the input layer to obtain candidate driver image data.

[0031] It is understood that the execution entity of this application can be a driver distraction identification device, various driving terminals (such as vehicles), or driving assistance systems, intelligent driving systems, autonomous driving systems, etc., and, where appropriate, various road monitoring terminals and systems, without limitation. This application is described using a vehicle as the execution entity.

[0032] In this embodiment, the original driver image data can be derived from relevant videos collected by a vehicle-mounted camera or other channels, and the image data records the driver's status. For example, a vehicle-mounted camera installed above the windshield, near the rearview mirror, or below the dashboard captures the driver's real-time image data to ensure that the driver's facial expressions, head posture, eye movements, hands and other body parts can be clearly captured.

[0033] It is understandable that the distracted driving recognition model of this embodiment can be applied to the recognition of distracted driving behavior in a single image or in the entire video, and this embodiment does not impose any specific limitations.

[0034] It should be noted that for the recognition of distracted driving behavior in a single image, only one frame of original driver image data can be input, or a preset number of consecutive frames of original driver image data can be input to improve the single-frame recognition accuracy. For example, 10 frames of images with motion changes are input, and the 5th frame is recognized. The model extracts the temporal change information in these 10 frames, combines the temporal change information to adjust the attention weight, and determines whether there is distracted driving behavior in the 5th frame.

[0035] It is understood that the network architecture provided in this embodiment can be used for both video and single-image applications to identify distracted driving. For scenarios where distracted driving is detected throughout an entire video, the architecture provided in this embodiment can utilize multiple frames of image data to extract temporal variations by differencing or fusing information from consecutive frames. Specifically, inter-frame variation features are extracted from each feature map to obtain inter-frame variation features. The accuracy of the allocation of attention weights in the multi-spectral attention layer can then be further evaluated. To extract inter-frame variation features, optical flow, recurrent neural networks (RNNs), long short-term memory (LSTMs), or transformer architectures can be considered to improve the accuracy of extracting temporal variation information. For details, refer to conventional solutions and are not detailed here.

[0036] For scenarios where the entire video contains distracted driving behavior, the evaluation result can be a comprehensive evaluation result of the image recognition results of each frame, for example, the proportion of image frames identified as containing distracted driving behavior in the total number of frames, the duration, frequency, etc., to improve the accuracy of the final evaluation; it can also include the total number of image frames identified as containing distracted driving behavior, the frame number, and the behavior type (i.e., classification label) to mark whether the video contains distracted driving behavior, and / or all image frames containing distracted driving behavior, without specific limitation.

[0037] To facilitate understanding, the following explanation will use the distracted driving behavior recognition scenario for a single image as an example. That is, the input is the original driver image data, a frame of the image is evaluated, and the remaining images are used to identify temporal change information to optimize the distribution of attention fusion operations.

[0038] The distracted driving recognition model of this embodiment introduces depthwise separable convolution and multi-spectral attention mechanisms to provide an efficient and accurate distracted driving detection solution, which is suitable for real-time monitoring and detection of driver distraction behavior.

[0039] In this embodiment, the network architecture of the driving distraction recognition model includes an input layer (or image data processing layer), a depthwise separable convolution layer (or feature extraction layer), a multi-spectral attention layer, a pooling layer, a feature flattening layer, and a classifier, which are connected in sequence.

[0040] In a feasible embodiment, the original driver image data is adjusted to the standard required by the subsequent depthwise separable convolutional layer processing through the input layer to be compatible with changes in various situations and to ensure that the driving distraction recognition model can perform stable inference. Exemplarily, the preprocessing of the input layer includes but is not limited to cropping, rotation, flipping and normalization techniques.

[0041] It should be noted that the driving distraction recognition model can be a pre-trained model, or the model parameters can be automatically iteratively optimized step by step during use to improve recognition accuracy. This embodiment does not limit the model training process.

[0042] 102. The candidate driver image data is subjected to feature extraction through a depthwise separable convolutional layer to obtain an initial driving behavior feature map.

[0043] In this embodiment, the depthwise separable convolution (DSC) layer mainly includes two steps: depthwise convolution and pointwise convolution. Depthwise convolution (also known as spatial convolution) performs convolution operations on each input channel separately without mixing information between channels, while pointwise convolution performs linear combinations in the channel dimension to generate a new feature map, namely the initial driving behavior feature map. Compared with standard convolution, depthwise separable convolution significantly reduces the amount of computation and the number of parameters while maintaining good performance.

[0044] In this embodiment, the depthwise separable convolutional layer can include at least one depthwise convolution module, each of which includes two steps: depthwise convolution and pointwise convolution. This embodiment does not limit the specific number of depthwise convolution modules. For two or more depthwise convolution modules, they are connected via module cascade, that is, the output of the previous depthwise convolution module serves as the input of the next depthwise convolution module, until the last depthwise convolution module outputs the initial driving behavior feature map. Furthermore, the spatial dimensions of each depthwise convolution module can be set to increase sequentially to gradually improve the accuracy of feature extraction.

[0045] It is understandable that as the number of deep volume modules increases, the accuracy of feature extraction increases, and the consumption of computing resources also increases. In practical applications, an appropriate number of deep volume modules can be selected to extract the initial driving behavior features.

[0046] Optionally, the depthwise separable convolution layer includes multiple depth sub-volume modules, which input the candidate driver image data into each depth sub-volume module in sequence, perform spatial convolution operation on the input feature map through depth convolution, and integrate cross-channel information through point-by-point convolution and then input it into the next depth sub-volume module, until the last depth sub-volume module outputs the initial driving behavior feature map.

[0047] Optionally, the depthwise separable convolution layer includes a depthwise convolution module. The depthwise convolution uses a 3×3 convolution kernel and performs convolution within each input channel. The formula is expressed as:

[0048]

[0049] in, Represents the feature map output after the depth convolution operation, is the depth convolution kernel, Represents the convolution operation; Represents the input feature map;

[0050] Point-by-point convolution uses a 1×1 convolution kernel for channel integration, and the formula is expressed as:

[0051]

[0052] in, Represents the feature map output after point-by-point convolution operation, It is a depth convolution kernel, which is used to integrate the channels of the feature map after depth convolution.

[0053] That is, when the depth-wise separable convolutional layer includes a depth-wise convolutional module, its formula can be expressed as:

[0054]

[0055] in, Represents the initial driving behavior feature map output by the depthwise separable convolutional layer.

[0056] In this embodiment, the structure adopted by the depthwise separable convolutional layer significantly reduces the computational complexity and number of parameters of the model while maintaining the effect of feature extraction. down to ,in is the size of the convolution kernel, that is, the spatial dimension of the convolution kernel, is the number of input channels, is the number of output channels.

[0057] In this embodiment, the initial driving behavior feature map is used to indicate the map of shallow features, or relatively low-level features, extracted after processing by the depthwise separable convolutional layer, such as directional edges, color changes, and local textures. These features are crucial for distinguishing the background from key parts (such as hands and faces). For example, edge detection helps identify hand contours, facial features (such as eyes and mouths), and the boundaries of objects in the car, while color changes and local textures can help detect the contrast differences between hands and steering wheels, mobile phones, or other objects. Using these low-level features, the model can preliminarily determine whether the driver's hands are on the steering wheel or whether there is an obvious handheld object, such as a mobile phone or a beverage bottle.

[0058] It should be noted that the introduction of inter-frame variation features can further improve the accuracy of distracted driving behavior recognition. For example, it has reference value for the allocation of attention weights in the subsequent multi-spectral attention mechanism. This embodiment does not limit the specific number of indicated initial driving behavior feature maps. Typically, each frame of the image corresponds to an initial driving behavior feature map to calculate the temporal variation information between consecutive frames, that is, the inter-frame variation features, to better guide the accurate recognition of instantaneous behavior features.

[0059] 103. Input the initial driving behavior feature map into the multi-spectral attention layer for spectrum segmentation, and perform attention fusion operation on the feature maps of each spectrum range to obtain a fused feature map.

[0060] In this embodiment, the Multi-Spectral Attention (MSA) layer mainly includes three steps: spectrum division, attention weight calculation, and feature weighted fusion. By dividing the initial driving behavior feature map into multiple spectral ranges, the feature map of each spectral range is assigned attention weights based on the attention mechanism and weighted fusion is performed to obtain a fused feature map. By using the multi-spectral attention mechanism, this embodiment compensates for the shortcomings of the existing channel attention mechanism in insufficient feature information. By introducing more frequency components to fully utilize feature information and dynamically adjusting the weights of different spectral features, the model's sensitivity and recognition ability for key features are enhanced, thereby improving the accuracy of distracted driving detection.

[0061] It is understandable that the multi-spectral attention mechanism can adaptively weight the different spectral features of the input image data, enhancing the model's sensitivity to key driving behavior characteristics. The characteristics of distracted driving often manifest as subtle changes in the driver's behavior, such as momentary eye deviations, slight head turns, or involuntary hand movements. These subtle changes are difficult to capture with traditional methods. However, after adopting the multi-spectral attention mechanism, the network can adaptively weight features in different spectral ranges, assigning greater weights to spectral features that are more closely related to distracted driving behavior and less weights to spectral features that are less closely related to distracted driving behavior. This effectively captures these subtle but crucial feature changes and greatly improves the accuracy of the model.

[0062] In this embodiment, the spectrum division of the multi-spectral attention layer can be processed by spectrum conversion functions such as discrete cosine transform function, discrete Fourier transform function, wavelet transform function, etc., without specific limitation.

[0063] After obtaining feature maps for each spectral range through the aforementioned spectral conversion function, a pre-set attention mechanism is used to first determine the importance of each spectral range's feature map and then determine the effectiveness of feature maps from different spectral ranges. This mechanism then learns to allocate attention weights to spectral features that are more relevant to identifying distracted driving. This network extracts image features, not only capturing semantic features but also preserving the location information of minute features within the image space. It selectively aggregates context with a global contextual view, making key minute features more compact and clustered, thereby improving subsequent classification accuracy with a smaller number of parameters.

[0064] The multi-spectral attention layer of this embodiment processes the initial driving behavior feature map output by the DSC. Compared to processing the ordinary feature map output by a conventional convolutional neural network (CNN), it can more fully utilize the technical advantages of the DSC. The DSC first effectively screens and optimizes the features before feeding them into the MSA for global feature enhancement, thereby reducing the MSA's computational workload while improving the attention enhancement effect. This approach is more effective than using the MSA alone, more accurately capturing key areas of the driving scene (such as gestures and facial movements). It is also more lightweight than the MSA solution that directly operates on CNN features, enabling good recognition results even with limited computing resources. By utilizing the additional spectral variable information provided by the feature maps in each spectral range to more accurately assign attention weights, it can effectively increase the focus on features associated with distracted driving behavior in the fused feature map.

[0065] Compared to traditional CNN architectures, the distracted driving recognition model in this embodiment features multiple optimizations in its network architecture. While traditional CNNs are computationally complex, this architecture optimizes computational efficiency, feature extraction capabilities, and the integration of attention mechanisms. This approach improves recognition accuracy while maintaining computational efficiency, making it particularly suitable for fine-grained classification tasks such as distracted driving detection.

[0066] In this embodiment, the fused feature map is used to indicate the intermediate features after further processing by the multi-spectral attention layer. The DSC at this stage is combined with the MSA mechanism to extract more complex structural features and behavioral patterns. For example, the model can detect dynamic changes in hand position to determine whether the hand has left the steering wheel or is operating a mobile phone. In addition, facial posture information is also effectively extracted at this level, such as the driver's gaze direction, head angle, etc., to help determine whether the driver is paying attention to the road. At the same time, the multi-spectral attention mechanism (MSA) enhances the focus on key areas, enabling the model to more accurately distinguish different object interaction modes. For example, if hand features frequently overlap with a highlighted area (possibly a mobile phone), it can be inferred that the driver is using a mobile phone.

[0067] 104. Input the fused feature map into the pooling layer and the feature flattening layer for processing to obtain the feature vector of the driving behavior to be identified.

[0068] This embodiment uses a pooling layer to downsample the feature map after attention mechanism processing to effectively reduce the spatial dimension of the feature map, while retaining important spatial information and reducing the computational complexity of subsequent processing. The feature flattening layer is used to convert the downsampled feature map into a one-dimensional feature vector to ensure that the feature vector of the driving behavior to be identified is compatible with the subsequent classifier, facilitating subsequent classification and detection tasks.

[0069] In this embodiment, the driving behavior feature vector to be identified is [a missing character]. At the back end of the network, pooling layers and MSA further extract high-level features. The pooling layer may include global max pooling (GMP), enabling the model to understand the driver's overall behavior patterns. This allows the model to not only identify local information about the driver's hands and face but also infer the complete driving behavior pattern based on these features. For example, by combining hand, facial, and body posture information, the model can accurately determine whether the driver is engaging in typical distracted driving behaviors, such as talking on the phone, eating, or looking down at the central control screen. MSA also helps the model better understand the global context and reduce environmental interference, such as changes in interior lighting and background objects (such as passengers in the passenger seat), thereby improving recognition robustness.

[0070] 105. Input the feature vector of the driving behavior to be identified into the classifier to perform distracted driving behavior identification and obtain a distracted driving identification result.

[0071] The classifier of this embodiment is responsible for real-time detection and classification of distracted driving behavior and outputting the final detection results, which may include: forward propagating the flattened feature vector through the classifier to obtain a category probability distribution, converting the output into a probability distribution through the Softmax function, and selecting the category with the highest probability as the final prediction result.

[0072] It can be understood that the distracted driving recognition model of this embodiment covers three levels of feature extraction: low-level features (such as edges, colors, and textures) in the initial driving behavior feature map, intermediate features (such as gestures, faces, and object interactions) in the fused feature map, and high-level features (global behavior patterns) in the driving behavior feature vector to be identified. This enables it to accurately capture information related to distracted driving behavior. Among them, low-level features help distinguish between background and key parts, intermediate features establish the driver's behavior pattern, and high-level features ultimately complete the judgment of the driver's behavior.

[0073] In this embodiment, the driving analysis recognition result is used to indicate whether the driver in the image is engaging in distracted driving behavior. The driving behavior labels (or categories) in this embodiment may include, but are not limited to, safe driving, texting using right hand, talking on the phone using right hand, texting using left hand, talking on the phone using left hand, operating the radio, drinking water, reaching behind, doing hair and makeup, and talking to passengers.

[0074] Optionally, when the driving distraction recognition model identifies that the driver is distracted, it can further remind the driver through various prompts or activate corresponding assisted driving solutions, such as controlling the vehicle speed to reduce, to ensure driving safety, without specific restrictions.

[0075] It is understandable that the driving distraction recognition model provided in this embodiment can be applied to the recognition of real-time behavior, and can also be applied to the recognition of non-real-time behavior, such as inputting a violation video for image extraction and then identifying whether there is driving distraction recognition.

[0076] This embodiment significantly improves the performance of distracted driving detection by introducing a model with deep separable convolution and multi-spectral attention mechanisms. In particular, it achieves a good balance between efficient real-time and high-precision detection. This addresses the shortcomings of traditional distracted driving detection schemes, which rely on manually designed local feature information such as edges and corners, but are unable to capture complex driver behavior characteristics, especially subtle changes in driving behavior. Traditional shallow neural networks have difficulty in deeply understanding the deep semantic information in image data. This application significantly reduces the computational complexity and parameter count through deep separable convolution while effectively retaining feature extraction capabilities, thus meeting the dual requirements of real-time and precision. The multi-spectral attention mechanism can adaptively weight the different spectral features of the input image data, enhancing the model's sensitivity to key driving behavior characteristics, overcoming the shortcomings of traditional methods in capturing instantaneous and subtle behaviors, thereby effectively capturing these subtle but crucial feature changes and greatly improving the accuracy of the model.

[0077] See also Figure 2 and Figure 3 Another embodiment of the method for identifying distracted driving in this application includes:

[0078] 201. Input the original driver image data into a preset driving distraction recognition model and perform preprocessing through the input layer to obtain candidate driver image data.

[0079] Step 201 can be performed with reference to step 101 and will not be described in detail here.

[0080] 202. Perform feature extraction on the candidate driver image data through a depthwise separable convolutional layer to obtain an initial driving behavior feature map.

[0081] Reference Figure 3 Schematic diagram of the network architecture. In this embodiment, the depthwise separable convolution layer is illustrated using two cascaded depthwise convolution modules as an example. Each depthwise convolution module includes depthwise convolution and pointwise convolution. That is, the depthwise separable convolution layer includes a first depthwise convolution module and a second depthwise convolution module. The spatial dimension corresponding to the convolution kernel in the first depthwise convolution module is smaller than the spatial dimension corresponding to the convolution kernel in the second depthwise convolution module.

[0082] Exemplarily, the candidate driver image data is input into the first deep convolution module, and a spatial convolution operation is performed on the input feature map through deep convolution, and the cross-channel information is integrated through point-by-point convolution before being input into the second deep convolution module; the second deep convolution module performs a spatial convolution operation on the input feature map through deep convolution, and the cross-channel information is integrated through point-by-point convolution to obtain an initial driving behavior feature map.

[0083] Optionally, the first depth convolution module can adopt a 1*1 convolution kernel, and the second depth convolution module can adopt a 3*3 convolution kernel.

[0084] It is understandable that this embodiment uses a two-layer cascaded DSC structure for depthwise separable convolution (DSC), rather than the traditional single-layer DSC. The first DSC layer uses 1×1 convolutions to perform channel transformations, reducing computational redundancy and improving expressiveness, while the second DSC layer uses 3×3 convolutions to extract local features, enhancing the network's spatial perception. This two-layer DSC structure has stronger nonlinear expressive power than a single-layer DSC while maintaining a low computational load. Compared to standard CNNs, it significantly reduces the number of parameters and computational complexity while enhancing feature extraction capabilities. It is particularly suitable for tasks such as distracted driving detection, which require high-level, fine-grained features.

[0085] The network architecture of this embodiment optimizes the MSA's target, utilizing the first DSC to reduce computational complexity while simultaneously enhancing feature extraction with the second DSC, achieving a better balance between computational efficiency and attention-enhancing effects. Compared to a conventional single-layer DSC+MSA architecture, the dual-layer DSC improves local feature extraction and, combined with MSA for global feature enhancement, enhances recognition accuracy while maintaining computational efficiency. This approach is particularly well-suited for fine-grained classification tasks such as distracted driving detection.

[0086] 203. Divide the spectrum according to the corresponding spectrum range through each spectrum module, extract the global information of each spectrum feature map, and obtain the global feature map corresponding to each spectrum range.

[0087] The multi-spectral attention layer of this embodiment may include multiple spectrum modules, attention mechanism modules and fusion modules. Each spectrum module independently processes the features of different spectrum ranges, calculates the attention weights of the global feature maps corresponding to each spectrum range based on the attention mechanism, and dynamically adjusts the weights of each spectrum feature in the final feature fusion process.

[0088] It should be understood that each spectrum module corresponds to a spectrum range, and each spectrum module outputs a spectrum feature graph for its corresponding spectrum range. The number of spectrum feature graphs is equal to the number of spectrum modules, and the spectrum range corresponding to each spectrum module can be set according to actual conditions. By dividing the spectrum range, this embodiment enables the model to more finely capture the driver's behavioral characteristics within different spectrum ranges, especially those subtle but critical behavioral changes, thereby significantly improving the accuracy and robustness of distracted driving detection.

[0089] Specifically, the initial driving behavior feature map is divided according to several preset spectrum ranges to obtain multiple spectrum feature maps; global information of each spectrum feature map is extracted through global average pooling and global maximum pooling operations.

[0090] For example, assuming there are D spectrum modules, each spectrum feature graph can be expressed as , Each spectrum module may include a first 2D convolution, a ReLU function, a second 2D convolution and a Sigmoid function, wherein the first 2D convolution and the second 2D convolution may use a 1*1 convolution kernel.

[0091] Optionally, taking the cosine discrete transformation function as an example, the processing process of each spectrum module is explained: the above-mentioned initial driving behavior characteristic map is divided according to several preset spectrum ranges to obtain multiple spectrum characteristic maps, including: using the cosine discrete transformation function to change the initial driving behavior characteristic map according to the preset spectrum range, mapping the coordinates in the spatial domain with the coordinates in the frequency domain to obtain corresponding frequency information, and the high-frequency area in the spectrum characteristic map generally stores feature information that is highly correlated with the identification of distracted driving behavior.

[0092] In one feasible implementation, the initial driving behavior feature map is evenly split into multiple parts along the channel dimension. Each part is assigned a frequency index, and frequency features are obtained through a DCT transform based on the assigned frequency index. A larger index indicates a higher frequency and faster change. This high-frequency information can be used to extract subtle and transient behavioral features, such as instantaneous eye deviations, slight head movements, or involuntary hand movements.

[0093] The above global average pooling formula is as follows:

[0094]

[0095] in, Used to represent the height of the feature map, Used to indicate the width of the feature map; Indicates the The feature map height of the spectrum, Indicates the The feature map width of the spectrum.

[0096] For example, the global maximum pooling formula is as follows:

[0097]

[0098] 204. Calculate the attention weight of the global feature map corresponding to each spectrum range through the attention mechanism module.

[0099] Optionally, the pooled features are passed through a fully connected layer and an activation function to generate the attention weights for each spectral submodule. The formula is as follows:

[0100]

[0101] in, Used to indicate the The attention weight of each spectrum; represents the Sigmoid activation function, The attention weight of each spectral feature map in this embodiment reflects the importance of the spectral feature map, so that more attention is paid to features associated with distracted driving behavior during fusion.

[0102] Optionally, the attention mechanism module can further combine temporal variation information to assign attention weights to the global feature maps corresponding to each spectral range, where temporal variation information indicates feature changes between consecutive frames. This embodiment utilizes the additional spectral variable information provided in the feature maps of each spectral range to improve the recognition accuracy of subtle behaviors. Further combining inter-frame variation features allows for more accurate allocation of attention weights. The resulting fused feature map can account for both instantaneous and subtle behaviors, facilitating subsequent driving behavior classification and improving overall classification accuracy.

[0103] Exemplarily, the extraction of the above-mentioned temporal change information can be to extract the inter-frame change features of the initial driving behavior feature map corresponding to each frame image, that is, to identify the changes of the same features between consecutive frames. In actual applications, the driver's instantaneous and unconscious movements and behaviors will cause the features recognized as hands to change in position between the above-mentioned consecutive frames.

[0104] It should be understood that in the above scenario, the depthwise separable convolution layer of step 202 outputs the initial driving behavior feature map of continuous frames. It is possible to choose to set a temporal change recognition layer between the depthwise separable convolution layer and the multi-spectral attention layer, or to set a temporal change recognition module in parallel with multiple spectrum modules in the multi-spectral attention layer. The temporal change recognition layer or the temporal change recognition module can identify the temporal change information of each region in the continuous frames, that is, to extract inter-frame change features.

[0105] In a feasible implementation, taking the example of setting a temporal change recognition layer between the depthwise separable convolution layer and the multi-spectral attention layer, the initial driving behavior feature map of continuous frames is extracted with the temporal change recognition layer to obtain the inter-frame change features and the temporal change information; the temporal change information and the initial driving behavior feature map are input into the multi-spectral attention layer; the initial driving behavior feature map is divided according to the corresponding spectrum range through each spectrum module, and the global information of each spectrum feature map is extracted to obtain the global feature map corresponding to each spectrum range; the attention weight of the global feature map corresponding to each spectrum range is allocated in combination with the temporal change information through the attention mechanism module; each spectrum feature map is multiplied by the corresponding attention weight and fused through the fusion module to obtain a fused feature map.

[0106] In a feasible implementation, a multi-spectral attention layer is set up as a temporal change recognition module in parallel with multiple spectrum modules. The attention mechanism module is connected to the temporal change recognition module and each spectrum module respectively. Specifically, the initial driving behavior feature map of consecutive frames is input into the temporal change recognition module to extract the inter-frame change features, obtain the temporal change information, and input it into the attention mechanism module.

[0107] The initial driving behavior feature map to be identified is input into each spectrum module and divided according to the corresponding spectrum range. The global information of each spectrum feature map is extracted to obtain the global feature map corresponding to each spectrum range and input into the attention mechanism module.

[0108] The attention mechanism module is used to allocate attention weights to the global feature maps corresponding to each spectral range in combination with the temporal change information; the fusion module multiplies each spectral feature map with the corresponding attention weight and then fuses them to obtain a fused feature map.

[0109] It should be understood that if only a single image needs to be identified for distracted driving, then the input to each spectrum module is only the initial driving behavior feature map of the single frame to be identified; if the entire video or multiple images need to be identified for distracted driving, then the input to each spectrum module is the initial driving behavior feature map of each frame to be identified.

[0110] The above-mentioned continuous frames are used to indicate images of a continuous preset number of frames. For example, the initial driving behavior feature map corresponding to the 1st frame to the 8th frame obtains the temporal feature changes of the continuous frames by differentiating or fusing the information of the continuous frames.

[0111] To facilitate understanding, an example is provided: if the features of a local area (such as the driver's eyes, hands, or head) change significantly in consecutive frames, this usually means that there is dynamic behavior in the area (such as rapid eye movement, hands leaving the steering wheel, or involuntary hand movements). In this embodiment, the multi-spectral attention mechanism can use this temporal change information to assign higher attention weights to these dynamic areas during the feature fusion process, so that the model pays more attention to these important areas that may be related to distracted driving. On the other hand, by analyzing information from different spectra, multi-spectral attention can identify which areas show high-frequency changes (which may correspond to rapid movements) or low-frequency changes (which may correspond to continuous behavior) in the time domain, and then adjust the attention distribution of the corresponding areas. This spectral adaptive adjustment enables the model to more accurately capture subtle changes in driver behavior.

[0112] 205. Each spectral feature map is multiplied by the corresponding attention weight through the fusion module and then fused to obtain a fused feature map.

[0113] Each spectral feature map is multiplied by the corresponding attention weight and then fused. The formula is as follows:

[0114]

[0115] This embodiment dynamically adjusts the weights of different spectral features to enhance the model's ability to focus on key features and improve the accuracy of distracted driving detection.

[0116] 206. Input the fused feature map into the pooling layer and the feature flattening layer for processing to obtain the feature vector of the driving behavior to be identified.

[0117] Perform the maximum pooling operation on the fused feature map, and the formula is expressed as:

[0118]

[0119] in, Represents the pooled feature map, which is used to represent the output after the maximum pooling operation. is the spatial position index of the fused feature map; Represents the fused feature map, which is used to indicate the output after the fusion of each spectrum feature map; g represents the vertical offset of the pooling kernel, and q represents the horizontal offset of the pooling kernel; the pooling feature map is flattened into a one-dimensional vector to obtain the driving behavior feature vector to be identified.

[0120] For example, using a 28×28 pooling kernel and a stride of 1, the value range of g and q is [0, 27].

[0121] The feature map Flattened to a one-dimensional vector , its formula is expressed as:

[0122]

[0123] Assuming that the size of the feature map after pooling is C×28×28, the length of the flattened feature vector is C×28×28.

[0124] In this embodiment, downsampling through the maximum pooling operation can effectively reduce the spatial dimension of the feature map while retaining important spatial information, reducing the computational complexity of subsequent processing, and the downsampled feature map is flattened and converted into a one-dimensional feature vector to meet the input requirements of the subsequent classifier.

[0125] 207. Input the feature vector of the driving behavior to be identified into the classifier to perform distracted driving behavior identification, and obtain a distracted driving identification result.

[0126] In this embodiment, the classifier architecture may include a first batch of normalization layers, a first fully connected layer, a Dropout layer, a second batch of normalization layers, an ELU activation function, and a second fully connected layer.

[0127] Specifically, the driving behavior feature vector to be identified is normalized through the first normalization layer and mapped to the target dimension through the first fully connected layer to obtain the preferred driving behavior feature vector; the preferred driving behavior feature vector is randomly set to zero with a preset probability through the Dropout layer, normalized through the second normalization layer, activated through the ELU activation function, and then input into the second fully connected layer for mapping to obtain the driving distraction recognition result.

[0128] In this embodiment, the flattened feature vector is forward propagated through the classifier to obtain the category probability distribution, the output is converted into a probability distribution through the Softmax function, and the category with the highest probability is selected as the final prediction result.

[0129] The batch normalization layer described above can speed up the training process, stabilize the model, and reduce internal covariate shift.

[0130]

[0131] in, Represents input; Expressed as the mean of the batch; Expressed as the variance of the batch, and is a trainable parameter, A small constant to prevent division by zero.

[0132] The first fully connected layer above uses a linear transformation to map the input feature vector to a lower-dimensional space. The formula is as follows:

[0133]

[0134] in, is the weight matrix, is the bias vector, Represents the feature vector input to the fully connected layer. For example, the standardized feature vector is mapped to a 512-dimensional space.

[0135] The above Dropout layer randomly sets the input units to zero with a certain probability during training to prevent overfitting. The formula is as follows:

[0136]

[0137] Here, x represents the input, and p represents the drop probability. For example, the input unit is randomly set to zero with a probability of 50%, that is, p=0.5, to prevent overfitting and improve the generalization ability of the model.

[0138] The above ELU activation function (Exponential Linear Unit) can enhance nonlinear expression and is defined as follows:

[0139]

[0140] Where x represents the input; α is an adjustable parameter, usually set to 1.0; ELU can enhance the nonlinear expression ability of the model and accelerate training.

[0141] The second fully connected layer maps the feature vector to the final multiple category outputs, and its formula is:

[0142]

[0143] in, represents the first fully connected layer; To represent the second fully connected layer, represents the first batch of normalization layers; To represent the second batch of normalized layers, the weight matrix and bias parameters of the second fully connected layer can be set differently from those of the first fully connected layer. The second fully connected layer maps the feature vector to a final target number J of category outputs, such as 10 categories or another number, without specific limitation.

[0144] The Softmax function converts the output into a probability distribution for easy classification. Its formula is:

[0145]

[0146] in, is the probability of the predicted category; is the category index; is the probability of the mth category currently being considered, that is, the one-hot encoding of the true label; It represents the probability of predicting the class with index j among the total number of classes J.

[0147] In order to further improve the performance of classification and detection, the following mechanisms can also be introduced into the classifier: such as residual connections, attention mechanism, multi-layer perceptron (MLP) and normalization strategy.

[0148] The above residual connections add residual connections between fully connected layers to alleviate the gradient vanishing problem and promote the training of deep networks.

[0149]

[0150] The aforementioned attention mechanism embeds a self-attention mechanism in the classifier, enabling the model to dynamically focus on different parts of the feature vector during the classification process, thereby improving classification accuracy. Its formula is:

[0151]

[0152] Among them, Q, K, and V are query, key, and value matrices respectively. The dimension of the key.

[0153] The above-mentioned Multi-Layer Perceptron (MLP) adds multiple fully connected layers and nonlinear activation functions to the classifier to enhance the expressive power of the model.

[0154]

[0155] The above normalization strategy further stabilizes the training process and prevents overfitting by introducing a strategy combining batch normalization and Dropout.

[0156] Through the above design, the classifier not only has the basic batch normalization, full connection, dropout and activation functions, but also can significantly improve the classification performance and stability of the model through structures such as residual connection, attention mechanism and multi-layer perceptron.

[0157] This embodiment significantly improves the performance of distracted driving detection by introducing a model based on deep separable convolution and multi-spectral attention mechanisms. In particular, it achieves a good balance between efficient real-time performance and high-precision detection. This addresses the shortcomings of traditional distracted driving detection schemes, which rely on manually designed local feature information such as edges and corners, in capturing complex driver behavior characteristics, especially subtle changes in driving behavior. Furthermore, shallow neural networks struggle to deeply understand the deep semantic information in image data. This embodiment employs a two-layer cascaded DSC structure for deep separable convolution. The first-layer DSC module performs channel transformation to reduce computational redundancy and improve expression capabilities. The second-layer DSC performs local feature extraction, enhancing the network's spatial perception capabilities. The deep separable convolution layer of this embodiment has stronger nonlinear expression capabilities while maintaining a low computational load, making it particularly suitable for tasks such as distracted driving detection that require high fine-grained features. By optimizing the object of MSA, performing spectral division on the initial driving behavior feature map, and combining global average pooling, global maximum pooling, and attention mechanisms to weightedly fuse the spectral feature maps, the model can more finely capture the driver's behavioral characteristics in different spectral ranges, especially those small but critical behavioral changes, thereby significantly improving the accuracy and robustness of distracted driving detection. Furthermore, the MSA layer of this embodiment can further combine temporal change information to guide the allocation of attention to multiple spectral feature maps, allowing the model to pay more attention to instantaneous behavior and small behaviors. Finally, the classifier significantly improves the classification performance and stability of the model through batch normalization, full connection, dropout and activation functions, as well as other optimization mechanisms.

[0158] See also Figure 4 The third embodiment of the method for identifying distracted driving in this application describes the training process of the distracted driving identification model of this application, including:

[0159] 401. Build training set and validation set.

[0160] Specifically, the driver's behavior video data or image data is collected through the vehicle-mounted camera, the sample video data or sample image data is preprocessed, and the training set and the validation set are divided according to a preset ratio.

[0161] It is understandable that this embodiment can collect driver behavior sample video data or sample image data through the vehicle-mounted camera to cover videos of different time periods (daytime, nighttime), weather conditions (sunny, rainy), road scenes (urban roads, highways) and individual differences of drivers (age, gender, accessories worn, etc.); and pre-label corresponding labels, including normal driving (holding the steering wheel with both hands), distracted behavior (using mobile phones, eating, turning the head to talk, etc.).

[0162] This preprocessing can include, but is not limited to, extracting frames from the video data to obtain corresponding images. After performing image resizing, random cropping and padding, random horizontal flipping, and normalization on each image, the data can be randomly split into an 8:2 ratio to ensure a balanced distribution of lighting, viewing angles, and distracting behavior categories between the two datasets. The validation set can also include data from extreme scenarios (such as strong glare and severe occlusion) to assess model robustness.

[0163] In a feasible embodiment, the sample image data is preprocessed, including: adjusting each sample image to a preset first target pixel, performing random cropping and padding on the adjusted image, and further performing random horizontal flipping and normalization to obtain sample image data to enhance the generalization ability and robustness of the model.

[0164] For example, the input image data may be scaled to 256×256 pixels. This process uses a bilinear interpolation method, which can smoothly adjust the image size and retain more detail information.

[0165] Exemplarily, the random cropping and padding is performed on a 256×256 image to randomly crop to 224×224 pixels, and a 4-pixel padding is applied during the cropping process to increase the diversity of the image data.

[0166] Exemplarily, the random horizontal flipping performs horizontal flipping on the image with a probability of 50% to simulate different perspective changes of the driver.

[0167] For example, the above normalization process normalizes the pixel values ​​of the image data to the range of [-1, 1], eliminating the influence of illumination changes by converting the pixel values ​​to the same scale range; normalization is performed on each channel (R, G, B channels) separately to avoid deviations between different channels and enhance the model's adaptability to color changes under different lighting environments. The formula is expressed as:

[0168]

[0169] in, Indicates the input The image is standardized, and the input image includes R, G, and B channels. The mean of each of the R, G, and B channels, Indicates the standard deviation of each of the R, G, and B channels.

[0170] For example, you can set , , whose values ​​correspond to the three RGB channels respectively. This embodiment eliminates the influence of illumination changes by converting the RGB channels of the pixel to the same scale range.

[0171] For example, to adapt to changing lighting conditions, this embodiment uses brightness changes and normalization to simulate different lighting environments such as daytime, nighttime, cloudy days, and strong sunlight. This helps the model extract effective features under complex lighting conditions and improves its robustness.

[0172] For example, in order to adapt to changes in camera perspectives, this embodiment introduces operations such as random cropping, random padding, and horizontal flipping to simulate changes in perspective caused by factors such as camera angle offset or vehicle shaking during driving, thereby improving the model's adaptability in multi-angle scenarios.

[0173] For example, considering the various interference factors that may exist during driving, the complexity of the driving environment, such as seat adjustment, occlusion of the sun visor or rearview mirror, and the driver wearing glasses, hats, or masks, all of which can interfere with image features. This embodiment uses data augmentation strategies to effectively increase the model's robustness to these changes.

[0174] In this embodiment, the preprocessing steps are designed to enhance the model's generalization and robustness. Brightness variations and normalization processing help the model perform stable inference under different lighting conditions (such as daytime, nighttime, cloudy days, and strong sunlight). Random cropping, padding, and horizontal flipping simulate different perspective changes, helping the model adapt to the driver's possible camera angle changes while driving. These steps ensure that the model can adapt to different lighting, perspectives, and driving environments. The preprocessing performed at the model input layer is to input unified and standardized candidate image data. The actual detection environment of the model is based on the actual original image data. Preprocessing includes adjusting the image data to a preset size and other normalization procedures.

[0175] 402. Train the initial model using the training set, and optimize the model parameters by backpropagation using a preset loss function to obtain a candidate model.

[0176] Specifically, the training set is input into the initial model for driver distraction detection, generating predicted labels. A preset cross-entropy loss function is used to calculate the loss between the predicted and true labels. The model parameters are then updated using a preset optimization strategy and loss value to generate a candidate model. This embodiment uses repeated iterative training to adjust the model parameters, gradually reducing the cross-entropy loss function and continuously improving detection accuracy and system stability.

[0177] For example, the above loss function can adopt the formula of cross-entropy loss function (Cross-Entropy Loss):

[0178]

[0179] Among them, L is the cross entropy loss function; J is the total number of categories; is the probability of the true label, that is, the probability of the category with the actual classification index o, which can be represented by one-hot encoding. is the predicted probability, that is, the probability of predicting the category with classification index o.

[0180] Exemplarily, the above-mentioned backpropagation can use a gradient descent method, such as Stochastic Gradient Descent (SGD), to update model parameters. Specifically, the gradient of the loss function with respect to the model parameters is calculated by the chain rule, and the model parameters are adjusted according to the gradient information and the learning rate.

[0181] Exemplarily, this embodiment can further introduce optimization strategies, such as momentum optimization and weight decay strategies. Specifically, by accumulating historical gradient information, momentum optimization can reduce the update step size in the steep direction and increase the update step size in the gentle direction. By limiting the size of the model weight, overfitting is prevented, thereby improving the generalization ability of the model. By penalizing larger weight values, weight decay can encourage the model to choose simpler solutions and avoid over-reliance on certain features. In high-dimensional space, weight decay helps maintain the numerical stability of model parameters and avoid numerical problems caused by excessively large weight values.

[0182] Exemplarily, this embodiment may also introduce a cosine annealing learning rate scheduling strategy to dynamically adjust the learning rate, accelerate convergence speed and improve model performance, and the learning rate gradually decreases as the training progresses.

[0183] This embodiment further ensures the training efficiency and convergence performance of the model by adjusting the learning rate through backpropagation based on the cross-entropy loss function and a preset optimization strategy. While maintaining high efficiency and real-time performance, this embodiment achieves more accurate and stable recognition of distracted driving behavior.

[0184] 403. Evaluate the candidate model using the validation set, and use the candidate model corresponding to the best model parameters as the driving distraction recognition model.

[0185] In this embodiment, the validation set can be used to calculate the accuracy, precision, recall rate, F1 score and other indicators of the candidate model for the classification task, and the candidate model corresponding to the best model parameters after the evaluation is confirmed as the driving distraction recognition model.

[0186] 404. Input the original driver image data into a preset driving distraction recognition model and perform preprocessing through the input layer to obtain candidate driver image data;

[0187] 405. Perform feature extraction on the candidate driver image data through a depthwise separable convolutional layer to obtain an initial driving behavior feature map;

[0188] 406. Input the initial driving behavior feature map into the multi-spectral attention layer for spectrum segmentation, and perform an attention fusion operation on the feature maps of each spectrum range to obtain a fused feature map;

[0189] 407. Input the fused feature map into the pooling layer and the feature flattening layer for processing to obtain a feature vector of the driving behavior to be identified;

[0190] 408. Input the feature vector of the driving behavior to be identified into a classifier to perform distracted driving behavior identification and obtain a distracted driving identification result.

[0191] Steps 404-408 may be performed with reference to the above steps 101-105 and will not be repeated here.

[0192] This embodiment pre-processes and enriches sample image data in the training and validation sets to ensure the model's adaptability to varying lighting, viewing angles, and driving environments. Backpropagation through a loss function is then used to improve the efficiency of the training process based on an optimization strategy. The driver distraction recognition model, which has passed the evaluation, is then used for distracted driving recognition, significantly improving the performance of distracted driving detection and achieving a good balance between efficient real-time detection and high-precision detection. This embodiment significantly reduces computational complexity and parameter count through depthwise separable convolution while effectively preserving feature extraction capabilities, thus meeting the dual requirements of real-time performance and accuracy. A multi-spectral attention mechanism adaptively weights the different spectral features of the input image data, enhancing the model's sensitivity to key driving behavior characteristics. This overcomes the shortcomings of traditional methods in capturing transient behavior, effectively capturing these subtle but crucial feature changes and significantly improving the model's accuracy.

[0193] The above describes the method for identifying distracted driving in this application. The following describes the device for identifying distracted driving in this application. Figure 5 In this application, an embodiment of a distracted driving identification device includes:

[0194] A pre-processing unit 501 is used to input the original driver image data into a preset driving distraction recognition model and perform pre-processing through the input layer to obtain candidate driver image data;

[0195] A feature extraction unit 502 is used to extract features from the candidate driver image data through a depthwise separable convolutional layer to obtain an initial driving behavior feature map;

[0196] The frequency division attention unit 503 is used to input the initial driving behavior feature map into the multi-spectral attention layer for spectrum division, and perform attention fusion operation on the feature maps of each spectrum range to obtain a fused feature map;

[0197] The feature processing unit 504 is used to input the fused feature map into the pooling layer and the feature flattening layer for processing to obtain a feature vector of the driving behavior to be identified;

[0198] The identification unit 505 is used to input the feature vector of the driving behavior to be identified into the classifier to perform driving distraction behavior identification and obtain a driving distraction identification result.

[0199] This embodiment significantly improves the performance of distracted driving detection by introducing a model with deep separable convolution and multi-spectral attention mechanisms. In particular, it achieves a good balance between efficient real-time and high-precision detection. This addresses the problem that traditional distracted driving detection solutions rely on manually designed local feature information such as edges and corners, but are insufficient in capturing complex driver behavior characteristics, especially subtle changes in driving behavior. Shallow neural networks have difficulty in deeply understanding the deep semantic information in image data. This application significantly reduces the computational complexity and parameter count through deep separable convolution while effectively retaining feature extraction capabilities, thus meeting the dual requirements of real-time and precision. The multi-spectral attention mechanism can adaptively weight the different spectral features of the input image data, enhancing the model's sensitivity to key driving behavior characteristics and overcoming the traditional method's lack of ability to capture instantaneous behavior. This effectively captures these subtle but crucial feature changes, greatly improving the accuracy of the model.

[0200] See also Figure 6 Another embodiment of the driving distraction identification device in the present application includes:

[0201] A pre-processing unit 501 is used to input the original driver image data into a preset driving distraction recognition model and perform pre-processing through the input layer to obtain candidate driver image data;

[0202] A feature extraction unit 502 is used to extract features from the candidate driver image data through a depthwise separable convolutional layer to obtain an initial driving behavior feature map;

[0203] The frequency division attention unit 503 is used to input the initial driving behavior feature map into the multi-spectral attention layer for spectrum division, and perform attention fusion operation on the feature maps of each spectrum range to obtain a fused feature map;

[0204] The feature processing unit 504 is used to input the fused feature map into the pooling layer and the feature flattening layer for processing to obtain a feature vector of the driving behavior to be identified;

[0205] The identification unit 505 is used to input the feature vector of the driving behavior to be identified into the classifier to perform driving distraction behavior identification and obtain a driving distraction identification result.

[0206] Optionally, the depthwise separable convolution layer includes a first depthwise convolution module and a second depthwise convolution module, the spatial dimension corresponding to the convolution kernel in the first depthwise convolution module is smaller than the spatial dimension corresponding to the convolution kernel in the second depthwise convolution module, and the feature extraction unit 502 is specifically configured to: input the candidate driver image data into the first depthwise convolution module, perform a spatial convolution operation on the input feature map through depthwise convolution, and integrate cross-channel information through point-by-point convolution before inputting the data into the second depthwise convolution module;

[0207] The second deep convolution module performs spatial convolution on the input feature map through depth-wise convolution, and integrates cross-channel information through point-by-point convolution to obtain the initial driving behavior feature map.

[0208] Optionally, the multi-spectral attention layer includes multiple spectrum modules, attention mechanism modules, and fusion modules; the frequency division attention unit 503 includes:

[0209] The frequency division subunit 5031 is used to divide the spectrum according to the corresponding spectrum range through each spectrum module, and extract the global information of each spectrum feature map to obtain the global feature map corresponding to each spectrum range;

[0210] An attention calculation subunit 5032 is used to calculate the attention weight of the global feature map corresponding to each spectrum range through the attention mechanism module;

[0211] The attention fusion subunit 5033 is used to multiply each spectral feature map by the corresponding attention weight through the fusion module and then fuse them to obtain a fused feature map.

[0212] Optionally, the multi-spectral attention layer further includes a temporal change recognition module, and the frequency division attention unit 503 further includes:

[0213] The temporal change subunit 5034 is used to extract inter-frame change features from the initial driving behavior feature graph of consecutive frames through the temporal change recognition layer, obtain temporal change information, and input it into the attention mechanism module.

[0214] Optionally, the attention calculation subunit 5032 is also used for: the attention mechanism module allocates attention weights to the global feature maps corresponding to each spectral range in combination with the temporal change information, wherein the temporal change information is used to indicate feature changes between consecutive frames.

[0215] Optionally, the feature processing unit 504 is specifically configured to perform a maximum pooling operation on the fused feature map to obtain a pooled feature map;

[0216] Flatten the pooled feature map into a one-dimensional vector to obtain the feature vector of the driving behavior to be identified.

[0217] Optionally, the classifier includes a first batch of normalization layers, a first fully connected layer, a Dropout layer, a second batch of normalization layers, an ELU activation function, and a second fully connected layer connected in sequence; the recognition unit 505 is specifically used to normalize the driving behavior feature vector to be identified through the first batch of normalization layers, and map it to the target dimension through the first fully connected layer to obtain the preferred driving behavior feature vector;

[0218] The preferred driving behavior feature vector is randomly set to zero with a preset probability through the Dropout layer, standardized by the second batch normalization layer, activated by the ELU activation function, and input into the second fully connected layer for mapping to obtain the driving distraction recognition result.

[0219] Optionally, the driving distraction recognition device also includes:

[0220] A construction unit 506 is used to construct a training set and a validation set;

[0221] A training unit 507 is used to train the initial model using a training set and perform back propagation optimization on the model parameters using a preset loss function to obtain a candidate model;

[0222] The evaluation unit 508 is configured to evaluate the candidate model using a validation set and use the candidate model corresponding to the best model parameters as the driving distraction recognition model.

[0223] Optionally, the evaluation unit 508 is specifically used to input the training set into the initial model for driving distraction recognition to obtain a predicted label; calculate the loss value between the predicted label and the true label through a preset cross entropy loss function; update the model parameters through a preset optimization strategy and loss value to obtain a candidate model.

[0224] This embodiment pre-processes and enriches sample image data in the training and validation sets to ensure the model's adaptability to varying lighting, viewing angles, and driving environments. Backpropagation is then performed through a loss function, improving the efficiency of the training process based on an optimization strategy. Successful distracted driving recognition models are then used for distracted driving recognition, significantly improving the performance of distracted driving detection. By incorporating deep separable convolution and a multi-spectral attention mechanism into the distracted driving recognition model, the performance of distracted driving detection is significantly improved, particularly by achieving a good balance between efficient, real-time performance and high-precision detection. This addresses the shortcomings of traditional distracted driving recognition schemes, which rely on hand-crafted local features such as edges and corners, in capturing complex driver behavior, especially subtle changes in driving behavior. Furthermore, shallow neural networks struggle to deeply understand the deep semantic information in image data. This embodiment employs a two-layer cascaded DSC structure for deep separable convolution. The first-layer DSC module performs channel transformation to reduce computational redundancy and improve representational power. The second-layer DSC extracts local features, enhancing the network's spatial perception capabilities. The depth-wise separable convolutional layer of this embodiment has stronger nonlinear expression capabilities while still maintaining a low amount of computation, making it particularly suitable for tasks such as distracted driving detection that require high fine-grained features. By optimizing the object of MSA, performing spectral division on the initial driving behavior feature map, and combining global average pooling, global maximum pooling, and attention mechanisms to weightedly fuse the spectral feature maps, the model can more finely capture the driver's behavioral characteristics in different spectral ranges, especially those small but critical behavioral changes, thereby significantly improving the accuracy and robustness of distracted driving detection. Furthermore, the MSA layer of this embodiment can further combine temporal change information to guide the attention allocation of multiple spectral feature maps, allowing the model to pay more attention to instantaneous behavior and small behaviors. Finally, the classifier significantly improves the classification performance and stability of the model through batch normalization, full connection, Dropout and activation functions, as well as other optimization mechanisms.

[0225] above Figure 5 and Figure 6 The driving distraction identification device in this application is described in detail from the perspective of modular functional entities. The driving distraction identification device in this application is described in detail from the perspective of hardware processing.

[0226] See also Figure 7 As shown, the driving distraction identification device includes a processor 700 and a memory 701. The memory 701 stores machine-executable instructions that can be executed by the processor 700. The processor 700 executes the machine-executable instructions to implement the above-mentioned driving distraction identification method.

[0227] further, Figure 7The illustrated driving distraction identification device further includes a bus 702 and a communication interface 703 , and the processor 700 , the communication interface 703 and the memory 701 are connected via the bus 702 .

[0228] The memory 701 may include a high-speed random access memory (RAM) and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. The communication connection between the system network element and at least one other network element is achieved through at least one communication interface 703 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 702 may be an ISA bus, a PCI bus, or an EISA bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0229] The processor 700 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method may be completed by hardware integrated logic circuits or software instructions in the processor 700. The processor 700 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It may implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present disclosure. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present disclosure may be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory 701 , and the processor 700 reads the information in the memory 701 and completes the method steps of the aforementioned embodiment in combination with its hardware.

[0230] The present application also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to execute the steps of the above-mentioned method for identifying distracted driving.

[0231] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0232] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program code.

[0233] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for identifying distracted driving, characterized in that: The method for identifying distracted driving includes: The original driver image data is input into the preset driving distraction recognition model and preprocessed through the input layer to obtain candidate driver image data; Extract features from the candidate driver image data through a depthwise separable convolutional layer to obtain an initial driving behavior feature map; Inputting the initial driving behavior feature map into a multi-spectral attention layer for spectrum segmentation, and performing an attention fusion operation on the feature maps of each spectrum range to obtain a fused feature map, wherein the multi-spectral attention layer includes multiple spectrum modules, an attention mechanism module, and a fusion module; Inputting the fused feature map into the pooling layer and the feature flattening layer for processing to obtain a feature vector of the driving behavior to be identified; Inputting the driving behavior feature vector to be identified into a classifier to perform driving distraction behavior identification, and obtaining a driving distraction identification result; The initial driving behavior feature map is input into the multi-spectral attention layer for spectrum division, and the feature maps of each spectrum range are subjected to attention fusion operation to obtain a fused feature map, including: Dividing the initial driving behavior characteristic graph according to corresponding spectrum ranges through each spectrum module, and extracting global information of each spectrum characteristic graph to obtain a global characteristic graph corresponding to each spectrum range; The attention mechanism module calculates the attention weight of the global feature map corresponding to each spectral range, wherein the attention distribution of the corresponding region is adjusted according to whether each region exhibits high-frequency or low-frequency changes in the time domain to capture subtle changes in driver behavior; The fusion module multiplies each of the spectral feature maps with the corresponding attention weights and then fuses them to obtain a fused feature map.

2. The method for identifying distracted driving according to claim 1, wherein: The depth-wise separable convolution layer includes a first depth-wise convolution module and a second depth-wise convolution module, wherein the spatial dimension corresponding to the convolution kernel in the first depth-wise convolution module is smaller than the spatial dimension corresponding to the convolution kernel in the second depth-wise convolution module; The step of extracting features from the candidate driver image data through a depthwise separable convolutional layer to obtain an initial driving behavior feature map includes: Input the candidate driver image data into the first deep convolution module, perform spatial convolution on the input feature map through depthwise convolution, integrate cross-channel information through point-by-point convolution, and then input it into the second deep convolution module; The second deep convolution module performs spatial convolution on the input feature map through depth convolution, and integrates cross-channel information through point-by-point convolution to obtain the initial driving behavior feature map.

3. The method for identifying distracted driving according to claim 1, wherein: The attention mechanism module further includes: The attention weights of the global feature maps corresponding to each spectral range are assigned in combination with the temporal change information, where the temporal change information is used to indicate the feature changes between consecutive frames.

4. The method for identifying distracted driving according to claim 1 or 3, wherein: The multi-spectral attention layer also includes a temporal change recognition module; the depthwise separable convolution layer outputs initial driving behavior feature maps of consecutive frames, where one frame of candidate driver image corresponds to one frame of initial driving behavior feature map; The method for identifying distracted driving also includes: The initial driving behavior feature map of continuous frames is passed through the temporal change recognition layer to extract the inter-frame change features, obtain the temporal change information, and input it into the attention mechanism module.

5. The method for identifying distracted driving according to claim 1, wherein: The classifier includes a first batch of normalization layers, a first fully connected layer, a Dropout layer, a second batch of normalization layers, an ELU activation function, and a second fully connected layer connected in sequence; The step of inputting the to-be-identified driving behavior feature vector into a classifier to perform distracted driving behavior identification to obtain a distracted driving behavior identification result includes: Normalizing the driving behavior feature vector to be identified through the first normalization layer, and mapping it to the target dimension through the first fully connected layer to obtain a preferred driving behavior feature vector; The preferred driving behavior feature vector is randomly set to zero with a preset probability through the Dropout layer, normalized through the second batch normalization layer, activated through the ELU activation function, and then input into the second fully connected layer for mapping to obtain a driving distraction recognition result.

6. The method for identifying distracted driving according to claim 1, wherein: Before inputting the original driver image data into the preset driving distraction recognition model and performing preprocessing through the input layer to obtain candidate driver image data, the method further includes: Construct training and validation sets; The initial model is trained using the training set, and model parameters are optimized by backpropagation using a preset loss function to obtain a candidate model; The candidate model is evaluated using a validation set, and the candidate model corresponding to the best model parameters is used as a driving distraction recognition model.

7. The method for identifying distracted driving according to claim 6, characterized in that: The initial model is trained using the training set, and model parameters are optimized by backpropagation using a preset loss function to obtain a candidate model, including: Inputting the training set into the initial model to perform driving distraction recognition and obtain a predicted label; Calculate the loss value between the predicted label and the true label using a preset cross entropy loss function; The model parameters are updated using a preset optimization strategy and the loss value to obtain a candidate model.

8. A driving distraction identification device, characterized in that: The driving distraction identification device comprises: a preprocessing unit, configured to input the original driver image data into a preset driving distraction recognition model, perform preprocessing through an input layer, and obtain candidate driver image data; a feature extraction unit, configured to extract features from the candidate driver image data through a depthwise separable convolutional layer to obtain an initial driving behavior feature map; A frequency division attention unit is configured to input the initial driving behavior feature map into a multi-spectral attention layer for spectrum division, and perform an attention fusion operation on the feature maps of each spectrum range to obtain a fused feature map, wherein the multi-spectral attention layer includes multiple spectrum modules, an attention mechanism module, and a fusion module; A feature processing unit, configured to input the fused feature map into a pooling layer and a feature flattening layer for processing to obtain a feature vector of the driving behavior to be identified; an identification unit, configured to input the driving behavior feature vector to be identified into a classifier to perform driving distraction behavior identification and obtain a driving distraction identification result; The frequency division attention unit is specifically used to: Dividing the initial driving behavior characteristic graph according to corresponding spectrum ranges through each spectrum module, and extracting global information of each spectrum characteristic graph to obtain a global characteristic graph corresponding to each spectrum range; The attention mechanism module calculates the attention weight of the global feature map corresponding to each spectral range, wherein the attention distribution of the corresponding region is adjusted according to whether each region exhibits high-frequency or low-frequency changes in the time domain to capture subtle changes in driver behavior; The fusion module multiplies each of the spectral feature maps with the corresponding attention weights and then fuses them to obtain a fused feature map.

9. A driving distraction identification device, characterized in that: The driving distraction identification device includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor calls the instructions in the memory to enable the driving distraction identification device to execute the driving distraction identification method according to any one of claims 1 to 7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are read and executed, the method for identifying distracted driving according to any one of claims 1 to 7 is executed.

Images