A driver distraction detection and recognition method based on wavelet transform

Abstract

The application provides a driver distraction detection and recognition method based on wavelet transform, which comprises the following steps of constructing a data set, designing a driver distraction detection and recognition model based on wavelet transform, constructing a multi-classification network model based on multi-label and training, and model evaluation.Through the technical scheme of the application, firstly, an adaptive wavelet transform module for processing complex driving environment is constructed, the module can effectively reduce noise and enhance the capturing ability of the model to important details and edges, thereby improving the accuracy and robustness of classification.Secondly, an attention module is introduced into the backbone network model, wherein a plug-and-play multi-scale fusion attention module MSA is designed, the module can more effectively capture key features by combining multi-scale convolution, channel attention module and spatial attention module, accurately classify easily confused categories, and significantly improve the accuracy and classification efficiency of the overall model.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and face recognition, and more particularly to a driver distraction detection and recognition method based on wavelet transform. Background Technology

[0002] Distracted driving poses a serious threat to road safety in my country. According to a report by the World Health Organization (WHO), approximately 1.19 million people die in road traffic accidents each year, and road traffic injuries have become the leading cause of death for children and young adults aged 5 to 29. Furthermore, road traffic accidents cause enormous losses to my country's national economy; statistics show that losses due to road traffic accidents account for 3% of the GDP in most countries. In recent years, thanks to the rapid development of computing technology and the widespread deployment of communication mechanisms, autonomous driving technology has been highly anticipated as a means to reduce reliance on human drivers. However, the research and development of autonomous driving computing systems still faces many challenges and unresolved issues; therefore, real-time detection of distracted driving behavior remains crucial.

[0003] To address the issue of driver distraction detection in real-world environments, methods can be broadly categorized into two types: one relies on physiological signal sensors to monitor the driver's heart rate, brain waves, and other physiological signals; the other is camera-based detection, which uses images to capture the driver's facial features and body posture. While physiological signal-based detection is currently the most widely used method, prolonged sensor wear can cause driver discomfort and negatively impact the driving experience. Furthermore, physiological signals exhibit a certain delay in responding to fatigue, sometimes failing to reflect the driver's real-time state.

[0004] Thanks to significant advancements in deep learning, camera-based computer vision methods have made remarkable progress in distracted driving detection. However, many studies have failed to adequately consider the complexity of real-world driving scenarios and the diversity of driver distraction behaviors when designing their models, resulting in insufficient classification accuracy for certain categories of distraction. Furthermore, many existing detection methods employ overly complex deep learning models, making them difficult to deploy on resource-constrained edge devices, which greatly limits their widespread application. Therefore, developing highly accurate distracted driving detection methods suitable for complex environments remains a pressing challenge. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this invention provides a driver distraction detection and identification method based on wavelet transform.

[0006] This invention is achieved through the following technical solution: a driver distraction identification method based on wavelet transform, specifically including the following steps:

[0007] S1. Dataset Construction: First, the dataset is divided into training and testing sets, using the StateFarm public dataset on Kaggle. The original labels of this dataset are Ci (i=0,1,2,3,4,5,6,7,8,9), where C0 represents safe driving, C1 represents texting with the right hand, C2 represents making a phone call with the right hand, C3 represents texting with the left hand, C4 represents making a phone call with the left hand, C5 represents operating the radio, C6 represents reaching behind the hand, C7 represents reaching behind the hand, C8 represents reaching behind the hand, and C9 represents talking to a passenger. Then, data augmentation is performed on the image samples. Finally, the sample images are detected and the frontal face region is cropped, and the image scale is normalized to 224×224.

[0008] S2. Design a driver distraction detection model based on wavelet transform: The model includes an adaptive wavelet transform module, a backbone network, and a classifier module, which will process the pre-processed images... As input to the network model, the image is first denoised by an adaptive wavelet transform module, then preliminarily feature-extracted by a feature fusion module, and finally the input features are adjusted using the gating mechanism of the Sigmoid function to obtain the result. Then, global feature extraction is performed through a backbone network composed of multiple stacked RepViT blocks, with a plug-and-play multi-scale fusion attention module designed in the last RepViT block. The constructed classifier module consists of a strip pooling layer and a linear layer. Finally, the classifier module outputs the model's prediction results for driver behavior classification. ;

[0009] S3. Training the model: Predicting driver behavior by comparing the models. With respect to the driver's actual behavior The optimal model is trained by analyzing the gap between the two values. The cross-entropy loss function is used to construct the model optimization objective, which is mathematically expressed as:

[0010] (1)

[0011] Where N is the size of the training batch. This represents the true class label of the nth sample. This represents the model's predicted probability that the nth sample belongs to the i-th class;

[0012] S4. Model Evaluation: Input the test set into the trained network model in sequence to make predictions and obtain the final prediction results. Use the accuracy, precision, recall, and F1-score metrics to comprehensively evaluate the classification performance.

[0013] S5. Using the model: Input the driver image captured in the real scene into the trained network model to obtain the classification result.

[0014] As a preferred option, step S1 specifically includes the following steps:

[0015] S1-1. Divide the dataset into the original training set and the test set in an 8:2 ratio;

[0016] S1-2. Data augmentation was performed on the image samples. Color jitter, motion blur, Gaussian noise, horizontal flip and random scaling were added to the dataset to simulate complex driving environments under natural conditions. At the same time, masks were added to the faces in the dataset using the dlib library toolkit.

[0017] S1-3. The open-source computer vision library OpenCV was used to uniformly detect and crop out the frontal face region of the sample images, and the image scale was normalized to 224×224.

[0018] As a preferred option, step S2 specifically includes the following steps:

[0019] S2-1. Design an adaptive wavelet transform module, including a wavelet transform module, a feature fusion module, and a gating mechanism based on the Sigmoid function, using Haar wavelets for wavelet discrete transform;

[0020] S2-2. Design the backbone network, which consists of six stacked RepViT blocks, numbered sequentially as follows: In the last RepViT block, a multi-scale fusion attention module (MSA) is introduced, called the RepViTA attention block.

[0021] S2-3. Construct a multi-scale fusion attention module (MSA), including a multi-scale convolution module, a channel attention module, and a spatial attention module;

[0022] S2-4. The classifier module consists of pooling layers and linear layers. Strip pooling is used. The final output of the linear layer is the model's prediction of the driver's behavior. .

[0023] Furthermore, step S2-1 specifically includes the following steps:

[0024] S2-1-1. After the input image I is decomposed by Haar wavelet, different frequency components F1, F2, and F3 are obtained, which can be mathematically represented as follows:

[0025] (2)

[0026] Where Wavelet represents the discrete wavelet transform operation;

[0027] S2-1-2. A 3×3 depthwise separable convolution is used to enhance the frequency components obtained from wavelet decomposition, and an activation function is added to introduce nonlinearity. The resulting feature components are then processed. , , Mathematically, it can be represented as:

[0028] (3)

[0029] in, It is a 3×3 depthwise separable convolution, Wavelet is wavelet transform, ReLU is ReLU activation function, and is a learnable training parameter;

[0030] S2-1-3, The components are trained using set learnable parameters. , , Weighted fusion is performed to obtain feature maps Mathematically represented as:

[0031] (4)

[0032] S2-1-4, Feature Map The feature map is obtained by upsampling after transposing the convolution to restore the input image size. Mathematically, it can be represented as:

[0033] (5)

[0034] Where UpConv is the upsampling convolution operation;

[0035] S2-1-5、 After processing with the Sigmoid activation function, each element of the output is mapped to a value between 0 and 1, generating a weight map; this weight map is then compared with the original image. Element-wise multiplication outputs a feature map. Mathematically, it can be represented as:

[0036] (6)

[0037] Where Sigmoid represents the Sigmoid activation function.

[0038] Furthermore, step S2-2 specifically includes the following steps:

[0039] S2-2-1, RepViT block in backbone network With the first RepViT block The structures are identical, consisting of a 3×3 depthwise separable convolution and two consecutive 1×1 convolutions;

[0040] S2-2-2, the last RepViT block It is called the RepViTA attention block, and its structure is: a 3×3 depthwise separable convolution, a multi-scale fused attention module (MSA), and two consecutive 1×1 convolutional blocks.

[0041] Furthermore, steps S2-3 specifically include the following steps:

[0042] S2-3-1, the multi-scale convolution module consists of three 3×3 depthwise separable dilated convolutions in parallel, with dilation rates set to 1, 2, and 3 respectively. The convolution with a dilation rate of 1 maintains the receptive field of a standard 3×3 convolution, the 3×3 dilated convolution with a dilation rate of 2 expands the receptive field to 5×5, and the 3×3 dilated convolution with a dilation rate of 3 further expands the receptive field to 7×7. The feature map input to the multi-scale fusion attention module is... The shape is C×H×W, and feature maps are generated after passing through a multi-scale convolution module. , Then, by adding element by element, , , Merge into a new feature map , The shape remains C×H×W;

[0043] Mathematical representation:

[0044] (X))(7)

[0045] (X))(8)

[0046] (X))(9)

[0047] (10)

[0048] in , and These represent a standard 3×3 depthwise separable convolution, a 3×3 depthwise separable convolution with a dilation rate of 2, and a 3×3 depthwise separable convolution with a dilation rate of 3, respectively. To keep the feature map size unchanged after convolution, appropriate padding was added to the convolution kernel. Represents the ReLU activation function; This indicates an element-wise addition operation;

[0049] S2-3-2. In the channel attention module, the feature map input to the channel attention module The feature map is first obtained by performing a 1×1 convolution to expand the channels. Its channel count is doubled. Then the feature map... A non-linear factor is introduced through the GELU activation function, and then the number of channels is compressed to 1 through a 1×1 convolution for channel compression; the compressed single-channel feature map and Subtract element-wise, then multiply by a learnable parameter θ as a channel scaling factor to redistribute channel features. The scaled feature map is then compared with... The elements are added one by one, and the final output is the feature map after channel feature adjustment. , The shape is 2C×H×W; mathematically represented as:

[0050] (11)

[0051] (12)

[0052] in, This represents a 1×1 convolution used to expand the channels; This represents a 1×1 convolution used to compress the channels; GELU represents the GELU activation function.

[0053] S2-3-3. In the spatial attention module, the spatial attention module includes a 1×1 convolution and a pooling module. Input feature map First, a 1×1 convolution is used to restore the number of channels from 2C to C, resulting in the feature map. ;Then The feature map is obtained after average pooling and max pooling in the pooling module. and , and The number of channels remains unchanged, and the size is 1×1. The feature map is then... With feature map By adding elements one by one, a fused feature map is obtained. Mathematically represented as:

[0054] (13)

[0055] (14)

[0056] (15)

[0057] (16)

[0058] in, This represents a 1×1 convolution operation in the spatial attention module; Pool represents the average pooling operation. This represents the max pooling operation; This indicates element-wise addition;

[0059] S2-3-4, Fusion Feature Map After applying the Sigmoid activation function, attention weights are obtained. Then attention weights With input feature map Multiply element by element and then combine with the input image. Element-wise addition yields the output feature map X' of the multi-scale fusion attention module (MSA), which can be mathematically represented as:

[0060] (17)

[0061] X'= (18)

[0062] Where Sigmoid represents the Sigmoid activation function; ⊙ indicates element-wise addition; ⊙ indicates element-wise multiplication.

[0063] Furthermore, step S4 specifically includes the following steps:

[0064] S4-1. Input the test set samples into the trained model to obtain 10-class classification results. If the final prediction result is... If the result is positive, it indicates that the model detects the driver is in a safe driving state.

[0065] S4-2, Compare the predicted values ​​of all samples in the test set. and tags Calculate the number of correctly classified items. Mathematically, it can be represented as:

[0066] (19)

[0067] Where N is the total number of samples in the test set, and i represents the i-th sample in the test set;

[0068] S4-3. Calculate the accuracy rate according to the formula to evaluate the model performance. The mathematical representation is as follows:

[0069] (20)

[0070] Where T represents the number of samples correctly classified by the model, and N represents the total number of samples;

[0071] S4-4. Calculate the model's precision, recall, and F1 score according to the formula to evaluate the model's performance. The mathematical representation is as follows:

[0072] (twenty one)

[0073] (twenty two)

[0074] (twenty three)

[0075] Where TP is the number of samples correctly predicted as positive by the model, FP is the number of samples incorrectly predicted as positive by the model, and FN is the number of samples that the model incorrectly predicted as negative when they were actually positive.

[0076] This invention, by employing the above technical solutions, offers the following advantages compared to existing technologies: The invention mainly comprises two parts. First, it constructs a driver distraction detection and recognition model based on wavelet transform. This model can accurately identify driver distraction behaviors and effectively segment them, thus improving the overall accuracy of classification. Second, it incorporates an attention module into the model, specifically a multi-scale fusion attention module (MSA). This module allows the model to capture features at different scales. By analyzing subtle eye movements, facial expression changes, head rotations, and other features at different scales, the model's ability to recognize various distraction behaviors is improved. This enables the model to effectively classify easily confused categories, thereby enhancing the accuracy of each stage of the model and the overall classification efficiency.

[0077] Additional aspects and advantages of the invention will become apparent in the following description or may be learned by practice of the invention. Attached Figure Description

[0078] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0079] Figure 1 This is the overall flowchart of the present invention;

[0080] Figure 2 This is a schematic diagram of the adaptive wavelet transform module;

[0081] Figure 3 A schematic diagram of the Multiscale Fusion Attention Module (MSA);

[0082] Figure 4 This is a schematic diagram of the RepViTA attention block structure;

[0083] Figure 5 This is a schematic diagram of the overall architecture of a driver distraction detection model based on wavelet transform;

[0084] Figure 6 This is a schematic flowchart for the invention. Detailed Implementation

[0085] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0086] Numerous specific details are set forth in the following description to provide a thorough understanding of the invention; however, the invention may also be practiced in other ways different from those described herein. Therefore, the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0087] The following is combined Figures 1 to 6 The wavelet transform-based driver distraction detection and identification method of the present invention will be described in detail in the embodiments of the present invention.

[0088] like Figure 6 As shown, this invention proposes a driver distraction recognition method based on wavelet transform, which specifically includes the following steps:

[0089] S1. Dataset Construction: First, the dataset is divided into training and testing sets, using the StateFarm public dataset from Kaggle. The original labels of this dataset are Ci (i=0,1,2,3,4,5,6,7,8,9), where C0 represents safe driving, C1 represents texting with the right hand, C2 represents making a phone call with the right hand, C3 represents texting with the left hand, C4 represents making a phone call with the left hand, C5 represents operating the radio, C6 represents reaching behind the hand, C7 represents reaching behind the hand, C8 represents reaching behind the hand, and C9 represents talking to a passenger. Then, data augmentation is performed on the image samples. Finally, the sample images are detected and the frontal facial regions are cropped, and the image scale is normalized to 224×224. The specific steps include:

[0090] S1-1. Divide the dataset into the original training set and the test set in an 8:2 ratio;

[0091] S1-2. Data augmentation was performed on the image samples to expand the size of the dataset. Color jitter, motion blur, Gaussian noise, horizontal flip and random scaling were added to the dataset to simulate complex driving environments under natural conditions. In order to accommodate the need for drivers to wear masks, masks were also added to the faces in the dataset. At the same time, the dlib library toolkit was used to add masks to the faces in the dataset.

[0092] S1-3. The open-source computer vision library OpenCV is used to uniformly detect and crop the frontal face region of the sample images, and the image scale is normalized to 224×224.

[0093] S2. Design a driver distraction detection and recognition model based on wavelet transform. The model includes an adaptive wavelet transform module, a backbone network, and a classifier module, such as... Figure 5 As shown; the preprocessed dataset images As input to the network model, the image is first denoised by an adaptive wavelet transform module, then preliminarily feature-extracted by a feature fusion module, and finally the input features are adjusted using the gating mechanism of the Sigmoid function to obtain the result. Then, global feature extraction is performed through a backbone network composed of multiple stacked RepViT blocks. A plug-and-play multi-scale fusion attention module (MSA module) is designed in the last RepViT block (called the RepViTA attention block) to enhance the model's ability to focus on key features. The MSA module aims to improve the model's understanding and classification capabilities for complex scenes, and includes a multi-scale convolution module, a channel attention module, and a spatial attention module. The multi-scale convolution module uses convolution kernels of different sizes to capture features in different receptive fields of the input image. The channel attention module not only enhances the information representation of important channels but also preserves the original features, achieving adaptive channel selectivity. The spatial attention module combines average pooling and max pooling operations to further strengthen the model's ability to capture key features. Finally, a sigmoid-based gating mechanism is used to weight and adjust the fused features. Deeper features are extracted through the backbone network; the constructed classifier module consists of a strip pooling layer and a linear layer; finally, the classifier module outputs the model's prediction results for driver behavior classification. Specifically, it includes the following steps:

[0094] S2-1. Design an adaptive wavelet transform module: including a wavelet transform module, a feature fusion module, and a gating mechanism based on the Sigmoid function; employing Haar wavelets for wavelet discretization; such as... Figure 2 As shown, the specific steps include:

[0095] S2-1-1. The input image I is processed by wavelet decomposition to obtain different frequency components F1, F2, and F3, which can be mathematically represented as follows:

[0096] (1)

[0097] Where Wavelet represents the discrete wavelet transform operation;

[0098] S2-1-2. A 3×3 depthwise separable convolution is used to enhance the frequency components obtained from wavelet decomposition, and an activation function is added to introduce nonlinearity. The resulting feature components are then processed. , , Mathematically, it can be represented as:

[0099] (2)

[0100] in, This is a 3×3 depthwise separable convolution, with ReLU as the ReLU activation function. Since depthwise separable convolutions enhance important features while suppressing useless information, compared to the original components... , , It contains more spatial context information.

[0101] S2-1-3, Components are learned through the set learnable parameters. , , Weighted fusion is performed to obtain feature maps Wavelet transform can be used in the model to effectively extract image features, while reducing data dimensionality and computational complexity, thus significantly improving the efficiency of classification tasks. Mathematically, this can be represented as:

[0102] (3)

[0103] S2-1-4, Feature Map The feature map is obtained by upsampling after transposing the convolution to restore the input image size. Mathematically, it can be represented as:

[0104] (4)

[0105] Where UpConv represents the upsampling convolution operation;

[0106] S2-1-5、 After processing with the Sigmoid activation function, each element of the output is mapped to a value between 0 and 1, generating a weight map. This weight map is then compared with the original image. Element-wise multiplication outputs a feature map. Mathematically, it can be represented as:

[0107] (5)

[0108] Where Sigmoid represents the Sigmoid activation function.

[0109] S2-2. Design the backbone network, which consists of six stacked RepViT blocks, numbered sequentially as follows: In the last RepViT block, a multi-scale fusion attention module (MSA) is introduced, called the RepViTA attention block.

[0110] like Figure 2 As shown, the specific steps include:

[0111] S2-2-1, RepViT block in backbone network With the first RepViT block The structures are identical, consisting of a 3×3 depthwise separable convolution and two consecutive 1×1 convolutions;

[0112] S2-2-2, the last RepViT block This is called a RepViTA attention block, such as Figure 3 Its structure, as shown, consists of: a 3×3 depthwise separable convolution, a multi-scale fusion attention module (MSA), and two consecutive 1×1 convolutional blocks, with feature maps... Deeper features are obtained after extraction through the backbone network.

[0113] S2-3. Construct a multi-scale fusion attention module (MSA), including a multi-scale convolution module, a channel attention module, and a spatial attention module;

[0114] like Figure 4 As shown, the specific steps include:

[0115] S2-3-1, the multi-scale convolution module consists of three 3×3 depthwise separable dilated convolutions in parallel, with dilation rates set to 1, 2, and 3 respectively. The convolution with a dilation rate of 1 maintains the receptive field of a standard 3×3 convolution, the 3×3 dilated convolution with a dilation rate of 2 expands the receptive field to 5×5, and the 3×3 dilated convolution with a dilation rate of 3 further expands the receptive field to 7×7. By using dilated convolutions, the receptive field can be expanded without increasing computational cost or parameters, enhancing the model's ability to capture global context and multi-scale features. The feature map input to the multi-scale fusion attention module is shown. Feature map The shape is C×H×W, and feature maps are generated after passing through a multi-scale convolution module. , Then, by adding element by element, , , Merge into a new feature map , The shape remains C×H×W; its mathematical representation is:

[0116] (X))(6)

[0117] (X))(7)

[0118] (X))(8)

[0119] (9)

[0120] in , and These represent a standard 3×3 depthwise separable convolution, a 3×3 depthwise separable convolution with a dilation rate of 2, and a 3×3 depthwise separable convolution with a dilation rate of 3, respectively. To keep the feature map size unchanged after convolution, appropriate padding was added to the convolution kernel. Represents the ReLU activation function; This indicates element-wise addition;

[0121] S2-3-2. In the channel attention module, the feature map input to the channel attention module The feature map is first obtained by performing a 1×1 convolution to expand the channels. Its channel count is doubled. Then the feature map... A non-linear factor is introduced by using the GELU activation function, and then the number of channels is compressed to 1 by a 1×1 convolution to compress the channels; the resulting single-channel feature map is... Element-wise subtraction is performed, followed by multiplication by a learnable parameter θ, which acts as a channel scaling factor, to redistribute channel features. This learnable factor adaptively adjusts based on feedback from the loss function during model training, thereby enhancing the model's expressive power. The scaled feature map is then compared to... The elements are added one by one, and the final output is the feature map after channel feature adjustment. , The shape is 2C×H×W; mathematically represented as:

[0122] (10)

[0123] (11)

[0124] in, This represents a 1×1 convolution used to expand the channels; This represents a 1×1 convolution used to compress the channels; GELU represents the GELU activation function.

[0125] S2-3-3, the spatial attention module includes a 1×1 convolution and a pooling module. The pooling module consists of average pooling and max pooling in parallel. Average pooling calculates the average value of pixels within the pooling window, while max pooling selects the maximum value within the pooling window as the output, thus preserving the most prominent local features. Input feature map First, a 1×1 convolution is used to restore the number of channels from 2C to C, resulting in the feature map. ;Then The feature map is obtained after average pooling and max pooling in the pooling module. and , and The number of channels remains unchanged, and the size is 1×1. The feature map is then... With feature map By adding elements one by one, a fused feature map is obtained. Mathematically represented as:

[0126] (12)

[0127] (13)

[0128] (14)

[0129] (15)

[0130] in This represents a 1×1 convolution operation in the spatial attention module; Pool represents the average pooling operation. This represents the max pooling operation; This indicates element-wise addition;

[0131] S2-3-4, Fusion Feature Map After applying the Sigmoid activation function, attention weights are obtained. Then, the attention weights are... With input feature map Element-wise multiplication involves weighting each element of the input feature map to highlight important regions. This is then multiplied by the input image. Element-wise addition preserves the basic information of the original image while superimposing attention-adjusted features, thereby further enriching and enhancing the model's expressive power. The final output feature map X' of the Multi-Scale Fusion Attention Module (MSA) is obtained, mathematically represented as:

[0132] (16)

[0133] X'= (17)

[0134] Where Sigmoid represents the Sigmoid activation function; ⊙ indicates element-wise addition; ⊙ indicates element-wise multiplication.

[0135] S2-4. The classifier module consists of pooling layers and linear layers, using strip pooling. The classifier module ultimately outputs the prediction results of the driver's behavior. ;

[0136] S3. Training the model: Predicting driver behavior by comparing the models. With respect to the driver's actual behavior The optimal model is trained by analyzing the gap between the two values. The cross-entropy loss function is used to construct the model optimization objective, which is mathematically expressed as:

[0137] (18)

[0138] Where N is the size of the training batch. This represents the true class label of the nth sample. This represents the model's predicted probability that the nth sample belongs to the i-th class;

[0139] S4. Model Evaluation: The test set is sequentially input into the trained network model for prediction to obtain the final prediction results. The classification performance is comprehensively evaluated using the accuracy, precision, recall, and F1-score metrics. This includes the following steps:

[0140] S4-1. Input the test set samples into the trained model to obtain 10-class classification results. If the final prediction result is... If the result is positive, it indicates that the model detects the driver is in a safe driving state.

[0141] S4-2, Compare the predicted values ​​of all samples in the test set. and tags Calculate the number of correctly classified items. Mathematically, it can be represented as:

[0142] (19)

[0143] Where N is the total number of samples in the test set, and i represents the i-th sample in the test set.

[0144] S4-3. Calculate the accuracy rate according to the formula to evaluate the model performance. The mathematical representation is as follows:

[0145] (20)

[0146] Where T represents the number of samples correctly classified by the model, and N represents the total number of samples.

[0147] S4-4. Calculate the model's precision, recall, and F1 score according to the formula to evaluate the model's performance. The mathematical representation is as follows:

[0148] (twenty one)

[0149] (twenty two)

[0150] (twenty three)

[0151] Where TP is the number of samples correctly predicted as positive by the model, FP is the number of samples incorrectly predicted as positive by the model, and FN is the number of samples that the model incorrectly predicted as negative when they were actually positive.

[0152] S5. Using the model: Input the driver image captured in the real scene into the trained network model to obtain the classification result.

[0153] In the description of this invention, the term "a plurality of" refers to two or more. Unless otherwise explicitly defined, the terms "upper," "lower," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. The terms "connection," "installation," "fixing," etc., should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a direct connection or an indirect connection through an intermediate medium. For those skilled in the art, the specific meaning of the above terms in this invention can be understood according to the specific circumstances.

[0154] In the description of this specification, the terms "one embodiment," "some embodiments," "specific embodiment," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0155] The above are merely preferred embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A driver distraction detection and identification method based on wavelet transform, characterized in that... Specifically, it includes the following steps: S1. Constructing the dataset: First, the dataset is divided into training and testing sets. The StateFarm public dataset on Kaggle is used, and its original label is C. i The values ​​are: i = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, where C0 represents safe driving, C1 represents texting with the right hand, C2 represents making a phone call with the right hand, C3 represents texting with the left hand, C4 represents making a phone call with the left hand, C5 represents operating the radio, C6 to C8 represent reaching behind the hand, and C9 represents talking to a passenger. Then, data augmentation is performed on the image samples. To accommodate the requirement of drivers wearing masks, masks are added to the faces in the dataset. Finally, the sample images are detected and the frontal facial regions are cropped, and the image scale is normalized to 224×224. S2. Design a driver distraction detection and recognition model based on wavelet transform: The model includes an adaptive wavelet transform module, a backbone network, and a classifier module, which will process the pre-processed images... As input to the network model, the image is first denoised by an adaptive wavelet transform module, then preliminarily feature-extracted by a feature fusion module, and finally the input features are adjusted using the gating mechanism of the Sigmoid function to obtain the result. Then, global feature extraction is performed through a backbone network composed of multiple stacked RepViT blocks, with a plug-and-play multi-scale fusion attention module (MSA) designed in the last RepViT block. The constructed classifier module consists of a strip pooling layer and a linear layer. Finally, the classifier module outputs the model's prediction results for driver behavior classification. Specifically, it includes the following steps: S2-1. Design of an adaptive wavelet transform module: including a wavelet transform module, a feature fusion module, and a gating mechanism based on the Sigmoid function; using Haar wavelets for wavelet discretization; specifically including the following steps: S2-1-1. After the input image I is decomposed by Haar wavelet, different frequency components F1, F2, and F3 are obtained, which can be mathematically represented as follows: （1） Where Wavelet represents the discrete wavelet transform operation; S2-1-2. A 3×3 depthwise separable convolution is used to enhance the frequency components obtained from wavelet decomposition, and an activation function is added to introduce nonlinearity. The resulting feature components are then processed. , , Mathematically, it can be represented as: （2） in, It is a 3×3 depth separable convolution, and ReLU is the ReLU activation function; S2-1-3, The components are trained using the set learnable parameters. , , Weighted fusion is performed to obtain feature maps Mathematically represented as: （3） Where λ is a learnable training parameter; S2-1-4, Feature Map The feature map is obtained by upsampling after transposing the convolution to restore the input image size. Mathematically, it can be represented as: （4） Where UpConv is the upsampling convolution operation; S2-1-5、 After processing with the Sigmoid activation function, each element of the output is mapped to a value between 0 and 1, generating a weight map; this weight map is then compared with the original image. Element-wise multiplication outputs feature maps. Mathematically, it can be represented as: （5） Where Sigmoid represents the Sigmoid activation function; S2-2. Design the backbone network, which consists of six stacked RepViT blocks, numbered sequentially as follows: The last RepViT block introduces a multi-scale fusion attention module (MSA), called the RepViTA attention block; specifically, it includes the following steps: S2-2-1, RepViT block in backbone network With the first RepViT block The structures are identical, consisting of a 3×3 depthwise separable convolution and two consecutive 1×1 convolutions; S2-2-2, the last RepViT block It is called the RepViTA attention block, and its structure is: a 3×3 depthwise separable convolution, a multi-scale fused attention module (MSA), and two consecutive 1×1 convolutional blocks; S2-3. Construct the Multi-Scale Fusion Attention Module (MSA), including a multi-scale convolution module, a channel attention module, and a spatial attention module; specifically, this includes the following steps: S2-3-1, The multi-scale convolution module consists of three 3×3 depthwise separable dilated convolutions in parallel, with dilation rates set to 1, 2, and 3 respectively. The convolution with a dilation rate of 1 maintains the receptive field of the standard 3×3 convolution, the 3×3 dilated convolution with a dilation rate of 2 expands the receptive field to 5×5, and the 3×3 dilated convolution with a dilation rate of 3 further expands the receptive field to 7×7. The feature map is input to the multi-scale fusion attention module. Feature map The shape is C×H×W, and feature maps are generated after passing through a multi-scale convolution module. , Then, by adding element by element, , , Merge into a new feature map , The shape remains C×H×W; its mathematical representation is: (X))（6） (X))（7） (X))（8） （9） in , and These represent a standard 3×3 depthwise separable convolution, a 3×3 depthwise separable convolution with a dilation rate of 2, and a 3×3 depthwise separable convolution with a dilation rate of 3, respectively. This indicates element-wise addition; S2-3-2. In the channel attention module, the feature map input to the channel attention module The feature map is first obtained by performing a 1×1 convolution to expand the channels. Its channel count is doubled; subsequently, the feature map... A non-linear factor is introduced through the GELU activation function, and then the number of channels is compressed to 1 through a 1×1 convolution for channel compression; the resulting single-channel feature map is... Subtract element-wise, then multiply by a learnable parameter θ as a channel scaling factor to redistribute channel features. The scaled feature map is then compared with... The summation results in a final output feature map after channel feature adjustment. , The shape is 2C×H×W; mathematically represented as: （10） （11） in This represents a 1×1 convolution with extended channels; represents a 1×1 convolution with compressed channels; GELU represents the GELU activation function; S2-3-3. In the spatial attention module, the spatial attention module includes a 1×1 convolution and a pooling module, with the input feature map... First, a 1×1 convolution is used to restore the number of channels from 2C to C, resulting in the feature map. ;Then The feature map is obtained after average pooling and max pooling in the pooling module. and , and The number of channels remains unchanged, and the size is 1×1. The feature map is then... With feature map By adding elements one by one, a fused feature map is obtained. Mathematically represented as: （12） （13） （14） （15） in This represents a 1×1 convolution operation in the spatial attention module; Pool represents the average pooling operation. This represents the max pooling operation; This indicates element-wise addition; S2-3-4, Fusion Feature Map After applying the sigmoid activation function, attention weights are obtained; then the attention weights are... With input feature map Multiply element by element and then combine with the input image. Element-wise addition yields the output feature map X of the multi-scale fusion attention module (MSA). ' Mathematically, it can be represented as: (16) X'= （17） Where Sigmoid represents the Sigmoid activation function; ⊙ indicates element-wise addition; ⊙ indicates element-wise multiplication. S2-4: The classifier module consists of pooling layers and linear layers. Strip pooling is used, and the final output of the linear layers is the model's prediction of driver behavior. ; S3. Training the Model: Input the training set into the model for training to obtain a trained model. Compare the model's predicted driver behavior with the driver's actual behavior. The optimal model is trained by analyzing the gap between the two, and the model optimization objective is constructed using the cross-entropy loss function. S4. Model Evaluation: Input the test set into the model for prediction and obtain the final prediction results. Use the accuracy, precision, recall, and F1-score metrics to comprehensively evaluate the model's classification performance. S5. Using the model: Input the driver image captured in the real scene into the trained network model to obtain the classification result.

2. The driver distraction detection and identification method based on wavelet transform according to claim 1, characterized in that... Step S1 specifically includes the following steps: S1-1. Divide the dataset into the original training set and the test set in an 8:2 ratio; S1-2. Perform data augmentation on image samples by adding color jitter, motion blur, Gaussian noise, horizontal flip and random scaling to the dataset to simulate complex driving environments under natural conditions. At the same time, use the dlib library toolkit to add masks to the faces in the dataset. S1-3. The open-source computer vision library OpenCV is used to uniformly detect and crop the frontal face region of the sample images, and the image scale is normalized to 224×224.

3. The driver distraction detection and identification method based on wavelet transform according to claim 1, characterized in that... Step S4 specifically includes the following steps: S4-1. Input the test set samples into the trained model to obtain 10-class classification results. If the final prediction result is... If the result is positive, it indicates that the model detects the driver is in a safe driving state. S4-2, Compare the predicted values ​​of all samples in the test set. and tags Calculate the number of correctly classified items. Mathematically, it can be represented as: （18） Where N is the total number of samples in the test set, and i represents the i-th sample in the test set; S4-3. Calculate the accuracy rate according to the formula to evaluate the model performance. The mathematical representation is as follows: （19） Where T represents the number of samples correctly classified by the model, and N represents the total number of samples; S4-4. Calculate the model's precision, recall, and F1 score according to the formula to evaluate the model's performance. The mathematical representation is as follows: （20） （21） （22） Where TP is the number of samples correctly predicted as positive by the model, FP is the number of samples incorrectly predicted as positive by the model, and FN is the number of samples that the model incorrectly predicted as negative when they were actually positive.

Images