Small sample distraction driving behavior recognition method based on integrated learning and comparative learning

Abstract

The invention discloses a small sample distracted driving behavior recognition method based on integrated learning and comparative learning, and belongs to the technical field of intelligent driving assistance, and the method comprises the following steps: 1, fusing a CA module into a local feature extraction network of MobileVIT-XXS, and carrying out the optimization to form MobileVIT-XXS-CA; step 2, enabling the MobileVIT-XXS-CA and the CCNET to be connected in parallel, and forming a dual-branch feature extraction network model CM; step 3, training the dual-branch feature extraction network model CM by using a comparative learning method; and 4, capturing a driver behavior image in real time by a vehicle-mounted camera, inputting the image into the double-branch feature extraction network model CM trained in the step 3 for processing, and identifying the distraction driving behavior. Local details and global context information of the image can be captured at the same time, and the recognition accuracy of distraction driving behaviors is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent driving assistance, and particularly relates to a small-sample distracted driving behavior recognition method based on ensemble learning and contrastive learning. BACKGROUND

[0002] In recent years, with the rapid development of deep learning technology in the field of image recognition and target detection, a series of classic models such as AlexNet, VGG, ResNet and MobileNet series have shown excellent performance and potential in the task of distracted driving behavior detection. However, although these models perform well in theory and experimental environment, they still face many challenges in real-world applications and need to be further optimized and improved. For example, most models such as MobileNet, ResNet and SqueezeNet mainly emphasize local feature extraction, and the feature capturing ability is insufficient. MobileViT has a global attention mechanism of Transformer, but its local attention mechanism cannot accurately locate the region of interest, making it unsuitable for small sample learning. Moreover, these models need to rely on a large amount of labeled data for full training in the training process, but training data is scarce in actual application scenarios, and the generalization ability of these models under small sample conditions is insufficient.

[0003] Specifically, the existing models still have insufficient feature capturing ability in complex driving environments, especially in the coordination mechanism of global and local features, which has not yet reached the ideal effect. In addition, in actual application scenarios, the scarcity of training data further limits the generalization ability of the model, especially under small sample conditions. These problems show that the existing technology still needs to break through key bottlenecks in the process of transformation from theory to practice to meet the high-precision detection needs in complex driving environments. SUMMARY

[0004] In view of the problems of insufficient feature capturing ability of the model and insufficient training data in actual application, the application proposes a small-sample distracted driving behavior recognition method based on ensemble learning and contrastive learning, which skillfully combines the local feature extraction ability of CCNET and the global attention mechanism of improved MobileViT-XXS-CA, and constructs a double-branch feature extraction network (CCNET MobileViT-XXS-CA, CM). The network can capture local details and global context information of the image at the same time, realize the cooperative optimization of local and global features, and thus improve the comprehensive performance of the model.

[0005] The technical scheme of the application is as follows:

[0006] A small-sample distracted driving behavior recognition method based on ensemble learning and contrastive learning includes the following steps:

[0007] Step 1: Integrate the CA module into the local feature extraction network of MobileViT-XXS to optimize it into MobileViT-XXS-CA;

[0008] Step 2: Connect MobileViT-XXS-CA and CCNET in parallel to form a dual-branch feature extraction network model CM;

[0009] Step 3: Use contrastive learning method to train the dual-branch feature extraction network model CM;

[0010] Step 4: The on-board camera captures the driver's behavior image in real time and inputs the image into the dual-branch feature extraction network model CM trained in step 3 for processing to identify distracted driving behavior.

[0011] Furthermore, in step 1, the CA module is integrated into the MobileViT block of MobileViT-XXS to form an improved MobileViT-CA block; and the improved MobileViT-CA block is used to replace the MobileViT block in the original network MobileViT-XXS to obtain MobileViT-XXS-CA;

[0012] The improved MobileViT-CA block consists of six layers connected sequentially, namely:

[0013] The first layer consists of a 3*3 convolutional layer and an inverted residual block connected in sequence, with an output channel of 16 and a stride of 1;

[0014] The second layer consists of three inverted residual blocks connected in series, with an output channel of 24 and a stride of 2;

[0015] The third layer includes the inverted residual block and the MobileViT-CA block connected in sequence, with an output channel of 48 and a stride of 2;

[0016] The fourth layer consists of an inverted residual block and a MobileViT-CA block connected sequentially, with an output channel of 64 and a stride of 2;

[0017] The fifth layer consists of an inverted residual block and a MobileViT-CA block connected sequentially, with an output channel of 80 and a stride of 2;

[0018] The sixth layer is a fully connected layer with 320 output channels.

[0019] Furthermore, in step 2, the working process of the dual-branch feature extraction network model CM is as follows: MobileViT-XXS-CA uses the global attention mechanism of its Transformer architecture to extract global features; CCNET performs local feature extraction; the global feature extraction and local feature extraction are integrated to obtain the final fusion feature;

[0020] Among them, CCNET consists of five convolutional layers, five CA modules and a global pooling layer; each convolutional layer is connected to a CA module, and the last CA module is connected to a global pooling layer; the five convolutional layers use an 11×11 filter with three convolution kernels, a 7×7 filter with three convolution kernels, a 5×5 filter with one convolution kernel, a 3×3 filter with three convolution kernels, and a 1×1 filter with three convolution kernels respectively; finally, the global pooling layer aggregates the feature maps output by the last CA module, and then performs a Softmax operation to achieve multi-classification tasks.

[0021] Furthermore, in step 3, the contrastive learning framework used is SimCLR; the contrastive learning method includes two stages: a contrastive learning pre-training stage and a supervised learning fine-tuning stage. The specific process is as follows:

[0022] Step 3.1: Select from two public datasets, AUC and StateFarm. Select some images from each public dataset as the training set and some images as the test set to construct a small dataset. The number of images in the training set is twice that of the images in the test set.

[0023] Step 3.2: Perform contrastive learning pre-training;

[0024] Step 3.3: Perform supervised learning fine-tuning.

[0025] Furthermore, the specific process of step 3.2 is as follows:

[0026] Step 3.2.1: Randomly select a small number of images from the small dataset as a pre-training set for contrastive learning.

[0027] Step 3.2.2: Perform two types of data augmentation on each image in the pre-training set to generate two different views as positive sample pairs. The data augmentation methods include random cropping, color jittering, and horizontal flipping.

[0028] Step 3.2.3: Feature extraction: Input the positive sample pairs into the backbone network of CCNET MobileViT-XXS-CA without a classifier to extract features, and then map the features to the latent space through the projection head; the projection head is composed of a multi-layer perceptron, including two fully connected layers, connected by a nonlinear activation function; the resulting projected features are used to calculate the contrast loss to achieve model training; the relevant calculation formula is:

[0029] Z i =Z(f(X i ));

[0030] Z j =Z(f(X j ));

[0031] S=Similary(Z i ,Z j );

[0032] Among them, i and j are the index numbers of different samples; X i 、X j is the enhanced view corresponding to different samples; Z i 、Z j is the projection feature corresponding to different samples; Z(·) is the projection head; f(·) is the backbone network ResNet; S is the contrast loss value; Similary(·) is the contrast loss calculation function;

[0033] Step 3.2.4. Optimize CCNET MobileViT-XXS-CA using NT-Xent loss function.

[0034] Step 3.2.5: After contrastive learning optimization, the pre-trained CCNET MobileViT-XXS-CA without classifier is obtained.

[0035] Furthermore, the specific process of step 3.3 is as follows:

[0036] Step 3.3.1. Randomly select a small number of images from the small dataset as the classifier training set for classifier training.

[0037] Step 3.3.2: Add a classifier to the pre-trained CCNET MobileViT-XXS-CA and optimize the classifier using the classifier training set and cross-entropy loss. During training, input each image in the classifier training set into the pre-trained CCNET MobileViT-XXS-CA to obtain the classification results of driver distraction behavior. Set a low learning rate for the backbone network.

[0038] Step 3.3.3, test set performance evaluation is performed; using a small sample data set, the classifier is fine-tuned;

[0039] Step 3.3.4, after the training, the complete CM model fine-tuned by the classifier is obtained, which is used to identify the distraction behavior of the driver.

[0040] The beneficial technical effects brought by the present application are:

[0041] Lightweight and efficient, suitable for embedded deployment: based on MobileViT-XXS design, combined with CA mechanism to enhance local features, and integrated CCNET model, while maintaining global perception ability with only 2.67M parameter amount, meeting the real-time computing needs of embedded devices;

[0042] Complementary fusion of local and global features: the dual-branch feature extraction network model (CCNETMobileViT-XXS-CA) constructed by the present application captures details and overall semantics respectively, and dynamically fuses to improve the recognition robustness of complex scenes.

[0043] Strong robustness, small sample precision significantly improved: combined with a contrast learning framework (such as SimCLR), the unsupervised contrast loss is used to optimize the feature representation ability of the model in a small sample scenario. Contrast learning significantly improves the classification accuracy of the model on the small sample validation set by enhancing the similarity between samples of the same class and widening the distance between samples of different classes, thereby enhancing the robustness of the model in actual complex scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 The flowchart of the small sample distraction driving behavior recognition method based on ensemble learning and contrast learning of the present application.

[0045] Figure 2 The structure diagram of the improved MobileViT-XXS-CA of the present application.

[0046] Figure 3 The structure diagram of the CA module of the present application.

[0047] Figure 4 The flowchart of the contrast learning method of the present application. DETAILED DESCRIPTION

[0048] The present application will be further described in detail below in combination with the drawings and specific embodiments:

[0049] The present invention innovatively proposes a small-sample distracted driving behavior recognition scheme based on ensemble learning and contrastive learning. This scheme uses an ensemble learning strategy to achieve feature integration and splicing with the help of a dual-branch feature extractor, and introduces a dual-branch feature extraction network model based on contrastive learning pre-training (CCNET MobileViT-XXS-CA, CM for short), aiming to significantly improve the detection performance of distracted driving behavior, and accurately overcome the problem of feature capture in complex driving scenarios and the dilemma of lack of training data. By cleverly embedding the contrastive learning optimization framework, the generalization ability of the model in small sample scenarios is greatly enhanced, while achieving coordinated optimization of local and global features, and improving the performance of the model in all aspects. Specifically, in response to the two key problems of poor model feature capture ability and shortage of training data in practical applications, the present invention selects MobileViT-XXS as the feature extractor. The global attention mechanism embedded in the Transformer architecture of this model complements the local feature extraction function of CCNET, and its lightweight parameter design makes it perfectly adapted to the application requirements of embedded devices. Such as Figure 1 As shown, the implementation steps of the present invention are as follows:

[0050] Step 1: Integrate the CA module into the local feature extraction network of MobileViT-XXS to optimize it to form MobileViT-XXS-CA.

[0051] In the local feature extraction stage of MobileViT-XXS (MobileViT-XXS is a lightweight visual model designed based on the Vision Transformer (ViT) architecture), that is, between the n*n convolution and the 1*1 convolution, a CA (Coordinate Attention, coordinate attention mechanism) module is embedded to form the improved MobileViT-XXS-CA, as shown in the following example: Figure 2 As shown in Figure 3, the features from the improved local feature extraction stage are combined with the features from the global feature extraction stage. The CA module is integrated into the MobileViT block to form the improved MobileViT-CA block, which then replaces the MobileViT block in the original network. By incorporating coordinate information, the model can more accurately focus on important local areas, enhancing its attention to spatial information.

[0052] The CA module uses pooling kernels along the vertical and horizontal directions to decompose the global pooling into one-dimensional feature encoding, encoding each channel along the horizontal axis and vertical axis respectively. This method retains the global spatial information while enhancing position sensitivity. In this invention, the CA module is used to improve the MobileViT block in the MobileViT-XXS model so that the local and global features of the block are coordinated to extract more valuable features. The structure of the CA module is as follows: Figure 3 As shown in Figure 2, the global pooling is decomposed into one-dimensional feature encoding, and the pooling kernel is used to encode each channel along the horizontal and vertical axes respectively. The specific working process is:

[0053] First, the input data is processed with residuals; then the processed data is average pooled along the horizontal axis (X-axis) and vertical axis (Y-axis) respectively; the two pooled results are combined and convolved, and then batch normalized and processed with nonlinear activation functions; the processed data is convolved along the horizontal axis (X-axis) and vertical axis (Y-axis) respectively, and the Sigmoid activation function is used after each convolution; finally, the processed results and the residual processed results are weighted to obtain the final output.

[0054] Table 1 shows the parameters and layer configuration of the improved MobileViT-CA block in detail.

[0055] Table 1 Parameters and layer configuration of the improved MobileViT-CA block

[0056]

[0057] Step 2: Build a dual-branch structure and fuse (in parallel) the features of the improved MobileViT-XXS-CA and CCNET feature extractors to form a dual-branch feature extraction network model (CCNET MobileViT-XXS-CA, abbreviated as CM). MobileViT-XXS-CA utilizes the global attention mechanism of its Transformer architecture to achieve global feature extraction; CCNET focuses on local feature extraction, leveraging its advantage in capturing local details. MobileViT-XXS-CA can achieve collaborative optimization of local and global features, effectively improving the overall performance of the model. In the dual-branch structure, a feature fusion strategy (feature splicing) integrates global and local features to form a complementary and optimized feature representation, thereby improving the model's ability to recognize distracted behaviors in complex driving scenarios.

[0058] The CCNET model uses neural architecture search and coordinate attention mechanism, and is a lightweight model designed, and the model structure is as follows, which is composed of five convolutional layers, and the specific structure is shown in Table 2, the first layer uses 11*11 filters (CondConv) with three convolutional kernels, the second layer uses 7*7 filters with three convolutional kernels, the third layer uses 5*5 filters with one convolutional kernel, the fourth layer uses 3*3 filters with three convolutional kernels, and the fifth layer uses 1*1 filters with three convolutional kernels. CA is introduced after each pooling layer, which considers the channel relationship and position information at the same time. This makes the model more sensitive to the discriminative features. Finally, the global pooling layer (FC) is used to aggregate the feature maps, and then the Softmax operation is performed to complete the multi-classification task.

[0059] Table 2 CCNET model structure

[0060]

[0061]

[0062] Step 3, in order to solve the problem of insufficient training data in practical application, a contrast learning method is used for model training, so that the double-branch feature extraction network model CM can learn feature representation in a self-supervised scene, enhance its applicability in small sample training and recognition task, and make it also have good effect in the case of data shortage.

[0063] The contrast learning framework used in the application is SimCLR, and the core idea is to learn effective feature representation through the similarity and difference between samples. Compared with the traditional supervised learning which needs a large amount of labeled data, SimCLR realizes self-feature learning of unlabeled data by using data enhancement. Specifically, the similarity between two views of the same original image generated by different data enhancement strategies is maximized, that is, the positive sample pair, and the similarity between different image views is minimized, that is, the negative sample pair. As shown in Figure 4 The specific process is as follows:

[0064] The contrast learning method of the application mainly includes two stages: contrast learning pre-training stage and supervised learning fine-tuning stage; the specific process is as follows:

[0065] Step 3.1, the data set used in this experiment is a small data set. Because there is no public data set for driver distraction behavior recognition, AUC and StateFarm are selected on the two public data sets, 50 images are selected as the training set on each public data set, and 25 images are selected as the test set, which are the required small data sets.

[0066] Step 3.2, contrast learning pre-training; the specific process is as follows:

[0067] Step 3.2.1, randomly select a small number of images from the small dataset as a pre-training set for pre-training of contrastive learning;

[0068] Step 3.2.2, perform data augmentation on each image in the pre-training set in two ways to generate two different views as positive sample pairs; data augmentation methods include multiple methods such as random cropping, color jittering, horizontal flipping, etc., and each time two methods are randomly selected for data augmentation;

[0069] Step 3.2.3, feature extraction: input the positive sample pairs into the backbone network part of CCNET MobileViT-XXS-CA without a classifier to extract features, and then map the features to the latent space through the projection head; the projection head is composed of multiple layers of perceptron (MLP), usually containing two fully connected layers and connected with a nonlinear activation function. The final projection features are used to calculate the contrastive loss, thereby realizing model training. The relevant calculation formula is:

[0070] Z i =Z(f(X i ));

[0071] Z j =Z(f(X j ));

[0072] S=Similary(Z i ,Z j );

[0073] Where i and j are the index numbers of different samples; X i , X j are the enhanced views corresponding to different samples; Z i , Z j are the projection features corresponding to different samples; Z(·) is the projection head; f(·) is the backbone network ResNet; S is the contrastive loss value; Similary(·) is the contrastive loss calculation function;

[0074] Step 3.2.4, loss optimization; specifically, use the NT-Xent loss function to optimize CCNET MobileViT-XXS-CA, and maximize the similarity of positive samples and minimize the similarity of negative samples to improve the feature discrimination ability.

[0075] Step 3.2.5, after contrastive learning optimization, a pre-trained CCNET MobileViT-XXS-CA network without a classifier is obtained;

[0076] Step 3.3, supervised learning fine-tuning; the specific process is as follows:

[0077] Step 3.3.1. Randomly select a small number of images from the small dataset as the classifier training set for classifier training.

[0078] Step 3.3.2: Add a classifier to the pre-trained CCNET MobileViT-XXS-CA and optimize the classifier using the classifier training set and cross-entropy loss. During training, input each image in the classifier training set into the pre-trained CCNET MobileViT-XXS-CA to obtain the classification results of driver distraction behavior. Set a low learning rate for the backbone network to maintain the adaptability of the pre-trained features.

[0079] Step 3.3.3. Evaluate the performance of the test set; use a small sample data set to tune the classifier; the classifier is tuned mainly through the layer-wise learning rate (set to 0.01 for the classifier and 0.001 for the feature extractor), the cosine annealing learning rate strategy, and the stochastic gradient descent (SGD) optimizer. Combined with data augmentation (such as random cropping) and early stopping to prevent overfitting, the model generalization ability is improved in 300 rounds of training.

[0080] Step 3.3.4: After training, supervised learning fine-tuning can be achieved. After training, a complete CM model with fine-tuned classifier is obtained, which can accurately identify the driver's distracted behavior.

[0081] Step 4: The on-board camera captures images of the driver's behavior in real time and inputs the images into the CM model trained in Step 3 for processing. The model fuses local and global features to identify distracted driving behaviors such as looking down at the phone, taking hands off the steering wheel, or looking away.

[0082] In order to demonstrate the feasibility and superiority of the present invention, the following experiments are given.

[0083] In the public dataset StateFarm, 50 images are selected from each behavior category as the training set, and 25 images are selected as the test set. There are a total of ten behavior categories, each of which contains approximately 2,000 images (20 of which are selected), including: (0) safe driving, (1) texting with the right hand, (2) making a phone call with the right hand, (3) texting with the left hand, (4) making a phone call with the left hand, (5) adjusting the radio, (6) drinking a drink, (7) reaching into the back seat, (8) fixing hair and makeup, and (9) talking to a passenger. The resolution of each image is 640×480 pixels. The AUC small dataset is also the same, with 50 images selected from each behavior category as the training set and 25 images as the test set. Therefore, a total of 500 images are selected from the original 4,331 images in the training set, and a total of 250 images are selected from the original 4,331 images in the test set.

[0084] In the pre-training process, the input images are passed through a contrastive learning framework, and multiple random data augmentation methods (such as random cropping, color jittering, horizontal flipping, Gaussian blur, etc.) are applied to generate two different views, forming a positive sample pair. This increases the data set, which in turn allows the feature extractor to be fully trained.

[0085] During the training process, all images are resized to 256x256x3 using bilinear interpolation before being input into the model. The resized images are randomly cropped to a 224x224 region during training and center-cropped to a 224x224 region during testing. Random cropping helps to reduce the model's over-reliance on certain local regions of the image. It also allows the image to be cropped to any size, increasing the diversity of the data set and enhancing the model's robustness and generalization ability.

[0086] In both learning processes, the number of learning rounds is set to 300, the training batch size is 8, and the Early Stopping algorithm is used. The model is implemented in the Pytorch framework and runs on an NVIDIA GeForce RTX 3060 with 16GB of system memory. The optimizer used is stochastic gradient descent. All learning rates are set using the cosine annealing learning rate strategy, and the learning rate after the pre-training process is set to 0.01. During the training process, the learning rate of the feature extractor is set to 0.001, and the learning rate of the classifier is set to 0.01.

[0087] AUC Experiment: In the AUC experiment, 50 images per class of behavior are used as the training set, and the feature extractor is trained using the contrastive learning framework. Then, using the training set (50 images per class of behavior) and the test set (25 images per class of behavior), the classifier is fine-tuned with the feature extractor, and the final accuracy reaches 88.8%.

[0088] StateFarm Experiment: The invention first uses the StateFarm data set containing 50 images per class of behavior and pre-trains the feature extractor using the contrastive learning method. This pre-training stage aims to enhance the model's feature representation ability and lay the foundation for subsequent training with the classification layer. Then, using the training set (50 images per class of behavior) and the test set (25 images per class of behavior), the classifier is fine-tuned with the feature extractor, and the final accuracy reaches 94.4%.

[0089] The above experiments demonstrate that even with only a few dozen training data in a real-world scenario, the model can still be optimized through contrastive learning to achieve a high accuracy of over 90%, ensuring reliability and robustness in complex driving environments.

[0090] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by technicians in this technical field within the essential scope of the present invention should also fall within the scope of protection of the present invention.

Claims

1. A small sample distracted driving behavior recognition method based on ensemble learning and contrastive learning, characterized by: The steps include: Step 1: Integrate the CA module into the local feature extraction network of MobileViT-XXS to optimize it into MobileViT-XXS-CA; Step 2: Connect MobileViT-XXS-CA and CCNET in parallel to form a dual-branch feature extraction network model CM; Step 3: Use contrastive learning method to train the dual-branch feature extraction network model CM; Step 4: The on-board camera captures the driver's behavior image in real time and inputs the image into the dual-branch feature extraction network model CM trained in step 3 for processing to identify distracted driving behavior.

2. The small sample distracted driving behavior recognition method based on ensemble learning and contrastive learning according to claim 1 is characterized in that: In step 1, the CA module is integrated into the MobileViT block of MobileViT-XXS to form an improved MobileViT-CA block; the improved MobileViT-CA block is then used to replace the MobileViT block in the original network MobileViT-XXS to obtain MobileViT-XXS-CA; The improved MobileViT-CA block consists of six layers connected sequentially, namely: The first layer consists of a 3*3 convolutional layer and an inverted residual block connected in sequence, with an output channel of 16 and a stride of 1; The second layer consists of three inverted residual blocks connected in series, with an output channel of 24 and a stride of 2; The third layer includes the inverted residual block and the MobileViT-CA block connected in sequence, with an output channel of 48 and a stride of 2; The fourth layer consists of an inverted residual block and a MobileViT-CA block connected sequentially, with an output channel of 64 and a stride of 2; The fifth layer consists of an inverted residual block and a MobileViT-CA block connected sequentially, with an output channel of 80 and a stride of 2; The sixth layer is a fully connected layer with 320 output channels.

3. The small sample distracted driving behavior recognition method based on ensemble learning and contrastive learning according to claim 2 is characterized in that: In step 2, the working process of the dual-branch feature extraction network model CM is as follows: MobileViT-XXS-CA uses the global attention mechanism of its Transformer architecture to extract global features; CCNET performs local feature extraction; the global feature extraction and local feature extraction are integrated to obtain the final fusion feature; Among them, CCNET consists of five convolutional layers, five CA modules and a global pooling layer; each convolutional layer is connected to a CA module, and the last CA module is connected to a global pooling layer; the five convolutional layers use an 11×11 filter with three convolution kernels, a 7×7 filter with three convolution kernels, a 5×5 filter with one convolution kernel, a 3×3 filter with three convolution kernels, and a 1×1 filter with three convolution kernels respectively; finally, the global pooling layer aggregates the feature maps output by the last CA module, and then performs a Softmax operation to achieve multi-classification tasks.

4. The small sample distracted driving behavior recognition method based on ensemble learning and contrastive learning according to claim 3 is characterized in that: In step 3, the contrastive learning framework used is SimCLR. The contrastive learning method includes two stages: contrastive learning pre-training stage and supervised learning fine-tuning stage. The specific process is as follows: Step 3.1: Select from two public datasets, AUC and StateFarm, and select some images from each public dataset as training sets and some images as test sets to construct a small dataset; The number of images in the training set is twice that of the images in the test set; Step 3.2: Perform contrastive learning pre-training; Step 3.3: Perform supervised learning fine-tuning.

5. The small sample distracted driving behavior recognition method based on ensemble learning and contrastive learning according to claim 4 is characterized in that: The specific process of step 3.2 is as follows: Step 3.2.1: Randomly select a small number of images from the small dataset as a pre-training set for contrastive learning. Step 3.2.2: Perform two types of data augmentation on each image in the pre-training set to generate two different views as positive sample pairs. The data augmentation methods include random cropping, color jittering, and horizontal flipping. Step 3.2.3, feature extraction: input the positive sample pairs into the backbone network of CCNET MobileViT-XXS-CA without classifier to extract features, and then map the features to the latent space through the projection head; The projection head is composed of a multi-layer perceptron, which contains two fully connected layers connected by a nonlinear activation function. The resulting projection features are used to calculate the contrast loss to achieve model training. The relevant calculation formula is: Z i =Z(f(X i )); Z j =Z(f(X j )); S=Similary(Z i ,Z j ); Among them, i and j are the index numbers of different samples; X i 、X j is the enhanced view corresponding to different samples; Z i , Z j is the projection feature corresponding to different samples; Z(·) is the projection head; f(·) is the backbone network ResNet; S is the contrast loss value; Similary(·) is the contrast loss calculation function; Step 3.2.

4. Optimize CCNET MobileViT-XXS-CA using NT-Xent loss function. Step 3.2.5: After contrastive learning optimization, the pre-trained CCNET MobileViT-XXS-CA without classifier is obtained.

6. The small sample distracted driving behavior recognition method based on ensemble learning and contrastive learning according to claim 5 is characterized in that: The specific process of step 3.3 is as follows: Step 3.3.

1. Randomly select a small number of images from the small dataset as the classifier training set for classifier training. Step 3.3.

2. Add a classifier to the pre-trained CCNET MobileViT-XXS-CA and optimize the classifier using the classifier training set and cross entropy loss. During training, each image in the classifier training set is input into the pre-trained CCNETMobileViT-XXS-CA to obtain the classification results of driver distraction behavior; Set a lower learning rate for the backbone network; Step 3.3.

3. Evaluate the performance of the test set; use a small sample data set to tune the classifier; Step 3.3.4: After training, a complete CM model fine-tuned by the classifier is obtained to identify driver distraction behavior.

Images