Driver bad behavior detection method, system and equipment based on single Beidou vehicle-mounted terminal drive

Abstract

The invention discloses a method, a system and equipment for detecting bad behaviors of a driver based on a single Beidou vehicle-mounted terminal driver, and belongs to the technical field of information. The method comprises the following steps: 1, acquiring video information and navigation information of a driver in a vehicle from a single Beidou vehicle-mounted terminal, and preprocessing the video information and the navigation information; 2, constructing a dynamic attention network based on a spatio-temporal behavior prototype, and performing feature extraction on the video information to obtain spatio-temporal features; 3, performing sample augmentation on the spatio-temporal features to obtain spatio-temporal augmentation features; 4, performing feature extraction on the semantic information to obtain semantic features; 5, fusing the obtained augmented features and semantic features to obtain multi-modal fusion features; and 6, inputting the multi-modal fusion features into the classification network to obtain a driver behavior detection result. According to the method, by introducing a space-time modeling and dynamic attention mechanism, a small sample augmentation mechanism and a multi-modal fusion mechanism, the learning efficiency of the method under the condition of data scarcity is improved, and meanwhile the false alarm rate is reduced.

Description

Technical Field

[0001] This invention relates to the fields of behavior detection and traffic safety, and in particular to a method, system and equipment for detecting driver misbehavior based on a single Beidou vehicle-mounted terminal. Background Technology

[0002] In recent years, with the rapid development of intelligent driving and vehicle-mounted sensing technologies, the detection of driver misbehavior has become an important research direction for ensuring driving safety. By identifying the driver's operational behavior and changes in state within the vehicle in real time, dangerous behaviors can be effectively warned, which is of great significance in reducing the incidence of traffic accidents.

[0003] However, behaviors in driving scenarios are significantly complex and unique. These behaviors often manifest as small-amplitude, short-duration, and subtle movements, such as a brief head tilt or a slight hand movement, lacking obvious motion features. Most current detection methods still primarily analyze single-frame images, focusing on judging the driver's current state from static appearance features. They generally suffer from insufficient modeling of the continuity of actions in spatiotemporal modeling, failing to fully explore the dynamic evolution of behavior over time, and thus struggling to accurately capture the essential characteristics of undesirable driver behaviors in complex and realistic in-vehicle scenarios.

[0004] Meanwhile, most methods rely solely on the collected visual information for analysis, with relatively limited utilization of non-visual information such as vehicle operating status. Vehicle status information such as speed changes does not play its due role, lacks multi-level information interaction, and has insufficient multi-modal collaborative modeling capabilities.

[0005] The collection of samples of poor driving behavior generally suffers from problems such as limited quantity, uneven distribution of categories, and significant individual differences. The same type of behavior may exhibit significant differences in performance among different drivers and under different driving conditions, while different categories may have highly similar action features. This characteristic of "large intra-class differences and small inter-class differences" makes the model prone to overfitting during the training phase, resulting in insufficient generalization ability in real-world applications and making it difficult to meet the reliability and robustness requirements of engineering deployment. Summary of the Invention

[0006] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a method, system and equipment for detecting driver misbehavior based on a single Beidou vehicle terminal, so as to solve the technical problem that the existing technology cannot achieve high accuracy in driver behavior detection.

[0007] To achieve the above objectives, in a first aspect, the present invention provides a method for detecting driver misbehavior based on a single Beidou vehicle-mounted terminal, comprising: The video and navigation information of the driver inside the vehicle are obtained from the single Beidou vehicle terminal. The video and navigation information are preprocessed to obtain visual data and semantic data, respectively. A dynamic attention network based on spatiotemporal behavioral prototypes is constructed, and spatiotemporal features are extracted from visual data through the dynamic attention network to obtain spatiotemporal features; A video feature learning mechanism based on few-sample augmentation is constructed to augment spatiotemporal features with few samples to obtain spatiotemporally augmented features. Semantic features are extracted from semantic data to obtain semantic features; By fusing spatiotemporal augmentation features and semantic features, multimodal fusion features are obtained; The multimodal fusion features are input into the classification network to obtain the driver behavior detection results.

[0008] According to one feasible method in an embodiment of this application, video information and navigation information are preprocessed to obtain visual data and semantic data, respectively, including: The video information is standardized and enhanced to obtain visual data. The standardization process includes unifying resolution, reducing frame rate, and compression. Enhancement operations include random cropping, horizontal flipping, time jittering, and frame order perturbation, which can be specifically represented as: in, For standardized calculations, For video information, , , These represent the target resolution, target frame rate, and target compression parameters, respectively. To enhance operation, , , These represent the transformation functions for random cropping, horizontal flipping, temporal jitter, and frame order perturbation, respectively. The navigation information is discretized and normalized to obtain semantic data. The navigation information includes time, vehicle speed, and acceleration, which can be specifically represented as: in, Discretization represents mapping continuous data to a discrete feature space. For navigation information, These represent time, vehicle speed, and acceleration, respectively. This indicates normalized calculation.

[0009] According to one feasible method in an embodiment of this application, a dynamic attention network based on spatiotemporal behavioral prototypes is constructed. The dynamic attention network is used to extract spatiotemporal features from visual data to obtain spatiotemporal features, including: The X3D-M network structure is improved by removing the classification network and retaining only its 3D convolutional network for spatiotemporal modeling to capture spatiotemporal information. The process of capturing spatiotemporal information can be represented as follows: in This represents the improved X3D-M network used. Represents network parameters, Represents the captured spatiotemporal information, The vector dimension representing spatiotemporal information; In order to better capture spatiotemporal features from limited training data, construct A prototype of spatiotemporal behavior This implicitly represents the distribution of typical driver spatiotemporal behavior in the feature space; based on the spatiotemporal behavior prototype, a dynamic attention network is constructed, and the spatiotemporal information is further refined. To obtain spatiotemporal characteristics : in For normalized exponential functions, Representative includes A column vector of 1s.

[0010] According to one feasible method in the embodiments of this application, a video feature learning mechanism based on few-shot augmentation is constructed, which performs few-shot augmentation on spatiotemporal behavioral features to obtain spatiotemporally augmented features, and then performs few-shot augmentation training, including: By combining sample blending enhancement and cropping enhancement, feature-level sample augmentation is performed on spatiotemporal features to obtain spatiotemporally augmented features. This significantly increases the size of the training samples for video data features; Sample blending enhancement can be specifically represented as: in, and The first i The and the first j The spatiotemporal characteristics of each sample; The mixing coefficients follow a Beta distribution; Clipping blending enhancement can be specifically represented as: in, and These are the spatiotemporal features of the i-th and j-th samples, respectively; Using a binary mask, a bounding box is randomly generated, with a length of [value missing]. , width is , and These represent the length and width of the sample, respectively. The value inside the bounding box is 0, and the value outside the bounding box is 1. is the mixing coefficient, which follows a Beta distribution.

[0011] According to one feasible method in an embodiment of this application, semantic features are extracted from semantic data to obtain semantic features, including: Generate semantic feature embedding table : in The length of the feature embedding table. Representing the One semantic embedding feature; For each semantic data The index in the semantic feature embedding table can be obtained by looking up the table. : in This refers to index lookup operations; By index Obtain semantic features : .

[0012] According to one feasible method in the embodiments of this application, spatiotemporal augmentation features and semantic features are fused to obtain multimodal fusion features, including: Based on the cross-attention mechanism, the spatiotemporal augmented features and the semantic features are fused using parallel branches. To effectively utilize the cross-attention mechanism, the spatiotemporal augmented features and the semantic features of the samples are first subjected to multi-head mapping and stacking operations: in, Represents spatiotemporal augmentation features by The first parameter obtained A mapping vector, Represents semantic features by The first parameter obtained A mapping vector, This represents the operation of stacking multiple vectors of the same dimension to generate a matrix; Then, a cross-attention mechanism is used to assign different weights to different mappings and perform a weighted summation to obtain a fused feature that incorporates cross-modal features: The fused features are then concatenated to obtain the multimodal fused features. : in This represents the concatenation function.

[0013] According to one feasible method in an embodiment of this application, multimodal fusion features are input into a classification network to obtain driver behavior detection results, including: The classification network is configured with two fully connected layers, specifically as follows: in, For the prediction vector, , As weight, , This is the bias value. It is the RuLU activation function; The multimodal fusion features are classified using a classification network to obtain driver behavior detection results.

[0014] Secondly, the present invention provides a driver misbehavior detection system based on a single Beidou vehicle-mounted terminal, comprising: The data preprocessing module is used to obtain video information and navigation information of the driver in the vehicle from the single Beidou vehicle terminal, and preprocess the video information and navigation information to obtain visual data and semantic data respectively. The spatiotemporal feature extraction module is used to build a dynamic attention network based on spatiotemporal behavioral prototypes. The dynamic attention network is used to extract spatiotemporal features from visual data to obtain spatiotemporal features. The few-shot augmentation module is used to construct a video feature learning mechanism based on few-shot enhancement, which performs few-shot augmentation on spatiotemporal features to obtain spatiotemporally augmented features. The semantic feature extraction module is used to extract semantic features from semantic data to obtain semantic features; The feature fusion module is used to fuse spatiotemporal augmented features and semantic features to obtain multimodal fusion features; The recognition module is used to input multimodal fusion features into the classification network to obtain driver behavior detection results.

[0015] Thirdly, the present invention provides a driver misbehavior detection device based on a single Beidou vehicle-mounted terminal, comprising a memory and a processor, wherein the memory is used to store instructions and data, and the processor is used to execute the instructions stored in the memory; wherein the instructions are used to implement a driver misbehavior detection method based on a single Beidou vehicle-mounted terminal.

[0016] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects: This invention designs a dynamic attention network based on spatiotemporal behavioral prototypes. By introducing spatiotemporal behavioral prototypes, it captures subtle driver movements and behavioral evolution processes, enhancing the modeling ability of key behavioral segments and enabling the algorithm to effectively migrate to real-world complex scenarios. Simultaneously, a semantically guided multimodal perception framework is constructed, achieving information complementarity between modalities through a cross-attention mechanism, significantly improving the robustness of the method. Addressing the issue of data scarcity, a hybrid few-sample enhancement mechanism is built to enhance discriminative ability with limited data scale, effectively improving the learning efficiency of the method in data-scarce scenarios. This also improves behavior recognition accuracy, reduces false positive rates, alleviates the burden of manual review, and enhances the practicality, intelligence, and feasibility of the driver behavior analysis system. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the driver misbehavior detection method based on a single Beidou vehicle terminal provided in an embodiment of the present invention. Figure 2 A schematic diagram of a dynamic attention network structure based on spatiotemporal behavioral prototypes provided in an embodiment of the present invention; Figure 3 A schematic diagram of a video feature learning mechanism based on few-shot enhancement provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the feature fusion module structure provided in an embodiment of the present invention; Figure 5 This is a test confusion matrix diagram provided for an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0019] Figure 1 This invention illustrates a method for detecting driver misbehavior based on a single Beidou vehicle-mounted terminal, comprising the following steps: The video information and navigation information of the driver inside the vehicle are obtained from a single Beidou vehicle terminal. The video information and navigation information are preprocessed to obtain visual data and semantic data, respectively. A dynamic attention network based on spatiotemporal behavioral prototypes is constructed, and spatiotemporal features are extracted from the visual data through the dynamic attention network to obtain spatiotemporal features; A video feature learning mechanism based on few-sample augmentation is constructed to perform few-sample augmentation on the spatiotemporal features to obtain spatiotemporally augmented features. Semantic features are extracted from the semantic data to obtain semantic features; The spatiotemporal augmentation features and the semantic features are fused to obtain multimodal fusion features; The multimodal fusion features are input into the classification network to obtain the driver's bad behavior detection results.

[0020] The following is combined with Figure 2 , Figure 3 and Figure 4 The embodiments of the present invention will be further described as follows: Video and navigation information of the driver inside the vehicle is acquired from a single Beidou vehicle terminal. The video and navigation information are preprocessed to obtain visual and semantic data, respectively. The video information is standardized by unifying the video frame resolution to 224×224×3, dividing each video segment into 16 frames based on time segments, and compressing the video to improve efficiency. Next, enhancement operations are performed using the Kornia library to randomly crop, horizontally flip, jitter, and perturb the frame order of the video to improve robustness, resulting in visual data. Navigation information, including time, vehicle speed, and acceleration, is discretized, mapping continuous data to a discrete feature space. The discretized navigation information is then normalized to obtain semantic data.

[0021] Building such Figure 2 The dynamic attention network shown is based on spatiotemporal behavioral prototypes. It extracts spatiotemporal features from visual data, and the steps include: The X3D-M network structure is improved by removing the classification network and retaining only its 3D convolutional network for spatiotemporal modeling to capture spatiotemporal information. The process of capturing spatiotemporal information can be represented as follows: in This represents the improved X3D-M network used. Represents network parameters, Represents the captured spatiotemporal information, The vector dimension representing spatiotemporal information; In order to better capture spatiotemporal features from limited training data, construct A prototype of spatiotemporal behavior This implicitly represents the distribution of typical driver spatiotemporal behavior in the feature space; based on the spatiotemporal behavior prototype, a dynamic attention network is constructed, and the spatiotemporal information is further refined. To obtain spatiotemporal characteristics : in For normalized exponential functions, Representative includes A column vector of 1s. Construct as follows: Figure 3 The video feature learning mechanism based on few-shot augmentation shown here performs few-shot augmentation on spatiotemporal behavioral features to obtain spatiotemporally augmented features, which are then used for few-shot augmentation training. The steps include: By combining sample blending enhancement and cropping enhancement, feature-level sample augmentation is performed on spatiotemporal features to obtain spatiotemporally augmented features. This significantly increases the size of the training samples for video data features; Sample blending enhancement can be specifically represented as: in, and The first i The and the first j The spatiotemporal characteristics of each sample; The mixing coefficients follow a Beta distribution. In this embodiment, Set to 0.5; Clipping blending enhancement can be specifically represented as: in, and These are the spatiotemporal features of the i-th and j-th samples, respectively; Using a binary mask, a bounding box is randomly generated, with a length of [value missing]. , width is , and These represent the length and width of the sample, respectively. The value inside the bounding box is 0, and the value outside the bounding box is 1. The mixing coefficients follow a Beta distribution. In this embodiment, Set it to 0.8.

[0022] Semantic feature extraction is performed on semantic data to obtain semantic features. The steps include: Generate semantic feature embedding table : in The length of the feature embedding table. Representing the Each feature is embedded in the feature; For each semantic data The index in the semantic feature embedding table can be obtained by looking up the table. : in This refers to index lookup operations; By index Obtain semantic features : like Figure 4 As shown, spatiotemporal augmentation features and semantic features are fused based on a cross-attention mechanism to obtain multimodal fusion features. The steps include: Based on the cross-attention mechanism, the spatiotemporal augmented features and the semantic features are fused using parallel branches. To effectively utilize the cross-attention mechanism, the spatiotemporal augmented features and the semantic features of the samples are first subjected to multi-head mapping and stacking operations: in, Represents spatiotemporal augmentation features by The first parameter obtained A mapping vector, Represents semantic features by The first parameter obtained A mapping vector, This represents the operation of stacking multiple vectors of the same dimension to generate a matrix; Then, a cross-attention mechanism is used to assign different weights to different mappings and perform a weighted summation to obtain a fused feature that incorporates cross-modal features: The fused features are then concatenated to obtain the multimodal fused features. : in This represents the concatenation function.

[0023] The multimodal fusion features are then input into the classification network to obtain the driver misbehavior detection results. The classification network is set up with two fully connected layers, specifically as follows: in, For the prediction vector, , As weight, , This is the bias value. It is the RuLU activation function; Finally, the multimodal fusion features are classified based on a classification network to obtain the results of driver misbehavior detection.

[0024] To verify the effectiveness of this invention, an experiment was conducted on the collected data from a single Beidou vehicle terminal. The dataset contains 2806 video segments, each 10 seconds long, with a resolution of 224*224 and a frame rate of 15 FPS. There are seven categories: making a phone call, taking both hands off the steering wheel, using a mobile phone, smoking, driving while fatigued, not wearing a seatbelt, and normal driving. Figure 5 This is the confusion matrix for testing the present invention on a self-built dataset. Rows represent the true categories, and columns represent the predicted categories. The numerical value of each element in the matrix represents the number of samples whose corresponding true category was correctly identified as a certain predicted category. The values ​​on the diagonal represent the number of samples whose behavior was correctly identified, and the values ​​off-diagonal represent the misclassification rates between different behaviors. The results show that the proportion of correctly identified samples is high, and the present invention can effectively improve the accuracy and stability of identifying multiple categories of driver behavior.

[0025] On the other hand, the present invention also provides a driver misbehavior detection system based on a single Beidou vehicle-mounted terminal, comprising: The data preprocessing module is used to obtain video information and navigation information of the driver in the vehicle from the single Beidou vehicle terminal, and preprocess the video information and navigation information to obtain visual data and semantic data respectively. The spatiotemporal feature extraction module is used to build a dynamic attention network based on spatiotemporal behavioral prototypes. The dynamic attention network is used to extract spatiotemporal features from visual data to obtain spatiotemporal features. The few-shot augmentation module is used to construct a video feature learning mechanism based on few-shot enhancement, which performs few-shot augmentation on spatiotemporal features to obtain spatiotemporally augmented features. The semantic feature extraction module is used to extract semantic features from semantic data to obtain semantic features; The feature fusion module is used to fuse spatiotemporal augmented features and semantic features to obtain multimodal fusion features; The recognition module is used to input multimodal fusion features into the classification network to obtain driver behavior detection results.

[0026] On the other hand, the present invention also provides a driver misbehavior detection device, including a memory and a processor, wherein the memory is used to store instructions and data, and the processor is used to execute the instructions stored in the memory; wherein the instructions are used to implement a driver misbehavior detection method based on a single Beidou vehicle terminal.

[0027] In summary, the driver misbehavior detection method of this invention effectively improves the accuracy and efficiency of behavior recognition by introducing a lightweight three-dimensional spatiotemporal convolutional network coupled with an attention mechanism. This mechanism utilizes three-dimensional spatiotemporal convolution to capture spatiotemporal behavioral information and the attention mechanism to capture the driver's potential spatiotemporal features. Furthermore, it constructs a two-layer augmentation mechanism to overcome problems such as data class imbalance and data scarcity, effectively improving the model's spatiotemporal feature learning ability and generalization performance under small sample conditions. Simultaneously, by implementing a cross-attention mechanism and a fusion mechanism, textual information is used as an important supplement to video information and semantic guidance. Based on the video understanding model, it achieves multimodal fusion perception of semantic and visual collaboration, enhancing the method's comprehensive perception and understanding of complex behavioral patterns.

[0028] It is understood that implementing any method or product of this application does not necessarily require achieving all of the advantages described above at the same time.

[0029] The same or similar parts among the above embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments.

Claims

1. A method for detecting driver misbehavior based on a single Beidou vehicle-mounted terminal, characterized in that, The method includes: The video information and navigation information of the driver inside the vehicle are obtained from a single Beidou vehicle terminal. The video information and navigation information are preprocessed to obtain visual data and semantic data, respectively. A dynamic attention network based on spatiotemporal behavioral prototypes is constructed, and spatiotemporal features are extracted from the visual data through the dynamic attention network to obtain spatiotemporal features; A video feature learning mechanism based on few-sample augmentation is constructed to perform few-sample augmentation on the spatiotemporal features to obtain spatiotemporally augmented features. Semantic features are extracted from the semantic data to obtain semantic features; The spatiotemporal augmentation features and the semantic features are fused to obtain multimodal fusion features; The multimodal fusion features are input into the classification network to obtain the driver behavior detection results.

2. The method according to claim 1, characterized in that, The video information and navigation information are preprocessed to obtain visual data and semantic data, respectively, including: The video information is standardized and enhanced to obtain visual data. The standardization process includes unifying resolution, reducing frame rate, and compression; the enhancement operations include random cropping, horizontal flipping, time jittering, and frame order perturbation, specifically as follows: in, For standardized calculations, The video information, , , These represent the target resolution, target frame rate, and target compression parameters, respectively. To enhance operation, , , These represent the transformation functions for random cropping, horizontal flipping, temporal jitter, and frame order perturbation, respectively. The navigation information is discretized and normalized to obtain semantic data. The navigation information includes time, vehicle speed, and acceleration, specifically expressed as follows: in, Discretization represents mapping continuous data to a discrete feature space. The navigation information, These represent time, vehicle speed, and acceleration, respectively. This indicates normalized calculation.

3. The method according to claim 1, characterized in that, A dynamic attention network based on spatiotemporal behavioral prototypes is constructed. The visual data is then processed through this dynamic attention network to extract spatiotemporal features, including: The X3D-M network structure is improved by removing the classification network and retaining only its 3D convolutional network for spatiotemporal modeling to capture spatiotemporal information. The process of capturing spatiotemporal information is represented as follows: in This represents the improved X3D-M network used. Represents network parameters, Represents the captured spatiotemporal information, The vector dimension representing spatiotemporal information; In order to better capture spatiotemporal features from limited training data, construct A prototype of spatiotemporal behavior This implicitly represents the distribution of typical driver spatiotemporal behavior in the feature space; based on the spatiotemporal behavior prototype, a dynamic attention network is constructed, and the spatiotemporal information is further refined. To obtain spatiotemporal characteristics : in For normalized exponential functions, Representative includes A column vector of 1s.

4. The method according to claim 1, characterized in that, A video feature learning mechanism based on few-shot augmentation is constructed, which performs few-shot augmentation on spatiotemporal behavioral features to obtain spatiotemporally augmented features for few-shot augmentation training, including: Under the combined effect of sample blending enhancement and cropping blending enhancement, feature-level sample augmentation is performed on the spatiotemporal features to obtain spatiotemporally augmented features. This significantly increases the size of the training samples for video data features; Sample blending enhancement is specifically represented as follows: in, and The first i The and the first j The spatiotemporal characteristics of each sample; The mixing coefficients follow a Beta distribution; The specific representation of clipping blend enhancement is as follows: in, Using a binary mask, a bounding box is randomly generated, with a length of [value missing]. , width is , and These represent the length and width of the sample, respectively. The value inside the bounding box is 0, and the value outside the bounding box is 1.

5. The method according to claim 1, characterized in that, Semantic features are extracted from the semantic data to obtain semantic features, including: Generate semantic feature embedding table : in The length of the feature embedding table. Representing the One semantic embedding feature; For each semantic data The index in the semantic feature embedding table is obtained by looking up the table. : in This refers to index lookup operations; By index Obtain the semantic features : 。 6. The method according to claim 1, characterized in that, The spatiotemporal augmentation features and the semantic features are fused to obtain multimodal fusion features, including: Based on the cross-attention mechanism, the spatiotemporal augmented features and semantic features are fused using parallel branches. To effectively utilize the cross-attention mechanism, the spatiotemporal augmented features and semantic features of the samples are first subjected to multi-head mapping and stacking operations: in, Represents spatiotemporal augmentation features by The first parameter obtained A mapping vector, Represents semantic features by The first parameter obtained A mapping vector, This represents the operation of stacking multiple vectors of the same dimension to generate a matrix; Then, a cross-attention mechanism is used to assign different weights to different mappings and perform a weighted summation to obtain a fused feature that incorporates cross-modal features: The fused features are then concatenated to obtain the multimodal fused features. : in This represents the concatenation function.

7. The method according to claim 1, characterized in that, The multimodal fusion features are input into a classification network to obtain driver behavior detection results, including: The classification network is configured with two fully connected layers, specifically as follows: in, For the prediction vector, , As weight, , This is the bias value. It is the RuLU activation function; The multimodal fusion features are classified based on the classification network to obtain driver behavior detection results.

8. A driver misbehavior detection system based on a single Beidou vehicle-mounted terminal, characterized in that, The system is used to implement the detection method of claim 1, including: The data preprocessing module is used to acquire video information and navigation information of the driver in the vehicle from a single Beidou vehicle terminal, and preprocess the video information and navigation information to obtain visual data and semantic data respectively. The spatiotemporal feature extraction module is used to build a dynamic attention network based on spatiotemporal behavioral prototypes, and to extract spatiotemporal features from the visual data through the dynamic attention network to obtain spatiotemporal features; The few-shot augmentation module is used to construct a video feature learning mechanism based on few-shot enhancement, which performs few-shot augmentation on spatiotemporal features to obtain spatiotemporally augmented features. The semantic feature extraction module is used to extract semantic features from the semantic data to obtain semantic features; The feature fusion module is used to fuse the spatiotemporal augmented features and the semantic features to obtain multimodal fusion features; The recognition module is used to input the multimodal fusion features into the classification network to obtain the driver behavior detection results.

9. A driver misbehavior detection device based on a single Beidou vehicle-mounted terminal, characterized in that, The driver misbehavior detection device includes a memory and a processor, wherein the memory is used to store instructions and data, and the processor is used to execute the instructions stored in the memory; wherein the instructions are used to implement the driver misbehavior detection method based on a single Beidou vehicle terminal as described in any one of claims 1-7.

Images