A few-shot image recognition method and system for traffic abnormal events

Abstract

The embodiment of the application provides a kind of few-shot image recognition method and system for traffic abnormal event, belong to traffic abnormal event identification technical field.The few-shot image recognition method includes: obtaining the image of traffic scene to be identified;Obtain original patch token feature;Generate traffic scene context token;Obtain query token feature;Based on the pre-constructed traffic abnormal event multi-prototype memory and query token feature, dynamic routing is carried out to determine the candidate prototype;According to candidate prototype, traffic scene context token, query token feature, reconstruction is carried out under traffic scene context constraint to obtain reconstructed token feature;According to the reconstruction error between reconstructed token feature and query token feature, the class score of each candidate abnormal event class corresponding to the traffic scene image to be identified is determined, and the identification result of the traffic scene image to be identified is obtained based on the class score.The few-shot image recognition method can carry out fine-grained identification for traffic abnormal situation under few-shot condition.

Description

Technical Field

[0001] This invention relates to the field of traffic anomaly event recognition technology, and more specifically to a method and system for few-sample image recognition of traffic anomalies. Background Technology

[0002] Intelligent transportation systems typically require real-time identification of abnormal events in road traffic scenarios, such as wrong-way driving, dangerous lane changes, illegal U-turns, sudden crossings of obscured areas, occupation of emergency lanes, abnormal parking, illegal overtaking, motor vehicles encroaching on non-motorized vehicle lanes, abnormal pedestrian crossings, and abnormal intrusions. Among these, sudden crossings of obscured areas can include abnormal events where pedestrians, non-motorized vehicles, motor vehicles, or other traffic participants suddenly enter the target vehicle's path from an area obscured by vehicles, road facilities, or blind spots. These abnormal events are characterized by low frequency of occurrence, difficulty in data collection, high annotation costs, and significant scene variations, resulting in a typically small number of abnormal samples available for training, exhibiting a pronounced long-tail distribution.

[0003] Existing traffic anomaly identification schemes mostly employ convolutional neural networks, ordinary Vision Transformers, or detection-tracking rules for identification. These schemes can achieve certain results when there are sufficient samples and the scene is fixed, but they are prone to overfitting in the identification of anomalies with few samples, and their generalization ability is insufficient for different weather conditions, road structures, camera angles, and traffic flow densities. Especially in long-tail anomaly categories, some anomalies only have a small number of labeled samples. If a large-scale classification network is directly trained, the model tends to memorize the background, perspective, or vehicle appearance from a small number of samples, making it difficult to learn generalizable traffic anomaly discrimination features. Although some schemes can use publicly available pre-trained visual backbone networks as feature extractors, these backbone networks are usually not self-supervised pre-trained based on traffic scene data, making it difficult to fully cover road structures and vehicle relationships in traffic scenarios such as nighttime, rain and fog, tunnels, complex intersections, congested traffic flow, and highways.

[0004] Traffic anomaly classification differs from ordinary static image classification. Many abnormal behaviors are not solely determined by the appearance of the target vehicle itself, but rather by the contextual relationships between the target vehicle and lane lines, road boundaries, surrounding vehicles, pedestrians, or non-motorized vehicles. For example, normal lane changes and dangerous lane changes may appear highly similar to individual vehicles, but their differences lie mainly in the distance between the target vehicle and adjacent vehicles, lane line positions, vehicle direction, and road structural relationships. Similarly, driving against traffic, illegal U-turns, or occupying emergency lanes require consideration of road direction, lane area, and relationships with other traffic participants. If only global image features, bounding box features, or classification labels are used for judgment, local structural information such as lane lines, road boundaries, and surrounding traffic participants is easily weakened, making it difficult for the model to identify fine-grained anomalies caused by changes in spatial relationships.

[0005] Existing few-shot classification methods based on linear classifiers or cosine similarity typically classify based on global image features or single-class prototypes, making it difficult to model local key relationships in traffic scenarios. When inter-class differences are small and intra-class differences are large, a single prototype cannot cover the multiple manifestations of the same abnormal event under different traffic environments such as highways, congestion, nighttime, intersections, and ramps. Furthermore, simple similarity classification usually only determines whether the query image is close to the class center, but cannot further determine whether the candidate class prototype can explain the local structure and contextual relationships in the current traffic scenario at the token level. Therefore, it is prone to confusion for fine-grained categories such as normal lane changes versus dangerous lane changes, and normal parking versus abnormal parking.

[0006] Therefore, a few-sample image recognition method and system for traffic anomalies is needed to address the problems of low recognition accuracy, weak robustness, and complex deployment when there are few samples, small inter-class differences, large intra-class differences, complex traffic context relationships, insufficient pre-trained features for traffic scenarios, and a high risk of false alarms in low-confidence scenarios. Summary of the Invention

[0007] The purpose of this invention is to provide a few-sample image recognition method and system for traffic anomalies. This few-sample image recognition method can perform fine-grained recognition of traffic anomalies under limited sample conditions, thereby solving the above-mentioned problems.

[0008] To achieve the above objectives, embodiments of the present invention provide a few-shot image recognition method for traffic anomaly events, the few-shot image recognition method comprising: Acquire images of the traffic scene to be identified; The traffic scene image to be identified is fed into a pre-trained visual Transformer feature extraction network to obtain the original patch token features. Generate a traffic scene context token based on the relevant regions in the image of the traffic scene to be identified; The original patch token features are processed to obtain enhanced traffic token features. The original patch token features and / or enhanced traffic token features are then aligned or fused with the traffic scenario context token to obtain query token features. Dynamic routing is performed based on a pre-built multi-prototype memory of traffic anomalies and query token features to determine candidate prototypes; Based on the candidate prototype, traffic scenario context token, and query token features, reconstruction is performed under the constraints of the traffic scenario context to obtain the reconstructed token features; The category score of the traffic scene image to be identified is determined based on the reconstruction error between the reconstructed token feature and the query token feature, corresponding to each candidate abnormal event category. The identification result of the traffic scene image to be identified is obtained based on the category score.

[0009] Optionally, the traffic scene image to be identified is fed into a pre-trained visual Transformer feature extraction network to obtain the original patch token features, including: Collect large-scale unlabeled traffic scene data, including traffic images or video frames in urban roads, highways, tunnels, intersections, rainy days, nights, foggy days, and congested scenarios; Perform multi-view data augmentation on the same traffic image or traffic video frame to obtain a global augmented view and a local augmented view; Construct a student visual Transformer and a teacher visual Transformer, where the student visual Transformer updates its parameters through backpropagation, and the teacher visual Transformer updates its parameters through an exponential moving average of the student visual Transformer's parameters. The global augmented view and the local augmented view are fed into the student visual Transformer and the teacher visual Transformer, and pre-trained based on the DINO self-supervised alignment loss between the predicted distribution output by the student visual Transformer and the target distribution output by the teacher visual Transformer. After pre-training, the student visual Transformer and projection head are discarded, and the Transformer Encoder in the teacher visual Transformer is retained as the frozen visual Transformer feature extraction network after pre-training.

[0010] Optionally, the traffic scene image to be identified is fed into a pre-trained visual Transformer feature extraction network to obtain the original patch token features, including: The traffic scene image to be identified is divided into multiple image blocks; Linear embedding and position encoding are performed on the image blocks to obtain the input token sequence; The input token sequence is fed into the visual Transformer feature extraction network; The classification labels output by the visual Transformer feature extraction network are removed, and the patch tokens corresponding to each image patch are retained as the original patch token features.

[0011] Optionally, the relevant regions include road structure regions, lane line regions, target vehicle regions, and surrounding traffic participant regions. A traffic scene context token is generated based on the relevant regions in the traffic scene image to be identified, including: The traffic scene image to be identified is subjected to at least one of the following: target detection, semantic segmentation, instance segmentation, region detection, region segmentation, attention hotspot extraction, visual Transformer activation region extraction, or preset region prior matching, in order to confirm the relevant regions. The patch tokens corresponding to the relevant areas are pooled, aggregated, or linearly mapped to obtain road structure tokens, lane line tokens, target vehicle tokens, and surrounding traffic participant tokens. A traffic scenario context token is generated based on the road structure token, lane line token, target vehicle token, and surrounding traffic participant tokens, as well as the location relationship encoding between relevant areas.

[0012] Optionally, the original patch token features are processed to obtain enhanced traffic token features, and the original patch token features and / or enhanced traffic token features are aligned or fused with the traffic scenario context token to obtain query token features, including: Global statistics are performed on the original patch token features and traffic scenario context tokens at the channel dimension, and channel attention weights are obtained through fully connected layers and activation functions; Based on the channel attention weights, the original patch token features and the traffic scenario context token are channel-weighted to obtain channel-enhanced token features; The original patch token features are restored into a two-dimensional token feature map according to the spatial location of the image patch, and spatial attention weights are obtained through convolutional layers and activation functions; Based on the spatial attention weights, the tokens in the original patch token features that are related to the target vehicle, lane lines, road boundaries and surrounding traffic participants are spatially weighted to obtain spatially enhanced token features. The channel enhancement token features and spatial enhancement token features are linearly mapped, residually connected, or fused into enhanced traffic token features using a Transformer layer; Perform linear projection, spatial broadcasting, or cross-attention injection on traffic scene context tokens to obtain context-aligned token features that are aligned with the original patch token features in terms of token quantity or feature dimension. The fusion ratio of the original patch token features, context-aligned token features, and / or enhanced traffic token features is determined based on learnable weights or gating coefficients. The original patch token features, context-aligned token features, and / or enhanced traffic token features are weighted and fused according to the fusion ratio to obtain the query token features, or The query token feature is obtained by concatenating the original patch token feature, context-aligned token feature, and / or enhanced traffic token feature through linear mapping.

[0013] Optionally, the traffic anomaly event multi-prototype memory includes multiple traffic anomaly event categories, each category corresponding to M learnable prototypes, and each learnable prototype includes... There are several prototype tokens, each with the same dimensions as the query token's features, where M and... All are positive integers, where M represents the number of learnable prototypes for each traffic anomaly event category. Used to represent the number of prototype tokens contained in each learnable prototype. Traffic abnormal event categories include at least one of the following: driving against the flow of traffic, dangerous lane change, illegal U-turn, sudden crossing of obstructed areas, occupation of emergency lane, abnormal parking, illegal overtaking, motor vehicle intrusion into non-motor vehicle lane, and abnormal crossing or intrusion of pedestrians. The acquisition of learnable prototypes includes: Obtain a small number of training images corresponding to each traffic anomaly event category; The visual Transformer feature extraction network, traffic scene context tokens, and training token features corresponding to few-sample training images are obtained through pre-trained visual Transformer feature extraction network, traffic scene context tokens, and feature enhancement, alignment or fusion processes. Pool the training token features corresponding to each few-sample training image of the same traffic anomaly event category to obtain the sample-level description vector of the few-sample training image; Clustering or grouping a small number of training images of the same traffic anomaly event category based on the sample-level description vectors, thereby obtaining multiple sample groups corresponding to multiple traffic scene performance patterns. For each sample group, the training token features are averaged, weighted averaged, attention pooled, or key token selected according to the token position to obtain the token-level initial prototype corresponding to the sample group. The token-level initial prototype corresponding to the sample group is used as a learnable prototype, and the learnable prototype is updated during the training process.

[0014] Optionally, dynamic routing is performed based on a pre-built multi-prototype memory of traffic anomaly events and query token features to determine candidate prototypes, including: Calculate the visual matching score between the query token features and the learnable prototypes of each traffic abnormality event category in the multi-prototype memory of traffic abnormality events; Calculate the context consistency score based on the consistency between the traffic scenario context token and the learnable prototype corresponding to each traffic anomaly event category in terms of traffic area relationships, location relationships, or prior traffic rules. The routing score for each traffic anomaly event category is calculated based on the visual matching score and the contextual consistency score. The traffic anomaly events are sorted according to the route scores. Select the top-k traffic anomaly event categories from the ranking as candidate anomaly event categories; The learnable prototypes corresponding to the candidate abnormal event categories are used as candidate prototypes for subsequent token-level reconstruction.

[0015] Optionally, based on the candidate prototype, the traffic scenario context token, and the query token features, reconstruction is performed under the constraints of the traffic scenario context to obtain reconstructed token features, including: Use the query token feature as the query vector; Use the prototype token of the candidate prototype and the traffic scene context token as reconstruction conditions; Based on the correspondence between the prototype token of the candidate prototype, the traffic scenario context token, and the query token features, reconstructed token features are generated.

[0016] Optionally, the category score of the traffic scene image to be identified corresponding to each candidate abnormal event category is determined based on the reconstruction error between the reconstructed token features and the query token features, and the identification result of the traffic scene image to be identified is obtained based on the category score, including: Calculate the reconstruction error between the reconstructed token feature and the query token feature for each candidate prototype under each candidate anomaly event category; Based on the consistency between the road structure, lane lines, target vehicle, surrounding traffic participants, or traffic rule priors in the traffic scenario context token and the features of the reconstructed token, the traffic relationship error is calculated. The reconstruction errors and traffic relationship errors corresponding to multiple candidate prototypes under the same candidate anomaly event category are aggregated to obtain the anomaly event category reconstruction error. The anomaly category score is determined based on the reconstruction error of the anomaly category. The category with the highest abnormal event category probability, i.e., the abnormal event category score, is determined as the traffic abnormal event identification result. If the maximum probability of an abnormal event category is less than a preset probability threshold, or if the probability of an abnormal event category for each candidate abnormal event category is greater than a preset error threshold, the traffic scene image to be identified is determined to be either a normal traffic state or an unknown abnormal state.

[0017] On the other hand, the present invention can also provide a few-shot image recognition system for traffic anomaly events, the few-shot image recognition system comprising: The image acquisition module is used to acquire images of the traffic scene to be identified. The feature extraction module is used to feed the traffic scene image to be identified into the pre-trained visual Transformer feature extraction network to obtain the original patch token features. The traffic context generation module is used to generate a traffic scene context token based on the traffic areas and traffic participants in the traffic scene image to be identified, combined with at least one of the inter-area relationship or traffic rule priors. The feature processing module is used to process the original patch token features to obtain enhanced traffic token features, and to align or fuse the original patch token features and / or enhanced traffic token features with the traffic scenario context token to obtain query token features. The dynamic routing module is used to perform dynamic routing based on a pre-built multi-prototype memory of traffic anomalies and query token features to determine candidate prototypes; The prototype reconstruction module is used to reconstruct the prototype based on the candidate prototype, the traffic scenario context token, and the query token features under the constraints of the traffic scenario context, so as to obtain the reconstructed token features. The abnormal event recognition module is used to determine the category score of the traffic scene image to be recognized corresponding to each candidate abnormal event category based on the reconstruction error between the reconstructed token features and the query token features, and to obtain the recognition result of the traffic scene image to be recognized based on the category score.

[0018] Through the above technical solution, the present invention provides a few-shot image recognition method and system for traffic anomaly events. This method acquires an image of a traffic scene to be identified, and then feeds this image into a pre-trained visual Transformer feature extraction network to extract original patch token features. This visual Transformer feature extraction network can be a self-supervised pre-trained and frozen visual Transformer feature extraction network based on large-scale unlabeled traffic scene data. A traffic scene context token can be generated based on relevant regions in the image of the traffic scene to be identified. Feature processing can be performed on the original patch token features to obtain enhanced traffic token features. The original patch token features and / or enhanced traffic token features can be aligned or fused with the traffic scene context token to obtain query token features. Dynamic routing is performed based on a pre-built multi-prototype memory for traffic anomaly events and the query token features. This allows for the determination of candidate prototypes corresponding to candidate anomaly event categories from the multi-prototype memory for traffic anomaly events. The multi-prototype memory for traffic anomaly events can include multiple learnable prototypes corresponding to each traffic anomaly event category. Based on candidate prototypes, traffic scene context tokens, and query token features, the query token features can be reconstructed under the constraints of the traffic scene context to obtain reconstructed token features. These reconstructed token features can be used to characterize the interpretation results of the query token features by the prototypes corresponding to the candidate abnormal event categories. The reconstruction error between the reconstructed token features and the query token features can be used to determine the category score of the traffic scene image to be identified for each candidate abnormal event category. The recognition result of the traffic scene image to be identified, i.e., the recognition result of the traffic abnormal events in the traffic scene image to be identified, can be obtained based on the category score. This few-shot image recognition method can perform fine-grained recognition of traffic abnormalities under few-shot conditions.

[0019] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0020] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a few-sample image recognition method for traffic anomaly events according to an embodiment of the present invention; Figure 2 This is a flowchart of a pre-trained visual Transformer feature extraction network for a few-shot image recognition method for traffic anomalies according to an embodiment of the present invention. Figure 3 This is a flowchart illustrating the acquisition of original patch token features in a few-sample image recognition method for traffic anomalies according to an embodiment of the present invention. Figure 4 This is a flowchart illustrating the generation of traffic scene context tokens in a few-sample image recognition method for traffic anomalies according to an embodiment of the present invention. Detailed Implementation

[0021] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0022] In the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, they do not mean that the solution has been or necessarily adopted.

[0023] In this invention, a patch refers to an image block obtained by dividing a traffic scene image; a token refers to a vectorized representation of an image block, traffic region, prior traffic rules, temporal relationship, or prototype obtained through embedding, aggregation, mapping, or learning; and a prototype refers to a learnable vector or learnable token sequence used to represent a typical visual pattern or contextual pattern among traffic anomaly event categories, normal traffic state categories, or unknown anomaly categories. Unless otherwise specified, patch tokens, traffic scene context tokens, prototype tokens, query tokens, and reconstruction tokens can all have the same or be aligned through linear mapping of feature dimensions. Specifically, the original patch token is mainly used to preserve the local visual structure in the traffic scene image, such as vehicle appearance, heading direction, wheel position, lane lines, road boundaries, and occluded areas; the traffic scene context token is mainly used to express the spatial relationships between road structure, lane lines, target vehicles, surrounding traffic participants, and traffic rule constraints; the enhanced traffic token is mainly used to strengthen the channel dimension and spatial region related to anomaly event discrimination; the query token is mainly used to represent the objects to be interpreted in the dynamic prototype routing and prototype reconstruction stages of the current traffic scene image to be identified; the prototype token is mainly used to represent multiple sub-patterns under a certain traffic anomaly event category; and the reconstruction token is mainly used to represent the interpretation result of the candidate category prototype on the query token under the traffic scene context constraints. By distinguishing the technical roles of the tokens at the above different stages, the traffic anomaly event recognition process can simultaneously utilize local visual differences, traffic rule context, and category prototype knowledge, rather than relying solely on global image classification features.

[0024] Figure 1 This is a flowchart of a few-sample image recognition method for traffic anomaly events according to an embodiment of the present invention. In this invention, the image recognition process may include: In step S1, an image of the traffic scene to be identified is acquired.

[0025] In step S2, the traffic scene image to be identified is fed into the pre-trained visual Transformer feature extraction network to obtain the original patch token features.

[0026] In step S3, a traffic scene context token is generated based on the relevant regions in the traffic scene image to be identified.

[0027] In step S4, the original patch token features are processed to obtain enhanced traffic token features, and the original patch token features and / or enhanced traffic token features are aligned or fused with the traffic scenario context token to obtain query token features.

[0028] In step S5, dynamic routing is performed based on a pre-built multi-prototype memory of traffic anomalies and query token features to determine candidate prototypes.

[0029] In step S6, reconstruction is performed under traffic scenario context constraints based on candidate prototypes, traffic scenario context tokens, and query token features to obtain reconstructed token features.

[0030] In step S7, the category score of the traffic scene image to be identified corresponding to each candidate abnormal event category is determined based on the reconstruction error between the reconstructed token feature and the query token feature, and the identification result of the traffic scene image to be identified is obtained based on the category score.

[0031] In this invention, during image recognition, an image of the traffic scene to be recognized can be acquired. This image can then be fed into a pre-trained visual Transformer feature extraction network to extract the original patch token features. This visual Transformer feature extraction network can be a self-supervised pre-trained and frozen visual Transformer feature extraction network based on large-scale unlabeled traffic scene data. The relevant regions include: road structure region, lane line region, target vehicle region, and surrounding traffic participant region. A traffic scene context token is generated based on the road structure region, lane line region, target vehicle region, and surrounding traffic participant region in the image of the traffic scene to be recognized. Feature processing can be performed on the original patch token features to obtain enhanced traffic token features. The original patch token features and / or enhanced traffic token features can be aligned or fused with the traffic scene context token to obtain query token features. Dynamic routing is performed based on a pre-built multi-prototype memory of traffic anomaly events and query token features. This allows for the determination of candidate prototypes corresponding to candidate anomaly event categories from the multi-prototype memory, which can include multiple learnable prototypes for each traffic anomaly event category. Based on the candidate prototypes, traffic scene context tokens, and query token features, the query token features can be reconstructed under traffic scene context constraints to obtain reconstructed token features. These reconstructed token features can be used to characterize the interpretation of the query token features by the prototypes corresponding to the candidate anomaly event categories. The reconstruction error between the reconstructed token features and the query token features determines the category score of the traffic scene image to be identified for each candidate anomaly event category. The recognition result of the traffic scene image, i.e., the recognition result of the traffic anomaly event in the traffic scene image, can be obtained based on the category score. This few-shot image recognition method can perform fine-grained recognition of traffic anomalies even with few samples.

[0032] In this invention, such as Figure 2 As shown, the process of pre-training a visual Transformer feature extraction network can include: In step S8, large-scale unlabeled traffic scene data is collected, including traffic images or video frames in urban roads, highways, tunnels, intersections, rainy days, nighttime, foggy days, and congested scenarios.

[0033] In step S9, multi-view data augmentation is performed on the same traffic image or traffic video frame to obtain a global augmented view and a local augmented view.

[0034] In step S10, a student visual Transformer and a teacher visual Transformer are constructed, wherein the student visual Transformer updates its parameters through backpropagation, and the teacher visual Transformer updates its parameters through an exponential moving average of the student visual Transformer parameters.

[0035] In step S11, the global augmented view and the local augmented view are fed into the student visual Transformer and the teacher visual Transformer, and pre-training is performed based on the DINO self-supervised alignment loss between the predicted distribution output by the student visual Transformer and the target distribution output by the teacher visual Transformer.

[0036] In step S12, after pre-training is completed, the student visual Transformer and projection head are discarded, and the Transformer Encoder in the teacher visual Transformer is retained as the frozen visual Transformer feature extraction network after pre-training.

[0037] In this invention, the DINO teacher visual transformer can be used for pre-training of the visual transformer feature extraction network. DINO refers to a label-free self-supervised visual representation learning method based on a Student-Teacher architecture, which learns visual representations by aligning the output distribution of the student visual transformer with the target distribution of the teacher visual transformer. This pre-training process can serve as a pre-stage for downstream fine-grained recognition models of few-sample traffic anomalies, used to obtain a stable traffic visual feature extraction network. During the pre-training stage, large-scale unlabeled traffic scene images or traffic video frames can be acquired. These unlabeled traffic scenes can cover urban roads, highways, tunnels, intersections, nighttime, rainy, foggy, congested scenarios, and complex road structure scenarios. Since this stage does not rely on anomaly event category labels, it can fully utilize the large number of existing unlabeled video frames and traffic monitoring images in the intelligent transportation system. After acquiring traffic images or traffic video frames, multi-view data augmentation can be performed on the same traffic image or traffic video frame to obtain a global augmented view and an image-based augmented view. This augmentation can include a global view and a local view. Multi-view data augmentation can include at least one of random cropping, scaling, color perturbation, Gaussian blurring, grayscale conversion, horizontal flipping, or occlusion augmentation. Multi-view augmentation allows pre-trained models to learn vehicle structure, road texture, lane relationships, the appearance of traffic participants, and complex weather and lighting variations, rather than simply memorizing the appearance of a single image. During the pre-training phase, a student visual Transformer and a teacher visual Transformer can be constructed. The student and teacher visual Transformers can have the same or similar network structures, both including patch embedding layers, positional encoding layers, a Transformer Encoder, and a projection head. The student visual Transformer receives multiple augmented views and updates its parameters through backpropagation. The teacher visual Transformer can receive a global augmented view or a target augmented view; its parameters are not updated through backpropagation but rather through an exponential moving average of the student visual Transformer parameters. The global and local augmented views can be fed into the student visual Transformer and the teacher visual Transformer, respectively. Pre-training can then be performed based on the DINO self-supervised alignment loss between the predicted distribution output by the student visual Transformer and the target distribution output by the teacher visual Transformer. Here, the student visual Transformer outputs the predicted distribution, and the teacher visual Transformer outputs the target distribution.Centering, sharpening, or temperature scaling can be applied to the output of the teacher's visual Transformer to stabilize the self-supervised pre-training process and reduce the risk of feature collapse. Specifically, the output of the student's visual Transformer can be subjected to a temperature coefficient. Normalization yields the predicted distribution The output of the teacher's vision Transformer can be processed by centralizing the parameter c and the temperature coefficient. Normalization yields the target distribution The DINO self-supervised alignment loss can include at least one of the distribution alignment loss at the classification label level and the distribution alignment loss at the patch token level. The DINO self-supervised alignment loss can be calculated according to formula (1): Formula (1) in, This indicates the DINO self-supervised alignment loss. Indicates the output distribution dimension. This represents the target probability value generated by the teacher's visual Transformer for the i-th output dimension. This represents the predicted probability value generated by the student visual Transformer for the i-th output dimension. This represents the predicted distribution of the student vision Transformer output, which can be derived from the student vision Transformer output. Normalized exponential function get; This represents the target distribution output by the teacher's visual Transformer, which can be obtained from the teacher's visual Transformer output. Normalized exponential function Obtain, among which and Let represent the temperature coefficient, and c represent the centering parameter. This loss allows the representation of the student visual Transformer to be aligned with the stable representation of the teacher visual Transformer.

[0038] The parameters of the teacher's visual Transformer can be updated using an exponential moving average according to formula (2): Formula (2) in, Indicates the first Teacher vision Transformer parameters in the next iteration Indicates the first Teacher vision Transformer parameters in the next iteration Indicates the first The student visual Transformer parameters for the next iteration. This represents the momentum coefficient. The momentum coefficient can be set to a range from 0.996 to close to 1, or gradually increased during training to improve the stability of the teacher's visual Transformer output representation.

[0039] After pre-training, the student visual Transformer and projector head can be discarded, retaining only the Transformer Encoder from the teacher visual Transformer. This Encoder is then used as the frozen feature extraction network in the downstream few-shot traffic anomaly recognition model—the visual Transformer feature extraction network. Since traffic monitoring images often suffer from nighttime noise, rain and fog blurring, abrupt changes in tunnel illumination, congestion occlusion, changes in target scale at intersections, and motion blur in high-speed scenes, using traffic scene data for DINO self-supervised pre-training allows the feature extraction network to obtain road structure and traffic participant relationship representations that are more closely aligned with the traffic anomaly recognition task than pre-training with general natural images. Therefore, subsequent application phases primarily update the learnable parameters for feature enhancement, alignment, or fusion processes, the traffic anomaly multi-prototype memory, the dynamic prototype routing module, and the prototype reconstruction module, without updating the main parameters of the teacher visual Transformer.

[0040] In the pre-training phase of this invention, unlabeled pre-training data is constructed for the traffic scene itself, and multi-view enhancement enables the teacher visual Transformer to learn road structure, lane relationships, vehicle appearance, and complex weather lighting changes. In the application phase, this frozen feature is combined with traffic rule prior tokens, a multi-prototype memory, dynamic prototype routing, and token-level reconstruction error classification. These features are not simply used in parallel, but rather enable candidate abnormal event prototypes to interpret the query token structure under traffic context constraints, thereby unifying few-sample generalization, fine-grained identification, and low-confidence rejection into the same reconstruction error classification framework. Specifically, the traffic scene context token not only participates in the enhanced fusion of query tokens but also serves as one of the sources of traffic relationship constraints in the prototype reconstruction process, ensuring that the category prototype's interpretation of the query image simultaneously satisfies both local visual structure and traffic rule context. Dynamic prototype routing is not directly used as the final classifier but rather as a candidate screening mechanism to reduce the entry of irrelevant category prototypes into the reconstruction stage and lower the risk of missing true abnormal categories. The final classification is based on the reconstruction error of the candidate prototypes on the query token and the consistency of traffic relationships, thus expressing the technical judgment process of "whether a certain traffic anomaly category can explain the current traffic scene." Through this process, the present invention differs from schemes that simply input traffic images into a pre-trained network and then connect a linear classification head, as well as schemes that perform few-sample classification based solely on support set mean prototypes or cosine similarity.

[0041] In one embodiment of the present invention, such as Figure 3 As shown, the process of obtaining the original patch token characteristics may include: In step S13, the traffic scene image to be identified, either a traffic image or a traffic video frame, is divided into multiple image blocks.

[0042] In step S14, the traffic image or traffic video frame image block is linearly embedded and superimposed with position encoding to obtain the input token sequence.

[0043] In step S15, the traffic image or traffic video frame input token sequence is input into the traffic image or traffic video frame visual Transformer feature extraction network.

[0044] In step S16, the classification labels output by the visual Transformer feature extraction network for traffic images or traffic video frames are removed, and the patch tokens corresponding to each image block are retained as the original patch token features.

[0045] In this invention, the traffic scene image to be identified can be a traffic scene image captured by roadside cameras, traffic monitoring cameras, or other image acquisition devices and uploaded to a traffic cloud platform, traffic management center server, or intelligent traffic analysis platform. The traffic scene image to be identified can be a single frame or a collection of multiple frames. The independent technical solution of this invention does not require the input of T consecutive video frames.

[0046] When the traffic scene image to be identified includes multiple frames, a temporal relationship token can be generated based on at least one of the following: patch token, frame order information, inter-frame time interval, adjacent frame token difference, target vehicle displacement, trajectory of surrounding traffic participants, lane lateral displacement change, target speed change, target orientation change, or optical flow features. This temporal relationship token, along with the traffic scene context token, is then input into the prototype reconstruction process. The temporal relationship token can be used to express lateral trajectory changes during dangerous lane changes, the target appearance process during sudden crossing of an occluded area, vehicle orientation changes during illegal U-turns, speed decreases during abnormal parking, or abnormal movement direction during wrong-way driving. Multiple frames of traffic scene images can be continuous video frames or a non-continuous set of images sampled at time intervals. When only a single frame of traffic scene image is input, a temporal relationship token may not be generated, or it can be set as a zero vector, a learnable default token, or a token estimated from the motion prior of a single frame. Therefore, multi-frame information, as preferred information for enhancing behavior recognition capabilities, does not constitute a necessary limitation on continuous T-frame video frames for an independent technical solution.

[0047] Before dividing the traffic scene image to be identified into multiple image blocks, preprocessing such as size normalization, pixel normalization, region cropping, perspective correction, noise reduction, low-light enhancement, or weather degradation repair can be performed on the traffic scene image to be identified. This reduces the impact of different camera angles, nighttime lighting, rainy / foggy weather, or differences in road background on abnormal event recognition. Then, the traffic scene image to be identified can be patch partitioned according to a preset image block size to obtain multiple image blocks. Each image block is then mapped to a corresponding token vector through a linear projection layer, and position encoding is superimposed to preserve the spatial position information of the image blocks, thereby obtaining the input token sequence.

[0048] After obtaining the input token sequence, the input token sequences corresponding to multiple image patches can be fed into a visual Transformer feature extraction network that has been pre-trained and frozen based on large-scale unlabeled traffic scene data under self-supervised conditions to obtain the original patch token features. The feature extraction network can preferably be the Transformer Encoder from the teacher visual Transformer retained after DINO pre-training, or it can be any other visual Transformer network pre-trained through traffic scene self-supervised learning that can output patch tokens. Since this network can learn general visual knowledge such as road structure, vehicle appearance, lane relationships, and motion patterns from large-scale unlabeled traffic scene images, it can provide stable visual representations in the identification of few-sample traffic anomalies.

[0049] The visual Transformer feature extraction network, pre-trained and frozen based on large-scale unlabeled traffic scene data through self-supervised learning, can include multiple Transformer Encoder Blocks. After processing the input token sequence through multiple layers of self-attention and feedforward networks, it can output classification labels and patch tokens. Since this embodiment of the invention focuses on local behavioral differences and contextual relationships within the traffic scene, the classification labels can be removed, retaining only the patch tokens to form the original patch token features.

[0050] In this invention, the road structure area may include road boundaries, intersection areas, ramp areas, emergency lane areas, non-motorized vehicle lane areas, or pedestrian crossing areas; the surrounding traffic participant area may include the area where surrounding vehicles, pedestrians, non-motorized vehicles, or other road targets are located.

[0051] In one embodiment of the present invention, such as Figure 4 As shown, the process of generating a traffic scenario context token may include: In step S17, at least one of the following is performed on the traffic image or traffic video frame to be identified: target detection, semantic segmentation, instance segmentation, region detection, region segmentation, attention hotspot extraction, visual Transformer activation region extraction, or preset region prior matching, in order to confirm the relevant region.

[0052] In step S18, the patch tokens corresponding to the relevant areas are subjected to regional pooling, regional aggregation, or linear mapping to obtain road structure tokens, lane line tokens, target vehicle tokens, and surrounding traffic participant tokens.

[0053] In step S19, a traffic scenario context token is generated based on the road structure token, lane line token, target vehicle token, and surrounding traffic participant tokens, as well as the location relationship encoding between relevant areas.

[0054] In this invention, the road structure region, lane line region, target vehicle region, and surrounding traffic participant region can be determined through object detection, lane line detection, road region segmentation, and traffic participant detection. Alternatively, at least one of the following methods can be performed on the traffic scene image to be identified: object detection, semantic segmentation, instance segmentation, region detection, region segmentation, attention hotspot extraction, visual Transformer activation region extraction, or preset region prior matching, thereby confirming the road structure region, lane line region, target vehicle region, and surrounding traffic participant region.

[0055] First, obtain the region mask or bounding box of the aforementioned regions in the image. Then, determine the set of patch tokens corresponding to each region based on the overlap relationship between the region mask or bounding box and the patch grid. For regions spanning multiple patches, the corresponding patch tokens can be weighted according to the overlap area or detection confidence. Then, perform region pooling or region aggregation on the patch tokens corresponding to each region to obtain road structure tokens, lane line tokens, target vehicle tokens, and surrounding traffic participant tokens. Positional relationship encoding can be generated based on the relative distance between the target vehicle and the lane line, the positional relationship between the target vehicle and the road boundary, and the relative direction or distance relationship between the target vehicle and surrounding traffic participants, thus generating a traffic scene context token. Furthermore, traffic rule prior tokens can be generated based on lane direction, road boundary, emergency lane, non-motorized vehicle lane, stop line, pedestrian crossing, prohibited area, or road topology. These traffic rule prior tokens can be generated based on the visual recognition results of visible lane lines, road boundaries, stop lines, pedestrian crossings, traffic markings, or the target vehicle's driving direction in the traffic scene image to be identified. For example, the traffic rule prior token can indicate whether the target vehicle is in a permitted passage area, whether it crosses a lane line, whether it enters an emergency lane or non-motorized vehicle lane, and whether it travels in the opposite direction to the permitted direction. After obtaining the above token, the traffic scenario context token can be obtained according to formula (3): Formula (3) in, Represents the road structure token. Indicates lane line token. Indicates the target vehicle token. This represents a token used by surrounding traffic participants. Indicates the positional relationship encoding. This represents a priori token for traffic rules. This represents linear mapping, attention fusion, or multilayer perceptron mapping. Therefore, traffic scene context tokens not only represent the visual content of each traffic area itself, but also the structural relationships and rule constraints required for traffic anomaly event discrimination.

[0056] In one embodiment of the present invention, the process of obtaining query token characteristics may include: In step S20, the original patch token features and traffic scenario context token are globally statistically analyzed in the channel dimension, and the channel attention weights are obtained through a fully connected layer and activation function.

[0057] In step S21, the original patch token features and the traffic scene context token are channel-weighted according to the channel attention weight to obtain channel-enhanced token features.

[0058] In step S22, the original patch token features are restored into a two-dimensional token feature map according to the spatial location of the image patch, and spatial attention weights are obtained through convolutional layers and activation functions.

[0059] In step S23, the tokens related to the target vehicle, lane lines, road boundaries and surrounding traffic participants in the original patch token features are spatially weighted according to the spatial attention weight to obtain spatially enhanced token features.

[0060] In step S24, the channel enhancement token features and spatial enhancement token features are linearly mapped, residually connected, or fused into enhanced traffic token features using a Transformer layer.

[0061] In step S25, the traffic scene context token is processed by linear projection, spatial broadcasting or cross-attention injection to obtain context-aligned token features that are aligned with the original patch token features in terms of token quantity or feature dimension.

[0062] In step S26, the fusion ratio of the original patch token features, context-aligned token features, and / or enhanced traffic token features is determined based on learnable weights or gating coefficients.

[0063] In step S27, the original patch token features, context-aligned token features, and / or enhanced traffic token features are weighted and fused according to the fusion ratio to obtain the query token features, or The query token feature is obtained by concatenating the original patch token feature, context-aligned token feature, and / or enhanced traffic token feature through linear mapping.

[0064] In this invention, feature enhancement of the original patch token features can include channel attention branch and spatial attention branch enhancement, or at least one of region attention, relational attention, or lightweight Transformer enhancement structures. The channel attention branch is used to highlight feature channels related to abnormal behavior semantics, such as vehicle heading, vehicle posture, lane crossing status, and traffic participant relationships; the spatial attention branch is used to highlight key local regions such as wheel positions, lane edges, road boundaries, target vehicles, and surrounding traffic participants. During enhancement, global statistics can be performed on the original patch token features and traffic scene context tokens in the channel dimension, and channel attention weights can be obtained through fully connected layers and activation functions. Based on these channel attention weights, the original patch token features and traffic scene context tokens can be channel-weighted to obtain channel-enhanced token features. The original patch token features can be restored to a two-dimensional token feature map according to the spatial location of image patches, and spatial attention weights can be obtained through convolutional layers and activation functions. Based on these spatial attention weights, tokens related to target vehicles, lane lines, road boundaries, and surrounding traffic participants in the original patch token features can be spatially weighted to obtain spatially enhanced token features.

[0065] Specifically, the original patch token features can be concatenated or fused with the traffic scene context token to obtain intermediate token features. Global average pooling and global max pooling are then applied to the intermediate token features along the token dimension, followed by a fully connected layer and a sigmoid activation function to obtain channel attention weights. Next, the original patch token features are restored to a two-dimensional token feature map according to the patch spatial location, and then processed through a convolutional layer and a sigmoid activation function to obtain spatial attention weights. The channel enhancement and spatial enhancement processes can be represented by formulas (3a) and (3b): Formula (3a) Formula (3b) in, This indicates the channel-enhanced token feature. Representation space augmented token features, Indicates channel attention weights. Represents spatial attention weights, and ⊙ represents element-wise multiplication. This means restoring the original patch token features into a two-dimensional token feature map according to the patch grid. This represents the characteristics of the original patch token. Finally, it can be... and Features are fused into enhanced traffic token features through linear mapping, residual connections, or Transformer layers.

[0066] When obtaining query token features, the original patch token features, traffic scenario context token, and enhanced traffic token features can be fused together.

[0067] Since the number of traffic scene context tokens can be less than the number of patch tokens, before fusion, the traffic scene context tokens can be linearly projected, spatially broadcast, injected across attention, or otherwise aligned to obtain context-aligned token features that correspond to the original patch token features in terms of token quantity or feature dimension. For example, each traffic scene context token can be projected to the same dimension as the patch token, and then broadcast to the corresponding patch location based on its corresponding road structure region, lane line region, target vehicle region, or surrounding traffic participant region. For traffic rule prior tokens without a clear region affiliation, they can be broadcast to the global patch location or injected into all patch tokens through cross-attention. Learnable weights or gating coefficients can be used to control the fusion ratio of the original patch token features, context-aligned token features, and enhanced traffic token features.

[0068] The query token characteristics can be obtained according to formula (4): Formula (4) in, This indicates a query for token characteristics. This represents the characteristics of the original patch token. This indicates the context-aligned token feature. This indicates enhanced traffic token characteristics. and , Indicates corresponding to respectively , , The learnable weight coefficients, and This represents the i-th weight among the three weights mentioned above. In other words, the original patch token feature, the context-aligned token feature, and / or the enhanced traffic token feature are weighted and fused according to the fusion ratio to obtain the query token feature.

[0069] As another alternative implementation, it is also possible to... , and The concatenated data is then input into a linear mapping layer or a lightweight Transformer layer to obtain query token features. This approach simultaneously preserves the stable visual representation of the self-supervised pre-trained teacher visual Transformer, traffic context relationships, and fine-grained behavioral information enhanced by attention. Furthermore, query token features can serve not only as classifier input but also as the interpreted object in subsequent dynamic prototype routing and prototype reconstruction processes. When a target vehicle slightly crosses the line, the distance between adjacent vehicles changes, a pedestrian appears from an occluded area, or a vehicle enters a prohibited area, these local differences can be preserved in the corresponding query token and participate in anomaly event discrimination during subsequent interactions with traffic scene context tokens and candidate prototype tokens.

[0070] In one embodiment of the present invention, the multi-prototype memory for traffic anomalies may include multiple traffic anomaly categories, each category corresponding to M learnable prototypes, and each learnable prototype including... There are several prototype tokens, each with the same dimensions as the query token's features, where M and... All are positive integers, where M represents the number of learnable prototypes for each traffic anomaly event category. This represents the number of prototype tokens contained in each learnable prototype. Traffic anomaly event categories include at least one of the following: driving against traffic, dangerous lane changes, illegal U-turns, sudden crossing of obstructed areas, occupation of emergency lanes, abnormal parking, illegal overtaking, motor vehicles entering non-motorized vehicle lanes, and abnormal crossing or intrusion of pedestrians. The traffic anomaly event multi-prototype memory is used to maintain multiple learnable prototypes for each traffic anomaly event category to express the multiple manifestations of the same anomaly event under different roads, different weather conditions, different traffic densities, different camera angles, and different time periods.

[0071] The process of acquiring a learnable prototype may include: In step S28, a small number of training images corresponding to each traffic anomaly event category are obtained.

[0072] In step S29, the visual Transformer feature extraction network, traffic scene context token, and training token features corresponding to the few-sample training images are obtained through pre-trained visual Transformer feature extraction network, traffic scene context token, and feature enhancement, alignment or fusion process.

[0073] In step S30, the training token features corresponding to each few-sample training image of the same traffic anomaly event category are pooled to obtain the sample-level description vector of the few-sample training image.

[0074] In step S31, clustering or grouping of a small number of training images of the same traffic anomaly event category is performed based on the sample-level description vector, thereby obtaining multiple sample groups corresponding to multiple traffic scene performance patterns.

[0075] In step S32, the training token features within each sample group are averaged, weighted averaged, attention pooled, or key token selected according to the token position to obtain the token-level initial prototype corresponding to the sample group.

[0076] In step S33, the initial prototype at the token level corresponding to the sample group is used as a learnable prototype, and the learnable prototype is updated during the training process.

[0077] In this invention, the prototype set of the c-th traffic anomaly event category in the multi-prototype memory can be represented by formula (4a): Formula (4a) in, This represents the set of prototypes corresponding to the c-th traffic incident category. This represents the m-th learnable prototype for the c-th traffic incident category. The feature dimension represents the prototype token. By employing a token-level multi-prototype representation, the same traffic anomaly event category can simultaneously preserve local structure, target relationships, and traffic context differences. Compared to representing a traffic anomaly event category using only a single category center vector, the prototype token sequence can separately carry local patterns such as target appearance, target-lane relationship, target-surrounding traffic participant relationship, and road structure constraints. This reduces the risk of losing the averaged differences of the same anomaly event category under different road morphologies, camera perspectives, weather conditions, and traffic densities.

[0078] Traffic anomaly events can be categorized into several types, including driving against traffic, dangerous lane changes, illegal U-turns, sudden crossings of obstructed areas, occupation of emergency lanes, abnormal parking, illegal overtaking, motor vehicles encroaching on non-motorized vehicle lanes, and pedestrians crossing or intruding abnormally. For example, dangerous lane changes can include multiple prototypes such as dangerous lane changes on highways, dangerous lane changes during congestion, dangerous lane changes at night, and dangerous lane changes at intersections. Similarly, driving against traffic can include multiple prototypes such as driving against traffic on urban roads, driving against traffic on highways, and driving against traffic on ramps. Sudden crossings of obstructed areas can include anomalies where pedestrians, non-motorized vehicles, motor vehicles, or other traffic participants suddenly enter the target vehicle's path from an area obstructed by vehicles, road facilities, or blind spots. Furthermore, learnable prototypes corresponding to normal traffic state categories or unknown anomaly categories can be set in the traffic anomaly event multi-prototype memory to distinguish between normal traffic scenarios and anomaly events during actual deployment, or to perform open-set rejection of traffic scenarios not covered by known anomaly event categories.

[0079] When acquiring learnable prototypes, we can first obtain few-sample training images for each traffic anomaly event category. Then, we use a pre-trained visual Transformer feature extraction network, traffic scene context tokens, and the training token features corresponding to these few-sample training images obtained during feature enhancement, alignment, or fusion processes. Next, we can obtain the traffic scene context tokens and traffic rule prior tokens. Then, we group the training samples within the same traffic anomaly event category according to visual appearance and traffic scene sub-patterns to obtain multiple initial prototypes. Finally, we set these initial prototypes as learnable parameters and update them in subsequent training. Specifically, we can pool the training token features corresponding to each few-sample training image within the same traffic anomaly event category to obtain sample-level description vectors for the few-sample training images. Based on these sample-level description vectors, we can cluster or group the few-sample training images within the same traffic anomaly event category to obtain multiple sample groups corresponding to multiple traffic scene representation patterns. For each sample group, the training token features are averaged, weighted averaged, attention pooled, or key token selected according to the token position to obtain the token-level initial prototype corresponding to the sample group. Then, the token-level initial prototype corresponding to the sample group can be used as the learnable prototype, and the learnable prototype is updated during the training process.

[0080] For example, suppose the c-th traffic anomaly category includes Zhang Shao's training images, number The training token features corresponding to the training images are ,in The traffic area token, location relationship code, and traffic rule prior token corresponding to the training image can be denoted as follows: , and We can first obtain sample-level traffic pattern description vectors based on the training token features and traffic scene context. Specifically, this can be expressed by formula (4b): Formula (4b) in, This represents a sample-level traffic pattern description function, used to generate sample description vectors based on training token features, traffic area tokens, location relationship encodings, and prior traffic rule tokens. Then, it iterates through the set of sample-level description vectors within the same traffic anomaly event category... The scene sub-patterns are grouped according to road type, camera angle, weather lighting, traffic density, target vehicle posture, and the relationship between the target and lane lines or the relationship between the target and surrounding traffic participants to obtain M sample groups. to For the m-th sample group of the c-th traffic anomaly event category The token features corresponding to the training images within this group can be weighted and aggregated according to token location, traffic area correspondence, and sample confidence to obtain a token-level initial prototype. Specifically, this can be represented by formula (4c). Formula (4c) in, This represents the initial prototype. This represents a relational alignment aggregation function. This represents the m-th sample group of the c-th traffic incident category. When the obtained... The number of tokens included is greater than the number of preset prototype tokens. At that time, you can select based on regional importance weight, sample confidence, or intra-category variability. Each key token can also be compressed into a single key token using linear mapping or attention pooling. A prototype token. When obtained... The number of tokens included is less than At this time, tokens can be supplemented through learnable methods, repeated sampling, or linear interpolation. Thus, each initial prototype... All were adjusted to The dimensions are defined and used as learnable parameters in the multi-prototype memory of traffic anomaly events. For normal traffic state categories or unknown anomaly categories, the corresponding prototypes can be initialized using normal traffic scene samples, low-confidence samples, or training token features that are far from known anomaly event categories. Here, normal traffic state indicates that there are no known traffic anomaly events in the traffic scene image to be identified, while unknown anomaly category indicates that the traffic scene image to be identified cannot be reliably explained by known traffic anomaly event categories but has anomaly or uncertain features.

[0081] The deployed multi-prototype memory for traffic anomalies can be incrementally updated based on manually verified new traffic anomaly samples, low-confidence samples, or unknown anomaly samples. Specifically, the query token features corresponding to the new samples can be matched with existing category prototypes based on similarity. If the distance between the new sample and the existing category prototype is less than a preset update threshold, the corresponding prototype can be updated using momentum update, weighted average, or mini-batch retraining. For example, momentum update can be performed according to formula (4d): Formula (4d) in, This represents a token-level prototype generated from a new sample or a group of new samples. This represents the prototype update momentum coefficient. If the distance between a new sample and existing category prototypes is greater than a preset addition threshold, and it is confirmed by manual review to belong to a novel anomalous event or unknown anomalous pattern, a new category prototype or unknown anomalous prototype can be generated. If the new sample is a false alarm sample or a normal traffic scene sample, it can be used to update the normal traffic state category prototype or to recalibrate the low confidence threshold. In this way, the system can adapt to new road scenes, new camera perspectives, or novel anomalous events without retraining the entire model.

[0082] In one embodiment of the present invention, the process of determining candidate prototypes may include: In step S34, the visual matching score between the query token features and the learnable prototypes of each traffic abnormality event category in the traffic abnormality event multi-prototype memory is calculated.

[0083] In step S35, a context consistency score is calculated based on the consistency between the traffic scenario context token and the traffic area relationship, location relationship, or traffic rule prior in the learnable prototype corresponding to each traffic abnormal event category.

[0084] In step S36, routing scores for each traffic anomaly event category are calculated based on visual matching scores and contextual consistency scores.

[0085] In step S37, the traffic anomaly events are sorted according to the routing score.

[0086] In step S38, the top-k traffic abnormality event categories are selected as candidate abnormality event categories.

[0087] In step S39, the learnable prototypes corresponding to the candidate abnormal event categories are used as candidate prototypes for subsequent token-level reconstruction.

[0088] In this invention, dynamic prototype routing is used to filter out candidate abnormal event categories or candidate prototypes that are more relevant to the current traffic scene image to be identified from all traffic abnormal event categories and all prototypes, so as to reduce the computational load of subsequent prototype reconstruction and reduce interference from irrelevant abnormal event category prototypes.

[0089] The visual matching score between the query token features and the learnable prototypes of each traffic anomaly event category in the multi-prototype memory of traffic anomaly events can be calculated separately. Then, based on the consistency between the traffic scene context token and the corresponding learnable prototypes of each traffic anomaly event category in terms of traffic area relationships, location relationships, or prior traffic rules, a context consistency score can be calculated. Based on the visual matching score and the context consistency score, a routing score can be calculated for each traffic anomaly event category. The traffic anomaly event categories can be sorted according to the routing score. The top-k traffic anomaly event categories can be selected as candidate anomaly event categories. The learnable prototypes corresponding to the candidate anomaly event categories are then used as candidate prototypes for subsequent token-level reconstruction.

[0090] Specifically, a query traffic description vector q can be generated first based on the query token feature Q and the traffic scenario context token C; then, a category prototype description vector can be generated based on multiple prototypes of each traffic anomaly event category and their corresponding traffic scenario sub-pattern descriptions. Then the routing score is calculated. The route score can include a visual matching term and a contextual consistency term. The visual matching term represents the proximity of the query token to the category prototype in local visual patterns, while the contextual consistency term represents whether the relationships between road structure, lane lines, target vehicles, surrounding traffic participants, and prior traffic rules are consistent with the candidate category prototype. For example, the normalized matching term in the route score can be calculated according to formula (5): Formula (5) in, represents the routing score between the traffic scene image to be identified and the c-th traffic anomaly event category, and q represents the query traffic description vector generated jointly by the query token feature and the traffic scene context token. This represents the category prototype description vector corresponding to the c-th traffic anomaly event category. Let L2 be the norm. Then, according to... Sort the traffic anomaly events from highest to lowest, and select k categories as candidate anomaly event categories. The set of candidate anomaly event categories can be represented by formula (5a): Formula (5a) in, This represents the set of candidate exception event categories obtained from dynamic prototype routing. to This represents the k candidate exception event categories selected from highest to lowest based on routing scores, where j indicates that none were selected. Other traffic anomaly event categories, where k is a positive integer, and k is less than or equal to the total number of traffic anomaly event categories in the multi-prototype memory of traffic anomaly events. During the training phase, routing supervision signals can be constructed based on the anomaly event category labels to make the routing score of the labeled category higher than that of the unlabeled category, or to make the labeled category fall into the candidate anomaly event category set, thereby reducing the probability of dynamic routing missing real anomaly event categories during the inference phase.

[0091] In one embodiment of the present invention, the process of obtaining the reconstructed token characteristics may include: In step S40, the query token feature is used as the query vector.

[0092] In step S41, the prototype token of the candidate prototype and the traffic scene context token are used as reconstruction conditions.

[0093] In step S42, a reconstructed token feature is generated based on the correspondence between the prototype token of the candidate prototype, the traffic scenario context token, and the query token feature.

[0094] In this invention, during reconstruction, the query token feature can be used as the query vector, and the prototype token of the candidate prototype and the traffic scene context token can be used as reconstruction conditions to generate reconstructed token features based on the correspondence between the prototype token of the candidate prototype, the traffic scene context token, and the query token feature. Specifically, the query token can be reconstructed under the common constraints of the candidate prototype and the traffic scene context token. This is done for each candidate anomaly event category c and each candidate prototype under that candidate anomaly event category. The query token feature Q can be used as the object to be explained, and the candidate prototype can be used as the object to be explained. The prototype token and the traffic scene context token C are used as reconstruction conditions to calculate the token-level correspondence. This process can be expressed as formula (6): Formula (6) in, This represents the context constraint correspondence result for the m-th candidate prototype of candidate anomaly event category c, where Q represents the query token feature. This represents a key vector generated from the candidate prototype token and the traffic scenario context token. This represents a value vector generated from the candidate prototype token and the traffic scenario context token. This represents an exponential function normalized along the token dimension. This represents the attention head dimension. Furthermore, and It can be obtained by concatenating and mapping candidate prototype tokens, traffic scene context tokens, and optional time relationship tokens, for example, it can be represented by formula (6a): Formula (6a) in, Represents a mapping function. This represents the temporal relationship token generated when the input is a multi-frame traffic scene image; when the input is only a single-frame traffic scene image... It can be an empty, zero vector, or a learnable default token. Furthermore, the result can be constrained according to the context. The reconstructed token feature corresponding to the m-th candidate prototype of candidate anomaly event category c is generated, which can be represented by, for example, formula (6b): Formula (6b) in, This represents the reconstructed token feature generated from the m-th candidate prototype of candidate anomaly event category c. This represents the reconstruction mapping function. The candidate anomaly prototype can reconstruct the query token features by combining road structure, lane lines, target vehicles, and relationships with surrounding traffic participants. Specifically, the query token provides the local areas and behavioral clues that need to be explained in the current traffic scene; the candidate prototype token provides typical sub-patterns of the candidate anomaly event category; the traffic scene context token provides road structure, positional relationships, and traffic rule constraints; and the reconstruction token represents the restoration result of the candidate anomaly event category on the local structure of the current traffic scene under the above contextual constraints. Therefore, when the candidate anomaly event category matches the current scene, its prototype token can interpret the query token well, with a small reconstruction error; when the candidate anomaly event category does not match the current scene, the reconstruction token struggles to restore the key local structures or rule relationships in the query token, resulting in a larger reconstruction error.

[0095] On the other hand, reconstruction can also be performed using attention-weighted reconstruction, low-rank mapping reconstruction, multilayer perceptron mapping reconstruction, Transformer Decoder-based reconstruction structures, or combinations thereof. During reconstruction, the input includes at least prototype tokens of candidate abnormal event categories and query token features. Under the constraint of introducing traffic scene context tokens or their equivalent context representations, the output is a reconstructed token feature that matches the query token feature dimension, so that the reconstruction error can be further calculated. All of the above alternative structures can be used to perform token-level interpretation and restoration of query token features based on the prototypes of candidate abnormal event categories, without changing the technical concept of classification based on the reconstruction error of candidate prototypes. Therefore, the focus of this invention is on the reconstruction interpretation mechanism of query token features based on the prototypes of candidate abnormal event categories, rather than being limited to a specific attention structure.

[0096] Unlike directly calculating the cosine similarity between the query image and the category prototype, prototype reconstruction can model the correspondence between local tokens and context tokens in a traffic scene. Traditional similarity classification mainly answers whether the query image and the category center are "similar," which is easily affected by background, viewpoint, vehicle appearance, or lighting conditions. The reconstruction error classification of this invention further determines whether the prototype of the candidate abnormal event category can explain and restore the local structure of the query token under traffic context constraints. When there are only subtle differences between events such as normal lane change and dangerous lane change, normal driving and wrong-way driving, and normal parking and abnormal parking, this reconstruction mechanism can amplify the differences in the explanatory power of the candidate abnormal event prototypes for the traffic scene, thereby improving fine-grained recognition capabilities. For example, for dangerous lane change events, the target vehicle appearance token itself may be similar to that of normal lane change, but the relationship between the target vehicle token and lane line token, adjacent vehicle token, and prior traffic rule token will change; for events of sudden crossing of an occluded area, the local patch token of the crossing target may appear weakly or briefly, but it, together with the occluded area, the target vehicle's driving path, and the positional relationship of surrounding traffic participants, constitutes the basis for anomaly discrimination. By storing the above information in different types of tokens and interacting with them during the reconstruction phase, this invention can avoid global average features from masking fine-grained anomaly clues.

[0097] In one embodiment of the present invention, the process of recognizing the traffic scene image to be recognized may include: In step S43, the reconstruction error between the reconstructed token feature and the query token feature corresponding to each candidate prototype under each candidate anomaly event category is calculated.

[0098] In step S44, the traffic relationship error is calculated based on the consistency between the road structure, lane lines, target vehicle, surrounding traffic participants, or traffic rule priors in the traffic scene context token and the features of the reconstructed token.

[0099] In step S45, the reconstruction errors and traffic relationship errors corresponding to multiple candidate prototypes under the same candidate abnormal event category are aggregated to obtain the abnormal event category reconstruction error.

[0100] In step S46, the abnormal event category score is determined based on the abnormal event category reconstruction error.

[0101] In step S47, the category with the highest abnormal event category probability, i.e., the abnormal event category score, is determined as the traffic abnormal event identification result.

[0102] In step S48, if the maximum value of the probability of the abnormal event category is less than the preset probability threshold, or if the probability of the abnormal event category of each candidate abnormal event category is greater than the preset error threshold, the traffic scene image to be identified is determined as a normal traffic state or an unknown abnormal state.

[0103] In this invention, during identification, the token reconstruction error between the reconstructed token feature and the query token feature corresponding to each candidate prototype under each candidate abnormal event category can be calculated first. Then, based on the consistency between the road structure, lane lines, target vehicles, surrounding traffic participants, or traffic rules priors in the traffic scene context token and the reconstructed token features, the traffic relationship error can be calculated. The token reconstruction errors and traffic relationship errors corresponding to multiple candidate prototypes under the same candidate abnormal event category can be aggregated to obtain the abnormal event category reconstruction error. Then, the abnormal event category score can be determined based on the abnormal event category reconstruction error. The category with the highest abnormal event category probability, i.e., the abnormal event category score, can be determined as the traffic abnormal event identification result. If the maximum abnormal event category probability is less than a preset probability threshold, or if the abnormal event category probability of each candidate abnormal event category is greater than a preset error threshold, the traffic scene image to be identified can be determined as a normal traffic state or an unknown abnormal state.

[0104] Specifically, for the m-th candidate prototype of candidate anomaly event category c, the token reconstruction error can be calculated first according to formula (7): Formula (7) in, This represents the token reconstruction error corresponding to the m-th candidate prototype of candidate exception event category c. This indicates the number of tokens being queried. This represents the i-th query token. This represents the i-th reconstructed token obtained from the m-th candidate prototype of the c-th candidate abnormal event category. Based on this, the traffic relationship error can be calculated according to the consistency between the road structure, lane lines, target vehicles, surrounding traffic participants, or traffic rule priors in the traffic scene context token and the features of the reconstructed token. The visual token reconstruction error and the traffic relationship error are then used together for category score calculation.

[0105] In one alternative implementation, the minimum reconstruction error corresponding to multiple candidate prototypes under the same candidate anomaly event category can be taken to obtain the anomaly event category reconstruction error, which can be expressed by, for example, according to formula (8): Formula (8) In another alternative implementation, the reconstruction errors of multiple candidate prototypes under the same category can be weighted and summed using learnable weights to obtain the reconstruction error of the abnormal event category.

[0106] After obtaining the category reconstruction error of the candidate abnormal event categories, the confidence level of the abnormal event category can be calculated according to formula (9): Formula (9) in, This represents the class confidence of candidate anomaly event category c, where c∈ This indicates that the confidence level is calculated within the set of candidate anomaly event categories obtained from dynamic prototype routing, and τ represents the temperature coefficient. This represents the category reconstruction error of the j-th candidate abnormal event category. Finally, the candidate abnormal event category with the highest category score and meeting the error threshold condition can be determined as the traffic abnormal event identification result, and at least one of the abnormal event category label, category confidence, and candidate identification result can be output. In an optional implementation, if the maximum category confidence is lower than a preset probability threshold, or if the category reconstruction error of each candidate abnormal event category is greater than the preset error threshold, the current traffic scene image to be identified can be output as normal traffic state or unknown abnormal state according to formula (10): Formula (10) in, This indicates a low confidence level. (·) indicates an indicator function. This indicates whether the traffic has entered a normal state or an unknown abnormal state. This indicates a preset probability threshold. ∨ represents a preset error threshold, and ∨ represents a logical OR. "Normal traffic state" indicates that no known abnormal events have been detected in the current traffic scenario, while "unknown abnormal state" indicates that the current traffic scenario does not match any of the known traffic abnormal event categories but exhibits low-confidence abnormal signs. Furthermore, when performing low-confidence judgment, the reconstruction error of the normal traffic state category prototype can be calculated separately. Reconstruction error of prototypes of unknown anomaly categories And perform a subdivision judgment according to formula (10a): Formula (10a) in, This represents the output of the low-confidence decision branch. Indicates normal traffic conditions. Indicates an unknown abnormal state. This indicates a preset normal error threshold; ∧ represents logical AND, and ∨ represents logical OR. In other words, if... Less than and If the error is less than the preset normal error threshold, output normal traffic status; if... Less than If neither the candidate abnormal event category nor the normal traffic state category can explain the query token features with a reconstruction error below a preset error threshold, then an unknown abnormal state is output. This approach avoids simply categorizing normal road scenarios as unknown abnormal states, and also avoids forcibly classifying unseen abnormal events into a known traffic abnormal event category.

[0107] In an optional implementation, embodiments of the present invention can also output a risk level. Specifically, a high-risk, medium-risk, or low-risk level can be determined based on at least one of the following: the probability of the abnormal event category, the reconstruction error of the abnormal event category, the positional relationship between the target vehicle and the lane lines, the distance relationship between the target vehicle and surrounding traffic participants, or the duration of the abnormal event. For example, a risk score can be calculated first. The risk level is determined based on the relationship between the risk score and the preset risk threshold. The risk score can be expressed by formula (10b): Formula (10b) in, This indicates the probability of the identified abnormal event category. This indicates the reconstruction error of the identified anomalous event category. The location risk factor is derived from the positional relationship between the target vehicle and lane lines, road boundaries, or surrounding traffic participants. This represents the time risk factor derived from the duration of the abnormal event or the temporal relationship across multiple frames, where ε is a constant used to avoid division by zero. to This indicates that weights can be assigned. If the risk score is greater than the high-risk threshold, then high risk is output; if the risk score is between the medium-risk threshold and the high-risk threshold, then medium risk is output; if the risk score is lower than the medium-risk threshold, then low risk is output. For example, when the probability of a dangerous lane-changing event is high and the distance between the target vehicle and surrounding vehicles is less than a preset threshold, high risk can be output.

[0108] During the training phase, the embodiments of the present invention can employ a joint optimization of classification loss, reconstruction loss, prototype constraint loss, context consistency loss, and route supervision loss. Training samples can be organized in the form of N-way K-shot few-shot tasks, or in the form of class-balanced mini-batch or long-tail class resampling. Each training batch can include N traffic abnormal event categories, K training images of each category or no more than a preset small number of samples, and corresponding abnormal event category labels. K can be set according to the deployment scenario, for example, 1, 3, 5, 10, or other small number of labeled samples. For each training batch, the original training token features are first extracted by a visual Transformer feature extraction network that is pre-trained and frozen based on large-scale unlabeled traffic scene data under self-supervision. Then, a traffic scene context token is generated, and the original training token features are processed and aligned or fused with the traffic scene context token. Subsequently, the abnormal event category score is obtained through the multi-prototype routing and prototype reconstruction module. The route supervision loss can be calculated according to formula (11): Formula (11) in, Indicates the routing supervision loss. Indicates the category of real abnormal events The corresponding route score, This represents the maximum route score among non-real categories, where δ represents the preset interval. This represents the positive part function. This loss can be used to increase the routing score of true anomaly categories compared to non-true categories, thereby increasing the probability that true anomaly categories will be selected into the candidate set by the dynamic prototype route. The classification loss and reconstruction loss can be calculated using formulas (11a) and (11b), respectively: Formula (11a) Formula (11b) in, Indicates the category of real abnormal events The class probability, Indicates the category of real abnormal events The category reconstruction error. Furthermore, to avoid multiple prototypes within the same traffic anomaly category collapsing into the same pattern, the prototype diversity loss can be calculated according to formula (11c): Formula (11c) in, Represents the prototype constraint loss, ρ( () represents the similarity between the m-th and n-th prototypes under the same traffic anomaly event category c. When aiming for prototype diversity, minimize... This can reduce the similarity between different prototypes within the same category, thereby preventing multiple prototypes from collapsing into the same pattern. To ensure that the traffic scene context token is consistent with the abnormal event category, the context consistency loss can be calculated according to formula (11d): Formula (11d) in, This represents a lightweight mapping function that predicts the category of abnormal events based on the context of a traffic scenario, token C, where σ represents the normalized exponential function. Indicates the category of real abnormal events The corresponding context prediction probability. The total loss function can be expressed as formula (12): Formula (12) in, Represents classification loss, Indicates the losses incurred during reconstruction. This represents the prototype diversity loss or prototype regularization loss. This indicates the loss of traffic context consistency. Indicates the routing supervision loss. , , and This represents the weighting coefficient. The route supervision loss can be calculated based on whether the traffic anomaly category corresponding to the label is selected into the candidate anomaly category set, or it can be calculated based on the interval between the route score of the label category and the route score of the non-label category.

[0109] In one optional implementation, the visual Transformer feature extraction network, pre-trained and frozen based on large-scale unlabeled traffic scene data through self-supervised training, does not update its parameters during training. Instead, it primarily updates learnable parameters for performing feature enhancement, alignment, or fusion processes, the multi-prototype memory for traffic anomalies, and learnable parameters in the dynamic prototype routing and prototype reconstruction modules. The feature enhancement process can preferably be implemented by the traffic feature enhancement module. During training, classification loss can be used to drive higher category confidence for correct anomaly categories, reconstruction loss can be used to help correct category prototypes better reconstruct query tokens, prototype diversity loss can be used to prevent multiple prototypes within the same category from collapsing into the same pattern, context consistency loss can be used to constrain the association between traffic scene context tokens and anomaly categories, and routing supervision loss can be used to increase the probability of label categories entering the candidate anomaly category set. By freezing the traffic scene self-supervised pre-trained backbone network and only updating subsequent modules, the parameter search space during few-sample training can be reduced, improving training stability.

[0110] During the inference phase, only the image of the traffic scene to be identified needs to be input, and the following steps are performed sequentially: image patch segmentation, frozen visual Transformer feature extraction, traffic scene context token generation, feature enhancement, alignment or fusion, dynamic prototype routing, candidate prototype reconstruction, and reconstruction error classification. The inference phase does not require input of support set images, nor does it require rebuilding of category prototypes; the trained multi-prototype memory of traffic anomalies can be directly used as the category knowledge base during the deployment phase. When none of the candidate anomaly categories selected by dynamic prototype routing can explain the traffic scene image to be identified with low reconstruction error, a low-confidence judgment can be performed based on a preset probability threshold or a preset error threshold, and further, the normal traffic state or unknown anomaly state can be output based on the reconstruction errors of the normal traffic state category prototype and the unknown anomaly category prototype.

[0111] In one optional implementation, the preset probability threshold used when performing low-confidence judgment, the preset error threshold used when performing low-confidence judgment, the number of Top-k candidate anomaly categories used when performing dynamic prototype routing, the risk level corresponding threshold used when performing risk level output, or the prototype update conditions used when performing prototype incremental updates can be calibrated based on the validation set, historical traffic anomaly event samples, or manual review results. Specifically, the maximum category probability, minimum reconstruction error, and risk score distribution corresponding to correctly identified samples, false positive samples, and missed samples can be statistically analyzed on the validation set; the probability threshold and error threshold can be selected based on the target false positive rate or target missed positive rate; the number of Top-k candidate anomaly categories can be selected based on the recall rate of real anomaly event categories entering the candidate set; and the risk level threshold can be selected based on the distribution of high-risk, medium-risk, and low-risk samples from manual review. For road scenarios with high false positive costs, the probability threshold can be increased or the sensitivity of unknown anomaly output conditions can be decreased; for highway, ramp, or intersection scenarios with high missed positive costs, the threshold for candidate categories to enter the reconstruction stage can be lowered or the number of Top-k candidates can be increased. By using scenario-based threshold calibration, a balance can be achieved between recognition accuracy, false alarm rate, false negative rate, and real-time computational load.

[0112] On the other hand, the present invention can also provide a few-shot image recognition system for traffic anomalies, the few-shot image recognition system including: The image acquisition module is used to acquire images of the traffic scene to be identified. The feature extraction module is used to feed the traffic scene image to be identified into the pre-trained visual Transformer feature extraction network to obtain the original patch token features. The traffic context generation module is used to generate a traffic scene context token based on the traffic areas and traffic participants in the traffic scene image to be identified, combined with at least one of the inter-area relationship or traffic rule priors. The feature processing module is used to process the original patch token features to obtain enhanced traffic token features, and to align or fuse the original patch token features and / or enhanced traffic token features with the traffic scenario context token to obtain query token features. The dynamic routing module is used to perform dynamic routing based on a pre-built multi-prototype memory of traffic anomalies and query token features to determine candidate prototypes; The prototype reconstruction module is used to reconstruct the prototype based on the candidate prototype, the traffic scenario context token, and the query token features under the constraints of the traffic scenario context, so as to obtain the reconstructed token features. The abnormal event recognition module is used to determine the category score of the traffic scene image to be recognized corresponding to each candidate abnormal event category based on the reconstruction error between the reconstructed token features and the query token features, and to obtain the recognition result of the traffic scene image to be recognized based on the category score.

[0113] Through the above technical solution, the present invention provides a few-shot image recognition method and system for traffic anomaly events. This method acquires an image of a traffic scene to be identified and then feeds it into a pre-trained visual Transformer feature extraction network to extract original patch token features. This visual Transformer feature extraction network can be a self-supervised pre-trained and frozen visual Transformer feature extraction network based on large-scale unlabeled traffic scene data. A traffic scene context token can be generated based on the road structure region, lane line region, target vehicle region, and surrounding traffic participant region in the image of the traffic scene to be identified. Feature processing can be performed on the original patch token features to obtain enhanced traffic token features. The original patch token features and / or enhanced traffic token features can be aligned or fused with the traffic scene context token to obtain query token features. Dynamic routing is performed based on a pre-built multi-prototype memory for traffic anomaly events and the query token features. This allows for the determination of candidate prototypes corresponding to candidate anomaly event categories from the multi-prototype memory for traffic anomaly events. The multi-prototype memory for traffic anomaly events can include multiple learnable prototypes corresponding to each traffic anomaly event category. Based on candidate prototypes, traffic scene context tokens, and query token features, the query token features can be reconstructed under the constraints of the traffic scene context to obtain reconstructed token features. These reconstructed token features can be used to characterize the interpretation results of the query token features by the prototypes corresponding to the candidate abnormal event categories. The reconstruction error between the reconstructed token features and the query token features can be used to determine the category score of the traffic scene image to be identified for each candidate abnormal event category. The recognition result of the traffic scene image to be identified, i.e., the recognition result of the traffic abnormal events in the traffic scene image to be identified, can be obtained based on the category score. This few-shot image recognition method can perform fine-grained recognition of traffic abnormalities under few-shot conditions.

[0114] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0115] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for few-sample image recognition of traffic anomalies, characterized in that, The few-sample image recognition method includes: Acquire images of the traffic scene to be identified; The traffic scene image to be identified is fed into a pre-trained visual Transformer feature extraction network to obtain the original patch token features. Generate a traffic scene context token based on the relevant regions in the image of the traffic scene to be identified; The original patch token features are processed to obtain enhanced traffic token features. The original patch token features and / or enhanced traffic token features are then aligned or fused with the traffic scenario context token to obtain query token features. Dynamic routing is performed based on a pre-built multi-prototype memory of traffic anomalies and query token features to determine candidate prototypes; Based on the candidate prototype, traffic scenario context token, and query token features, reconstruction is performed under the constraints of the traffic scenario context to obtain the reconstructed token features; The category score of the traffic scene image to be identified is determined based on the reconstruction error between the reconstructed token feature and the query token feature, corresponding to each candidate abnormal event category. The identification result of the traffic scene image to be identified is obtained based on the category score.

2. The few-sample image recognition method according to claim 1, characterized in that, The traffic scene image to be identified is fed into a pre-trained visual Transformer feature extraction network to obtain the original patch token features, including: Collect large-scale unlabeled traffic scene data, including traffic images or video frames in urban roads, highways, tunnels, intersections, rainy days, nights, foggy days, and congested scenarios; Perform multi-view data augmentation on the same traffic image or traffic video frame to obtain a global augmented view and a local augmented view; Construct a student visual Transformer and a teacher visual Transformer, where the student visual Transformer updates its parameters through backpropagation, and the teacher visual Transformer updates its parameters through an exponential moving average of the student visual Transformer's parameters. The global augmented view and the local augmented view are fed into the student visual Transformer and the teacher visual Transformer, and pre-trained based on the DINO self-supervised alignment loss between the predicted distribution output by the student visual Transformer and the target distribution output by the teacher visual Transformer. After pre-training, the student visual Transformer and projection head are discarded, and the Transformer Encoder in the teacher visual Transformer is retained as the frozen visual Transformer feature extraction network after pre-training.

3. The few-sample image recognition method according to claim 1, characterized in that, The traffic scene image to be identified is fed into a pre-trained visual Transformer feature extraction network to obtain the original patch token features, including: The traffic scene image to be identified is divided into multiple image blocks; Linear embedding and position encoding are performed on the image blocks to obtain the input token sequence; The input token sequence is fed into the visual Transformer feature extraction network; The classification labels output by the visual Transformer feature extraction network are removed, and the patch tokens corresponding to each image patch are retained as the original patch token features.

4. The few-sample image recognition method according to claim 1, wherein the relevant region includes a road structure region, a lane line region, a target vehicle region, and a surrounding traffic participant region, characterized in that, A traffic scene context token is generated based on the relevant regions in the image of the traffic scene to be identified, including: The traffic scene image to be identified is subjected to at least one of the following: target detection, semantic segmentation, instance segmentation, region detection, region segmentation, attention hotspot extraction, visual Transformer activation region extraction, or preset region prior matching, in order to confirm the relevant regions. The patch tokens corresponding to the relevant areas are pooled, aggregated, or linearly mapped to obtain road structure tokens, lane line tokens, target vehicle tokens, and surrounding traffic participant tokens. A traffic scenario context token is generated based on the road structure token, lane line token, target vehicle token, and surrounding traffic participant tokens, as well as the location relationship encoding between relevant areas.

5. The few-sample image recognition method according to claim 1, characterized in that, Enhanced traffic token features are obtained by performing feature processing on the original patch token features. The original patch token features and / or enhanced traffic token features are then aligned or fused with the traffic scenario context token to obtain query token features, including: Global statistics are performed on the original patch token features and traffic scenario context tokens at the channel dimension, and channel attention weights are obtained through fully connected layers and activation functions; Based on the channel attention weights, the original patch token features and the traffic scenario context token are channel-weighted to obtain channel-enhanced token features; The original patch token features are restored into a two-dimensional token feature map according to the spatial location of the image patch, and spatial attention weights are obtained through convolutional layers and activation functions; Based on the spatial attention weights, the tokens in the original patch token features that are related to the target vehicle, lane lines, road boundaries and surrounding traffic participants are spatially weighted to obtain spatially enhanced token features. The channel enhancement token features and spatial enhancement token features are linearly mapped, residually connected, or fused into enhanced traffic token features using a Transformer layer; Perform linear projection, spatial broadcasting, or cross-attention injection on traffic scene context tokens to obtain context-aligned token features that are aligned with the original patch token features in terms of token quantity or feature dimension. The fusion ratio of the original patch token features, context-aligned token features, and / or enhanced traffic token features is determined based on learnable weights or gating coefficients. The original patch token features, context-aligned token features, and / or enhanced traffic token features are weighted and fused according to the fusion ratio to obtain the query token features, or The query token feature is obtained by concatenating the original patch token feature, context-aligned token feature, and / or enhanced traffic token feature through linear mapping.

6. The few-sample image recognition method according to claim 1, characterized in that, The multi-prototype memory for traffic anomalies includes multiple traffic anomaly categories, each category corresponding to M learnable prototypes, and each learnable prototype includes... There are several prototype tokens, each with the same dimensions as the query token's features, where M and... All are positive integers, where M represents the number of learnable prototypes for each traffic anomaly event category. Used to represent the number of prototype tokens contained in each learnable prototype. Traffic abnormal event categories include at least one of the following: driving against the flow of traffic, dangerous lane change, illegal U-turn, sudden crossing of obstructed areas, occupation of emergency lane, abnormal parking, illegal overtaking, motor vehicle intrusion into non-motor vehicle lane, and abnormal crossing or intrusion of pedestrians. The acquisition of learnable prototypes includes: Obtain a small number of training images corresponding to each traffic anomaly event category; The visual Transformer feature extraction network, traffic scene context tokens, and training token features corresponding to few-sample training images are obtained through pre-trained visual Transformer feature extraction network, traffic scene context tokens, and feature enhancement, alignment or fusion processes. Pool the training token features corresponding to each few-sample training image of the same traffic anomaly event category to obtain the sample-level description vector of the few-sample training image; Clustering or grouping a small number of training images of the same traffic anomaly event category based on the sample-level description vectors, thereby obtaining multiple sample groups corresponding to multiple traffic scene performance patterns. For each sample group, the training token features are averaged, weighted averaged, attention pooled, or key token selected according to the token position to obtain the token-level initial prototype corresponding to the sample group. The token-level initial prototype corresponding to the sample group is used as a learnable prototype, and the learnable prototype is updated during the training process.

7. The few-sample image recognition method according to claim 6, characterized in that, Dynamic routing is performed based on a pre-built multi-prototype memory of traffic anomalies and query token features to determine candidate prototypes, including: Calculate the visual matching score between the query token features and the learnable prototypes of each traffic abnormality event category in the multi-prototype memory of traffic abnormality events; Calculate the context consistency score based on the consistency between the traffic scenario context token and the learnable prototype corresponding to each traffic anomaly event category in terms of traffic area relationships, location relationships, or prior traffic rules. The routing score for each traffic anomaly event category is calculated based on the visual matching score and the contextual consistency score. The traffic anomaly events are sorted according to the route scores. Select the top-k traffic anomaly event categories from the ranking as candidate anomaly event categories; The learnable prototypes corresponding to the candidate abnormal event categories are used as candidate prototypes for subsequent token-level reconstruction.

8. The few-sample image recognition method according to claim 1, characterized in that, Based on the candidate prototype, traffic scenario context token, and query token features, reconstruction is performed under traffic scenario context constraints to obtain reconstructed token features, including: Use the query token feature as the query vector; Use the prototype token of the candidate prototype and the traffic scene context token as reconstruction conditions; Based on the correspondence between the prototype token of the candidate prototype, the traffic scenario context token, and the query token features, reconstructed token features are generated.

9. The few-sample image recognition method according to claim 1, characterized in that, The category score of the traffic scene image to be identified corresponding to each candidate abnormal event category is determined based on the reconstruction error between the reconstructed token features and the query token features. The identification result of the traffic scene image to be identified is obtained based on the category score, including: Calculate the reconstruction error between the reconstructed token feature and the query token feature for each candidate prototype under each candidate anomaly event category; Based on the consistency between the road structure, lane lines, target vehicle, surrounding traffic participants, or traffic rule priors in the traffic scenario context token and the features of the reconstructed token, the traffic relationship error is calculated. The reconstruction errors and traffic relationship errors corresponding to multiple candidate prototypes under the same candidate anomaly event category are aggregated to obtain the anomaly event category reconstruction error. The anomaly category score is determined based on the reconstruction error of the anomaly category. The category with the highest abnormal event category probability, i.e., the abnormal event category score, is determined as the traffic abnormal event identification result. If the maximum probability of an abnormal event category is less than a preset probability threshold, or if the probability of an abnormal event category for each candidate abnormal event category is greater than a preset error threshold, the traffic scene image to be identified is determined to be either a normal traffic state or an unknown abnormal state.

10. A few-sample image recognition system for traffic anomaly events, characterized in that, The few-sample image recognition system includes: The image acquisition module is used to acquire images of the traffic scene to be identified. The feature extraction module is used to feed the traffic scene image to be identified into the pre-trained visual Transformer feature extraction network to obtain the original patch token features. The traffic context generation module is used to generate a traffic scene context token based on the traffic areas and traffic participants in the traffic scene image to be identified, combined with at least one of the inter-area relationship or traffic rule priors. The feature processing module is used to process the original patch token features to obtain enhanced traffic token features, and to align or fuse the original patch token features and / or enhanced traffic token features with the traffic scenario context token to obtain query token features. The dynamic routing module is used to perform dynamic routing based on a pre-built multi-prototype memory of traffic anomalies and query token features to determine candidate prototypes; The prototype reconstruction module is used to reconstruct the prototype based on the candidate prototype, the traffic scenario context token, and the query token features under the constraints of the traffic scenario context, so as to obtain the reconstructed token features. The abnormal event recognition module is used to determine the category score of the traffic scene image to be recognized corresponding to each candidate abnormal event category based on the reconstruction error between the reconstructed token features and the query token features, and to obtain the recognition result of the traffic scene image to be recognized based on the category score.

Images