Small sample anomaly detection method and system based on multi-domain interaction construction and situational awareness optimization

Abstract

The present invention proposes a small-sample anomaly detection method and system based on multi-domain interaction construction and scenario perception optimization, the method is as follows: 1. Traffic anomaly videos in multiple road scenarios are converted into labeled image datasets; 2. The image datasets are preprocessed and corresponding scenario tasks are constructed according to the small-sample learning training requirements; 3. The images of each scenario task are input into the backbone network for feature extraction, and the image features are mapped to multiple feature domain spaces using sequential feature embedding, and the average value of the features of similar samples is calculated to obtain a multi-domain prototype; 4. The multi-domain prototype is input into the multi-domain interaction structure; 5. The multi-domain prototype is adaptively grouped into channels based on the current scenario task; 6. The inter-class relationship under each scenario task is used to adjust the weight of the channel group within the domain; 7. Deep training under small-sample learning is performed through a loss function; 8. The trained small-sample classification model is performance tested, and the model training parameters with the best inference performance are retained.

Description

Technical Field

[0001] The present invention belongs to the technical field of small sample anomaly detection based on metric methods, and specifically relates to a small sample anomaly detection method and system based on multi-domain interaction construction and context-aware optimization. Background Art

[0002] Anomaly detection has a wide range of real-world applications. It helps people identify the differences between normal and abnormal patterns in massive amounts of data, thereby maintaining system security and performance. However, traditional deep learning-based anomaly detection methods typically require a large number of labeled normal and abnormal samples, which requires significant manpower and time. While fine-tuning deep models pre-trained on large amounts of labeled data can detect some anomalies, without sufficient labeled data, deep models are prone to overfitting, resulting in reduced model inference performance.

[0003] In order to solve the problem of scarcity of labeled samples mentioned above, researchers have proposed a small sample learning method inspired by the fact that when humans take on a new task, they quickly master relevant abilities based on past experience. Among the existing small sample learning methods, the metric-based small sample learning method has shown excellent development prospects. This type of method constructs a series of scenario tasks in the training phase and uses a small number of support set samples (labeled samples) in each scenario task to learn meta-knowledge and construct class prototypes. In the test phase, the same scenario setting is used to generalize the meta-knowledge to the new test task to complete the sample classification task. However, due to factors such as complex background, target scale and shooting angle, the class center prototype obtained from the small number of labeled samples that support small sample learning training has a large offset from the true class center, which is called the prototype offset problem.

[0004] To alleviate the prototype drift problem and improve anomaly detection accuracy, researchers have proposed multi-prototype metric structures to enhance the ability to capture information from training sample features. These structures use attention mechanisms or embedding mapping functions to perceive key information representations within prominent class prototypes. However, these existing methods neglect the consistency of multi-prototype metric structures and the adaptability of small-shot learning models to different training scenarios. This limits the inference accuracy of metric-based small-shot learning methods in anomaly detection. Summary of the Invention

[0005] In response to the above-mentioned problems existing in the prior art, the present invention proposes a small sample anomaly detection method and system based on multi-domain interaction construction and context-aware optimization.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] The small sample anomaly detection method based on multi-domain interaction construction and context-aware optimization has the following specific steps:

[0008] S1. Convert traffic anomaly videos in multiple road scenarios into labeled image datasets.

[0009] S2. Preprocess the image dataset and construct corresponding scenario tasks based on the small sample learning training requirements, including support sets and query sets.

[0010] S3. Input the image of each scenario task into the backbone network for feature extraction, use the sequential feature embedding method to map the image features to multiple feature domain spaces, and obtain the multi-domain prototype by calculating the average feature value of similar samples.

[0011] S4. Input the multi-domain prototypes into the multi-domain interaction structure, and complete the inter-domain information complementation through the multi-domain interaction guided by the inter-domain related information.

[0012] S5. Adaptively group channels of multi-domain prototypes based on the current scenario task, weight each channel group within the domain through cross-domain channel group perception, and highlight the key information representation of each class prototype.

[0013] S6. The contextual attention mechanism is introduced to use the inter-class relationship under each contextual task to adjust the weight of the channel group in the domain, thereby expanding the difference between prototypes of different classes under each classification contextual task.

[0014] S7. Deep training for small-sample learning using loss functions.

[0015] S8. Perform performance tests on the trained small sample classification model and retain the model training parameters with the best inference performance.

[0016] As a preferred method, in step S1, the traffic abnormality videos under multiple existing scenarios are labeled according to the scenarios and road traffic conditions to create a labeled image dataset D label Then the labeled image dataset of each scene is divided into training set D according to a certain ratio (7:3) train and the test set D test In order to improve the generalization ability of the deep learning model with small samples, the categories of the training set and the test set are disjoint, that is,

[0017] Preferably, in step S2, the input image dataset is preprocessed to reduce the possibility of the model learning irrelevant information. The preprocessing includes cropping and numerical changes. Specifically, first, the video perspective of the original image (980*1320) is cropped, and then the image is scaled to (256*256); the original RGB three-channel image dataset is converted into a single-channel image dataset by grayscale changes. Based on the preprocessed image dataset, a scenario task Episode that meets the requirements of small sample iterative learning is constructed. The scenario task is generally set to N-way K-shot, that is, each scenario task has N categories, and each category has K samples, where K is generally selected as an integer 1 or 5. Based on the scenario task construction requirements from the training set D train The support set is obtained by random sampling and queryset That is, Episode = {S, Q}, where x i ,y i is the support set image samples and corresponding labels, x j ,y j is the query set image sample and the corresponding label, B represents the number of samples of each category in the query set. Here, the support set S and the query set Q satisfy But share the same label space.

[0018] As a preferred method, in step S3, a small sample deep learning framework is constructed under the PyTorch framework to complete the traffic anomaly detection task; the constructed scenario task image set is input into the backbone network Resnet-12, and feature extraction is performed on it to obtain the feature output of the last three convolutional network layers of the Resnet-12 network. The feature information of the input image is mapped to multiple feature domain spaces using the sequential feature embedding method. Specifically, assuming that the last layer of the backbone network Output feature f1∈R NK×C×H×W , the second to last floor Output and the third from the bottom Output features C, H, and W represent the channel dimension, height, and width of the feature map, respectively. Since the abstract semantic representation capability of the last layer feature output f1 of the backbone network is strong, the basic domain feature F∈R is obtained after average pooling. NK×C The upsampled high-level features are cascaded with the bottom-level features and then passed through the feature aggregation module. Finally, the aggregated features are pooled to obtain the fusion domain features M∈R NK×C The reason for using maximum pooling here is to suppress the noise information of the fused features. The feature aggregation module consists of three convolution blocks, each of which consists of a Conv-BN-ReLU structure, namely a convolution layer, a normalization layer, and an activation layer.

[0019] Finally, the class prototype of each sample category is calculated in the two feature domain spaces. The class prototype here can be obtained by taking the feature average of the same type of samples under each scenario task. The specific formula is as follows:

[0020]

[0021] P F ′∈R N×C Represents the basic domain prototype, P′ M ∈R N×C Represents a mixed domain prototype.

[0022] As an example, in step S4, the multi-domain prototypes are input into the multi-domain interaction structure, and the multi-domain interaction is guided by the inter-domain related information to complete the inter-domain information complementation. Specifically, the basic domain prototype P F ′ and fusion domain prototype P′ M Will be cascaded into the adaptive information sensor ω f ,ω f To perceive the effective inter-domain correlation information between different domain features, the formula is as follows:

[0023]

[0024]

[0025] Among them, θ f and θ m They are information sensor ω m and ω m The learnable parameters of It is a cascade operation. Then, adaptive weighting and residual structure are used to improve the representation ability of the base domain prototype and the fusion domain prototype. The formula is as follows:

[0026] P F =P F ′+λ s ·(1-s f (P F ′))·informaiton FM

[0027] P M =P′ M +λ s ·(1-s m (P′ M ))·informaiton MF

[0028] s f (P F ′) and s m (P′ M) are P obtained by Sigmoid activation function F ′ and P′ M The confidence score of the updated basic domain features and fusion domain features is expressed as P F ,P M ∈R NK×C It is worth noting that only the part with low confidence in the domain needs to be supplemented with inter-domain related information. In order to avoid the supplemented inter-domain related information from trivializing the original domain prototype dominant features, this module adds a scaling factor λ to the inter-domain related information. s .

[0029] The multi-domain interaction induced by inter-domain information proposed in the present invention enables the basic feature domain F focusing on abstract semantic information and the fusion feature domain M focusing on fine-grained information to supplement the common key information between different feature domains, improve the consistency of multi-domain prototypes, and make the constructed prototype representation more stable.

[0030] Preferably, in step S5, adaptive channel grouping is performed on the multi-domain prototypes based on the current scenario task, and each channel group within the domain is weighted through cross-domain channel group perception. This is done to perceive and highlight the key channel group feature information within each domain of the prototype, suppress the feature representation of irrelevant channel group information, and thus optimize the prototype representation.

[0031] Based on the basic domain prototype P F Take cross-domain perception as an example. Assuming that the number of prototype channel groups for each class is h, which is a hyperparameter, simulation tests show that when h = 8 in the Resnet-12 network, the model inference performance is optimal. First, a learnable linear mapping function Ω is used to obtain the neighborhood P M The self-sensing weight of each channel group is as follows:

[0032] W M =σ(Ω(P M θ Ω ))

[0033] Among them, σ is the Softmax activation function, θ Ω W is the learnable parameter of the linear mapping function Ω. M ∈R N×h×1 Represents the self-perception weight of each channel group of the fusion domain prototype, reflecting the representation ability of each fusion domain channel group when only the fusion domain features are considered. F Obtained through dimensional expansion Then the fusion domain self-perception weight W is used through matrix product operation M The basic domain prototype after dimension expansion The embedding vectors are weighted to obtain the cross-domain channel group weight. The specific formula is as follows:

[0034]

[0035] W Fg ∈R N×h The cross-domain channel group weights of each channel group of the base domain prototype affected by the fusion domain features are represented by T, which represents the transposition operation. The transposition operation of the high-dimensional matrix involved in the present invention is only performed on the last two dimensions. M Cross-domain channel group weight W Mg ∈R N×h Also obtained through the above steps.

[0036] Preferably, in step S6, a scenario attention mechanism is introduced to adjust the weight of the channel group within the domain using the inter-class relationship under each scenario task, thereby expanding the difference between prototypes of different classes under each classification scenario task.

[0037] Or based on the basic domain prototype P F For example, the specific steps are as follows: First, P F Through three weights {A Q ,A K ,A V}∈R C×C The linear mapping constructs the Query, Key and Value triples after dimension transformation {P Q ,P K ,P V}∈R h×N×m The specific announcement is as follows:

[0038] {P Q ,P K ,P V}=[PA Q ,PA K ,PA V ]

[0039] Where h is the number of channel groups, N is the number of support set sample categories for each scenario task, and m = C / h is the feature dimension contained in each channel group. Here, the channel group-based scenario attention adjustment is beneficial for capturing more inter-class information. The scenario attention weight formula can be expressed as:

[0040]

[0041] Where W episode ∈R h×N×N Indicates the contextual attention weight that focuses on inter-class difference information, Is the proportional factor to maintain the gradient stability. episodeBy adjusting the channel group weights, we can enhance the perception and highlight of key channel group information of the prototype while improving the model's adaptability to different scenarios. Contextual attention also helps to improve the differences between class prototypes and optimize prototype representation. The specific formula is as follows:

[0042]

[0043] W fusion ∈R h×N×N Represents the fusion channel group weight after scene attention adjustment, diag represents the W Fg The second dimension is expanded into a two-dimensional diagonal matrix operation. The prototype W adjusted by the scene attention fusion P V The final optimized prototype representation of the basic domain is obtained through dimension conversion and residual structure. The specific formula is as follows:

[0044] P F ′=LN(P F +W fusion P V )

[0045] P F ′∈R N×C is the prototype representation of the base domain after the contextual attention mechanism adjusts the cross-domain channel group perception, and LN represents the normalization layer. Similarly, the mixed domain prototype P′ adjusted by the contextual attention mechanism M Also obtained through the above steps.

[0046] Preferably, in step S7, deep training under small sample learning is performed using a designed loss function. The overall loss function includes two parts: multi-domain classification loss and channel group sparse loss.

[0047] Specifically, for the final prediction results, a multi-domain prototype metric structure is used. The metric function Φ is used to calculate the Euclidean distance between the prototype and the query feature in multiple feature domains, and then the KNN algorithm is used to generate the final prediction label. The specific formula is as follows:

[0048]

[0049] in Denotes the predicted label, Q F,j and Q M,j They represent the feature representation of the query set samples in the base domain and the fusion domain after passing through the backbone network, J represents the number of query set samples in each batch, and argmin represents selecting the class prototype with the smallest distance from the query feature. The classification loss function is expressed as follows:

[0050]

[0051] in represents the cross entropy loss function, y j is the true label.

[0052] In order to ensure the diversity of multi-domain prototype channel groups, the channel group sparsity is constrained. The channel group sparsity loss is constructed based on the cross-domain channel group weights, and the formula is as follows:

[0053]

[0054] This loss uses cosine similarity to calculate the similarity between multi-domain channel groups. When the similarity of the channel groups is large, will be large. This loss is encouraged by back propagation to minimize To preserve the sparsity between multi-domain prototype channel groups and thus ensure their diversity.

[0055] Combined with the above overall objective loss function is as follows:

[0056]

[0057] λ in the formula represents the loss balance coefficient.

[0058] Preferably, in step S8, during the iterative training of the small sample classification anomaly detection model, after each iteration of the complete training set, a test scenario task is randomly sampled from the test dataset to test the model's anomaly detection performance. The anomaly detection accuracy of the previously trained model weights on the test dataset is obtained. Ultimately, the training weight with the highest anomaly detection accuracy is retained.

[0059] The present invention also provides a small sample anomaly detection system based on multi-domain interaction construction and context-aware optimization, based on the above method, including the following modules:

[0060] A data conversion module, used to convert traffic anomaly video data in multiple road scenarios into labeled image data;

[0061] The small sample scenario construction module is used to preprocess the image data and construct scenario tasks based on the preprocessed image dataset according to the small sample learning training requirements. The support set and query set for model training and testing are obtained by random sampling from the image dataset respectively;

[0062] The multi-feature domain construction module uses the backbone network to extract the positive features of the input image data for each scenario task. The image data is mapped to multiple feature domain spaces according to the sequential feature embedding method through the feature output of different layers of the backbone network, and the multi-domain prototype is obtained by calculating the average feature value of similar samples.

[0063] The multi-domain interaction module inputs the multi-domain prototypes into the multi-domain interaction structure and achieves inter-domain information complementation through multi-domain interaction guided by inter-domain related information. This complementation enhances the consistency of multi-domain features.

[0064] The cross-domain channel group perception module adaptively groups multi-domain features into channels based on the current scenario task. It weights each channel group within the domain through cross-domain channel group perception to highlight the key information representation within each domain of the prototype.

[0065] The context attention adjustment module is used to introduce the context attention mechanism to adjust the weight of the channel group in the domain by utilizing the inter-class relationship under each context task, thereby expanding the difference between class prototypes under each classification context task;

[0066] Network training module, used to train deep learning networks using loss functions for small-sample learning;

[0067] The testing module uses the test set data to test the performance of the small sample classification model and retains the model training parameters with the best anomaly detection performance.

[0068] Compared with the prior art, the present invention has the following beneficial effects:

[0069] (1) The present invention only requires a small number of traffic anomaly labeled image samples to perform deep learning and then train the anomaly detection model. It does not require a large amount of data collection and data labeling work, can quickly respond to new traffic anomaly detection scenarios, and facilitates fast and reliable deployment of new scenarios.

[0070] (2) The present invention maps image feature information to multiple feature domain spaces through the proposed sequential feature embedding, which is conducive to deep mining of available feature information of label data; and improves the consistency of multi-domain features through the multi-domain interaction module, avoiding the interference of multi-domain prototype measurement result conflicts on model reasoning performance, and multi-domain interaction completes inter-domain information complementation by mining inter-domain related information, which is conducive to enhancing prototype representation.

[0071] (3) The present invention designs an adaptive channel group perception prototype optimization structure that focuses on small sample training scenarios. It highlights the key information representation within the class through cross-domain channel group perception, and adjusts the perception of the cross-domain channel group by introducing scenario attention. This allows the final prototype optimization to highlight the important feature representation within the class while also paying attention to the inter-class relationship under the scenario task, improving the differences between class prototypes and preventing the appearance of similar prototypes of different classes during the prototype optimization process. The scenario-based prototype optimization method proposed by the present invention improves the scenario adaptability of anomaly detection under small sample learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1This is a flowchart of a small sample anomaly detection method based on multi-domain interaction construction and context-aware optimization in a preferred embodiment of the present invention;

[0073] Figure 2 A flowchart of multi-domain construction based on a sequential feature embedding method according to a preferred embodiment of the present invention;

[0074] Figure 3 This is a schematic diagram of the multi-domain interaction principle guided by inter-domain related information according to a preferred embodiment of the present invention;

[0075] Figure 4 This is a schematic diagram of the principle of adaptive grouping perception of basic domain prototypes based on current scenario tasks in a preferred embodiment of the present invention;

[0076] Figure 5 This is a block diagram of a small sample anomaly detection system based on multi-domain interaction construction and context-aware optimization in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0077] To more clearly illustrate the embodiments of the present invention, specific embodiments of the present invention will be described below with reference to the accompanying drawings. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings and other embodiments can be obtained based on these drawings without inventive efforts.

[0078] like Figure 1 As shown, this embodiment provides a small sample anomaly detection method based on multi-domain interaction construction and context-aware optimization, which specifically includes the following steps:

[0079] Step 1: The original traffic anomaly video data includes a total of 10 video streams in seven scenes. Among them, the scenes include data of a residential gate (A), two downtown gates (B, C), three national highways (D, E, F), and a park gate (G). Based on the observation of the entire video stream, this embodiment found that abnormal behaviors include changing lanes, non-motor vehicles driving on motor vehicle lanes, vehicles running red lights, pedestrians running red lights and other traffic anomalies. Specifically, the video stream segments are sampled according to the sampling standard of 4 frames per second, and then labeled according to the traffic scene and traffic conditions. The final labeled image dataset D label It includes seven normal scene categories and 16 abnormal scene categories. According to the ratio of 7:3, the labeled image dataset D label Divide into training set D train and the test set D test .

[0080] Step 2: Preprocess the input image dataset to reduce the likelihood of the model learning irrelevant information. This preprocessing includes cropping and grayscale conversion. Specifically, the original image (980*1320) is first cropped to remove the distant scene, and then scaled to a final image size of (256*256). The original RGB three-channel image dataset is then converted to a single-channel image dataset to prevent scene color from interfering with anomaly detection during model training.

[0081] Then, the preprocessed dataset is used to construct scenario tasks that are suitable for small sample learning. The scenario task Episode is constructed as follows: In the training phase, from the training set D train Randomly select N classes and randomly select K sample values ​​for each class. Here, N=5 and K=5 conform to the standard small sample learning paradigm 5-way 1shot. This constitutes the support set Select B samples from other samples of the current N classes as the query set The value of B in the actual project is 75. The final scenario task can be expressed as Episode = {S, Q}. i ,y i is the support set image samples and corresponding labels, x j ,y j is the query set image sample and the corresponding label, B represents the number of samples of each category in the query set. Note that the support set S and the query set Q satisfy But share the same label space.

[0082] Step 3: Build a small-sample deep learning framework in PyTorch to complete the traffic anomaly detection task. The backbone consists of a ResNet-12 feature extraction network. ResNet-12 is a 12-layer residual network. Note that the last fully connected layer of ResNet-12 has been removed. The channel group perceptron consists of linear layers. The metric function Φ uses the square of the Euclidean distance.

[0083] like Figure 2 As shown, the image is input into the backbone network to obtain the final three-layer feature output, and multi-domain features are obtained by sequential embedding, including basic domain features and fusion domain features. Specifically, assuming that the last layer of the backbone network outputs feature f1∈R NK×C×H×W , the penultimate layer output And the output features of the third layer from the bottom C, H, and W represent the channel dimension, height, and width of the feature map, respectively. Because the last three layers of the Resnet-12 network are selected, C = 640, H = 16, and W = 16. Because the abstract semantic representation ability of the last layer feature output f1 of the backbone network is strong, average pooling is selected to obtain the basic domain feature F∈R NK×C , where the average pooling layer parameter is (640,1). Let the upsampled high-level features be cascaded with the bottom-level features and then pass through the feature embedding module, and finally perform maximum pooling on the aggregated features to obtain the fusion domain feature M∈R NK×C , where the maximum pooling layer parameters are (960, 1). The reason for using maximum pooling here is to suppress the noise information of the fused features. The feature embedding module consists of three convolutional blocks, each of which consists of a Conv-BN-ReLU structure, namely a convolutional layer, a normalization layer, and an activation layer.

[0084] Finally, the class prototype of each sample category is calculated in the two feature domain spaces. The class prototype can be obtained by taking the feature average of the same type of samples under each scenario task. The specific formula is as follows:

[0085]

[0086] P F ′∈R N×C Represents the basic domain prototype, P′ M ∈R N×C Represents a mixed domain prototype.

[0087] Step 4: If Figure 3 As shown in the figure, the multi-domain prototype is input into the multi-domain interaction structure, and the multi-domain interaction is guided by the inter-domain related information to complete the inter-domain information complementation. It can be seen that the confidence scores of the features in each domain are different before the multi-domain interaction. It is believed that the feature parts with low information need to be supplemented with inter-domain information to enhance their representation ability. Therefore, the basic domain prototype P F ′ and fusion domain prototype P′ M Will be cascaded into the adaptive information sensor ω f ,ω f To perceive the effective inter-domain correlation information between different domain features, the formula is as follows:

[0088]

[0089]

[0090] Among them, θ f and θ m They are information sensor ω m and ω m The learnable parameters are implemented in the actual network as two convolutional layers with parameters (1280, 640). It is a cascade operation. Then, adaptive weighting and residual structure are used to improve the representation ability of the base domain prototype and the fusion domain prototype. The formula is as follows:

[0091] P F =P F ′+λ s ·(1-s f (P F ′))·informaiton FM

[0092] P M =P′ M +λ s ·(1-s m (P′ M ))·informaiton MF

[0093] Among them, s f (P F ′) and s m (P′ M ) are P obtained by Sigmoid activation function F ′ and P′ M The confidence score of the updated basic domain features and fusion domain features is expressed as P F ,P M ∈R NK ×C It is worth noting that only the part with low confidence in the domain needs to be supplemented with inter-domain related information. In order to avoid the supplemented inter-domain related information from trivializing the original domain prototype dominant features, this module adds a scaling factor λ to the inter-domain related information. s .

[0094] like Figure 3 As shown in the figure, after multi-domain interaction, the originally low-confidence features in each domain are supplemented by inter-domain related information, and their confidence scores are improved, indicating that their representational capabilities have been enhanced. The proposed multi-domain interaction induced by inter-domain information allows the basic feature domain F, which focuses on abstract semantic information, and the fused feature domain M, which focuses on fine-grained information, to supplement the common key information between different feature domains, improving the consistency of multi-domain prototypes and making the constructed prototype representation more stable.

[0095] Step 5: Adaptively group channels of multi-domain prototypes based on the current scenario task, and weight each channel group within the domain through cross-domain channel group perception. This allows perception to highlight the key channel group feature information within each domain of the prototype, suppress the feature representation of irrelevant channel group information, and thus optimize the prototype representation.

[0096] like Figure 4As shown, taking the channel group perception of the base domain as an example, it can be seen that the results of adaptive channel grouping of the base domain prototype based on different scenario tasks are different, which shows the effectiveness of scenario-based adaptive channel grouping. Then, the cross-domain channel group perception is used to weight the channel groups within the domain, highlight the key channel group information representation and suppress the irrelevant channel group information representation, which is conducive to improving the information representation of the class prototype and is beneficial to subsequent anomaly detection. Specifically, assuming that the number of channel groups for each class prototype is h, which is a hyperparameter, simulation tests show that when h = 8 in the Resnet-12 network, the model inference performance is optimal. First, a learnable linear mapping function Ω is used to obtain the neighborhood P M The self-sensing weight of each channel group is as follows:

[0097] W M =σ(Ω(P M θ Ω ))

[0098] Among them, σ is the Softmax activation function, θ Ω is the learnable parameter of the linear mapping function Ω, which is specifically implemented as a fully connected layer with parameters (640, 8). M ∈R N×h×1 Represents the self-perception weight of each channel group of the fusion domain prototype, reflecting the representation ability of each fusion domain channel group when only the fusion domain features are considered. F Obtained through dimensional expansion Then the fusion domain self-perception weight W is used through matrix product operation M The basic domain prototype after dimension expansion The embedding vectors are weighted to obtain the cross-domain channel group weight. The specific formula is as follows:

[0099]

[0100] Among them, W Fg ∈R N×h The cross-domain channel group weights of each channel group of the base domain prototype affected by the fusion domain features are represented by , and T represents the transposition operation. In this embodiment, the transposition operation of the high-dimensional matrix is ​​only performed on the last two dimensions. M Cross-domain channel group weight W Mg ∈R N×h Also obtained through the above steps.

[0101] Step 6: Introduce the contextual attention mechanism to use the inter-class relationship under each contextual task to adjust the weight of the channel group in the domain, and expand the difference between different class prototypes under each classification context task. F For example, the specific steps are as follows: First, P FThrough three weights {A Q ,A K ,A V}∈R C×C The linear mapping constructs the Query, Key and Value triples after dimension transformation {P Q ,P K ,P V}∈R h×N×m The specific announcement is as follows:

[0102] {P Q ,P K ,P V}=[PA Q ,PA K ,PA V ]

[0103] Where h is the number of channel groups, N is the number of support set sample categories for each scenario task, and m = C / h is the feature dimension contained in each channel group. Here, the channel group-based scenario attention adjustment is beneficial for capturing more inter-class information. The scenario attention weight formula can be expressed as:

[0104]

[0105] Among them, W episode ∈R h×N×N Indicates the contextual attention weight that focuses on inter-class difference information, Is the proportional factor to maintain the gradient stability. episode By adjusting the channel group weights, we can enhance the perception and highlight of key channel group information of the prototype while improving the model's adaptability to different scenarios. Contextual attention also helps to improve the differences between class prototypes and optimize prototype representation. The specific formula is as follows:

[0106]

[0107] Among them, W fusion ∈R h×N×N Represents the fusion channel group weight after scene attention adjustment, diag represents the W Fg The second dimension is expanded into a two-dimensional diagonal matrix operation. The prototype W adjusted by the scene attention fusion P V The final optimized prototype representation of the basic domain is obtained through dimension conversion and residual structure. The specific formula is as follows:

[0108] P F ′=LN(P F +W fusion P V )

[0109] Among them, PF ′∈R N×C is the prototype representation of the base domain after the contextual attention mechanism adjusts the cross-domain channel group perception, and LN represents the normalization layer. Similarly, the mixed domain prototype P′ adjusted by the contextual attention mechanism M Also obtained through the above steps.

[0110] Through the above steps, we can obtain the optimal prototype of each traffic anomaly class in multiple feature domain spaces, which will help the subsequent anomaly detection work.

[0111] Step 7: Perform deep training under small sample learning using the designed loss function. The overall loss function consists of two parts: multi-domain classification loss and channel group sparse loss.

[0112] Specifically, a multi-domain prototype metric structure was used to generate the final prediction results. The metric function Φ was used to calculate the Euclidean distance between the prototype and the query feature across multiple feature domains. The final prediction label was then generated using the KNN algorithm. The number of query samples was 75, and five categories were randomly selected for each scenario task. The specific formula is as follows:

[0113]

[0114] in, Denotes the predicted label, Q F,j and Q M,j They represent the feature representation of the query set samples in the base domain and the fusion domain after passing through the backbone network, J=75 represents the number of query set samples in each batch, and argmin represents selecting the class prototype with the smallest distance from the query feature. The classification loss function is expressed as follows:

[0115]

[0116] in, represents the cross entropy loss function, y j is the true label.

[0117] In order to ensure the diversity of multi-domain prototype channel groups, the channel group sparsity is constrained. The channel group sparsity loss is constructed based on the cross-domain channel group weights, and the formula is as follows:

[0118]

[0119] This loss uses cosine similarity to calculate the similarity between multi-domain channel groups. When the similarity of the channel groups is large, will be large. This loss is encouraged by back propagation to minimize To preserve the sparsity between multi-domain prototype channel groups and thus ensure their diversity.

[0120] Combined with the above overall objective loss function is as follows:

[0121]

[0122] The λ in the formula represents the loss balance coefficient, where λ = 0.8.

[0123] Step S8: During the iterative training of the small-sample classification anomaly detection model, after each iteration of the complete training set, a test scenario task is randomly sampled from the test dataset to test the model's anomaly detection performance. The anomaly detection accuracy of the previously trained model weights on the test dataset is obtained. The training weights with the highest anomaly detection accuracy are retained, and the model is saved as a .pth file. The specific training strategy is as follows: the total number of scenario tasks is set to 100,000. During training, the SGD optimization algorithm is used as the optimizer, with an initial learning rate of 0.0002, which is halved every 5,000 iterations, resulting in a weight decay of 0.0005.

[0124] The final anomaly detection accuracy (ACC) in seven road scenarios is shown in Table 1 below:

[0125] Table 1. Accuracy test results of traffic abnormality in 7 scenarios

[0126] Traffic scenes A B C D E F G ACC (%) 0.815 0.731 0.642 0.838 0.855 0.779 0.829

[0127] The classification accuracy ACC is an important indicator for anomaly detection. A larger ACC means that the anomaly detection model can better distinguish between normal traffic behavior categories and abnormal traffic behavior categories. From the test results in the table, it can be seen that the present invention can classify normal traffic behavior and abnormal traffic behavior in seven traffic scenarios. However, the accuracy in the two downtown scenes B and C is lower than the overall accuracy. Analysis of the abnormal traffic behaviors in the downtown scene is mainly pedestrians jaywalking, and the main detection is the traffic behavior of pedestrians. In the entire image, the human body accounts for a relatively small proportion, resulting in the semantic features of pedestrians obtained when the backbone network is used to extract image features. It is relatively vague, which makes the model unable to learn the abnormal traffic behavior of pedestrians well, resulting in poor final detection performance. In other scenarios, abnormal behavior detection can be achieved for abnormal behaviors of vehicles (such as running red lights, vehicles crossing solid lines and illegally changing lanes, and vehicle collisions). Detection performance in Scenario F was relatively low. This was attributed to the fact that the original video stream provided by Scenario F consisted of a short 23-minute video with very few abnormal behaviors. Consequently, the resulting dataset of abnormal behavior images for Scenario F was relatively small, preventing the model from effectively learning the differences between abnormal and normal traffic behavior in Scenario F. In other words, the low anomaly detection performance in Scenario F was due to insufficient training data.

[0128] like Figure 5 As shown, this embodiment provides a small sample anomaly detection system based on multi-domain interaction construction and context-aware optimization, based on the above method embodiment, including the following modules:

[0129] A data conversion module is used to convert the existing traffic anomaly video data under multiple road scenarios into labeled image data;

[0130] The small sample scenario construction module is used to preprocess the image data and construct scenario tasks based on the preprocessed image dataset according to the small sample learning training requirements. The support set and query set for model training and testing are obtained from the image dataset through random sampling.

[0131] The multi-feature domain construction module uses the backbone network to extract the positive features of the input image data for each scenario task. The image data is mapped to multiple feature domain spaces according to the sequential feature embedding method through the feature output of different layers of the backbone network, and the multi-domain prototype is obtained by calculating the average feature value of similar samples.

[0132] The multi-domain interaction module inputs the multi-domain prototypes into the multi-domain interaction structure and achieves inter-domain information complementation through multi-domain interaction guided by inter-domain related information. This complementation enhances the consistency of multi-domain features.

[0133] The cross-domain channel group perception module is used to adaptively group multi-domain features into channels based on the current scenario task, and weight each channel group within the domain through cross-domain channel group perception to highlight the key information representation in each domain of the prototype;

[0134] The context attention adjustment module is used to introduce the context attention mechanism to adjust the weight of the channel group in the domain by utilizing the inter-class relationship under each context task, thereby expanding the difference between class prototypes under each classification context task;

[0135] Network training module, used to train deep learning networks using loss functions for small-sample learning;

[0136] The testing module is used to test the performance of the small sample classification model using the test set data and retain the model training parameters with the best anomaly detection performance.

[0137] For other contents of this embodiment, please refer to the above method embodiment.

[0138] In summary, the present invention proposes a small sample anomaly detection method and system based on multi-domain interaction construction and scenario perception optimization, the method comprises the following steps: step 1, converting the existing traffic anomaly videos under multiple road scenarios into labeled image datasets; step 2, preprocessing the image datasets, and constructing corresponding scenario tasks, including support sets and query sets, from the preprocessed image datasets according to the small sample learning training requirements; step 3, inputting the images of each scenario task into the backbone network for feature extraction, and using the sequential feature embedding method to map the image features to multiple feature domain spaces, and obtaining the multi-domain prototype by calculating the average value of the features of similar samples; step 4, The domain prototypes are input into the multi-domain interaction structure, and the inter-domain information complementation is achieved through the multi-domain interaction guided by the inter-domain related information; Step 5, based on the current scenario task, the multi-domain prototypes are adaptively grouped into channels, and each channel group in the domain is weighted through cross-domain channel group perception to highlight the key information representation of each class prototype; Step 6, the scenario attention mechanism is introduced to adjust the weight of the channel group in the domain using the inter-class relationship under each scenario task, and expand the difference between different class prototypes under each classification scenario task; Step 7, deep training is performed under small sample learning through the loss function; Step 8, the performance of the trained small sample classification model is tested, and the model training parameters with the best inference performance are retained. The present invention can achieve classification detection of abnormal traffic behavior by assisting deep model training with only a small amount of labeled data, which helps to quickly adapt to new traffic anomaly detection scenarios, and the present invention improves the consistency and scenario adaptability of multi-domain features in metric-based small sample learning.

[0139] The above are merely preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will appreciate that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions are possible for those skilled in the art without departing from the scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the scope of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A small sample anomaly detection method based on multi-domain interaction construction and context-aware optimization, characterized by: Follow these steps: S1. Convert traffic anomaly videos in multiple road scenarios into labeled image datasets; S2. Preprocess the image dataset and construct corresponding scenario tasks based on the preprocessed image dataset according to the small sample learning training requirements, including support sets and query sets; S3. Input the image of each scenario task into the backbone network for feature extraction. Use the sequential feature embedding method to map the image features into multiple feature domain spaces, and obtain the multi-domain prototype by calculating the average feature value of similar samples. S4. Input the multi-domain prototypes into the multi-domain interaction structure, and achieve inter-domain information complementation through multi-domain interaction guided by inter-domain related information. This step is as follows: Input the multi-domain prototype into the multi-domain interaction structure, guide the multi-domain interaction through the inter-domain related information to complete the inter-domain information complementation; input the basic domain prototype into the multi-domain interaction structure, guide the multi-domain interaction through the inter-domain related information to complete the inter-domain information complementation; and fusion domain prototypes Will be cascaded into the adaptive information sensor , It is used to perceive the effective inter-domain correlation information between different domain features. The formula is as follows: in, Indicates inter-domain related information perceived by the basic domain, Represents inter-domain related information of hybrid domain awareness, and Information Sensor and The learnable parameters are implemented as two convolutional layers with parameters of (1280, 640) in the actual network. It is a cascade operation; then adaptive weighting and residual structure are used to improve the representation ability of the base domain prototype and the fusion domain prototype. The formula is as follows: in, and They are obtained through the Sigmoid activation function and The confidence score of , and the adaptive weight is obtained according to the confidence score; the updated basic domain prototype and fusion domain prototype are expressed as ; ; S5. Adaptively group channels of multi-domain prototypes based on the current scenario task, weight each channel group within the domain through cross-domain channel group perception, and highlight the key information representation of each class prototype; S6. Introducing a contextual attention mechanism to use the inter-class relationship under each contextual task to adjust the weight of the channel group within the domain, thereby expanding the differences between prototypes of different classes under each classification contextual task; S7. Deep training with small sample size learning using loss functions; S8. Perform performance tests on the trained small sample classification model and retain the model training parameters with the best inference performance.

2. The small sample anomaly detection method based on multi-domain interaction construction and context-aware optimization according to claim 1 is characterized by: In step S1, the traffic anomaly video is converted into an image dataset according to the sampling strategy of four frames per second, and labeled according to the scene and road traffic conditions to produce a labeled image dataset ; Divide the labeled image dataset for each scene into a training set in a ratio of 7:3 and test set .

3. The small sample anomaly detection method based on multi-domain interaction construction and context-aware optimization according to claim 1 or 2 is characterized by: In step S2, the input image dataset is preprocessed, including cropping and grayscale change. Specifically, the video perspective of the original image is first cropped, and then the image is scaled. The original RGB three-channel image dataset is converted into a single-channel image dataset through grayscale change.

4. The small sample anomaly detection method based on multi-domain interaction construction and context-aware optimization according to claim 3 is characterized by: In step S2, after preprocessing the data set, the corresponding traffic anomaly detection scenario task is constructed according to the small sample learning requirements; the Episode is constructed as follows: In the training phase, Randomly select N=5 classes, and randomly select K=5 sample values ​​for each class to form the support set , and then select B=75 samples from other samples of the current N classes as the query set , the value of B in the actual project is 75; the final scenario task is expressed as ;in is the support set of image samples and corresponding labels, are the query set image samples and corresponding labels, and B represents the number of samples of each category in the query set.

5. The small sample anomaly detection method based on multi-domain interaction construction and context-aware optimization according to claim 4 is characterized by: In step S3, a small sample deep learning framework is constructed under the PyTorch framework to complete the traffic anomaly detection task. The backbone is divided into a ResNet-12 feature extraction network. ResNet-12 is a 12-layer residual network, and the last fully connected layer of ResNet-12 is deleted. The image is input into the backbone network to obtain the final three-layer feature output, and multi-domain features are obtained by sequential embedding, including basic domain features and fusion domain features; specifically, assuming that the last layer of the backbone network outputs features , the penultimate layer output And the output features of the third layer from the bottom , R represents a standard matrix, the superscript of R represents the dimension of the matrix, and the dimension of the description vector is represented by this paradigm. C, H, and W represent the channel dimension, height, and width of the feature map respectively. Because the last three layers of the Resnet-12 network are selected, C=640, H=16, and W=16 here, because the last layer of the backbone network outputs The abstract semantic representation ability is strong, so the average pooling is selected to obtain the basic domain feature F , where the average pooling layer parameter is (640, 1); let the upsampled high-level features and the bottom-level features be cascaded and passed through the feature embedding module, and finally the embedded features are max-pooled to obtain the fusion domain feature M , where the maximum pooling layer parameter is (960, 1); the feature embedding module consists of three convolution blocks, each of which consists of a Conv-BN-ReLU structure, namely a convolution layer, a normalization layer and an activation layer; Finally, the average value of the features of similar samples is calculated in the two feature domain spaces to obtain the class prototype. The specific formula is as follows: in, Represents the basic domain prototype, Represents a hybrid domain prototype; Represents all basic domain features belonging to the same category under the same scenario task, Represents all mixed domain features belonging to the same category under the same scenario task; where K represents the number of samples contained in each category under a scenario task.

6. The small sample anomaly detection method based on multi-domain interaction construction and context-aware optimization according to claim 4 is characterized by: Step S5 is specifically as follows: Based on the basic domain prototype As an example of cross-domain perception; assuming that the number of prototype channel groups for each class is h; first, a learnable linear mapping function Get neighborhood The self-sensing weight of each channel group is as follows: in, is the Softmax activation function, is a linear mapping function The learnable parameters of Represents the self-perception weight of each channel group of the fusion domain prototype; Obtained through dimensional expansion ; Then the fusion domain self-perception weights are used through matrix product operation The basic domain prototype after dimension expansion The embedding vector is assigned weights to obtain the cross-domain channel group weights; the specific formula is as follows: in, Represents the cross-domain channel group weights of each channel group of the base domain prototype affected by the fusion domain features, T represents the transposition operation; the hybrid domain prototype affected by the base domain features Cross-domain channel group weight Also obtained through step S5.

7. The small sample anomaly detection method based on multi-domain interaction construction and context-aware optimization according to claim 6 is characterized by: Step S6 is specifically as follows: Based on the basic domain prototype For example, first Through three linear mapping matrices , construct the query after dimension conversion ,key Sum Triplet The specific announcement is as follows: Among them, h is the number of channel groups, N is the number of support set sample categories under each scenario task, is the feature dimension contained in each channel group; the scenario attention weight formula is expressed as: in, Indicates the contextual attention weight that focuses on inter-class difference information, is the proportional factor to maintain the gradient stability; the obtained Adjust the channel group weight; the specific formula is as follows: in, Represents the fusion channel group weight after scene attention adjustment, diag represents The second dimension is expanded into a two-dimensional diagonal matrix operation; the prototype adjusted by the context attention The final optimized basic domain prototype representation is obtained through dimension conversion and residual structure; the specific formula is as follows: in, Represents the updated basic domain prototype, It is the prototype representation of the basic domain after the cross-domain channel group perception adjustment by the contextual attention mechanism. Representation normalization layer; mixed domain prototype conditioned by contextual attention mechanism Also obtained through step S6.

8. The small sample anomaly detection method based on multi-domain interaction construction and context-aware optimization according to claim 7 is characterized by: Step S7 is specifically as follows: The overall loss function consists of two parts: multi-domain classification loss and channel group sparsity loss; For the final prediction results, a multi-domain prototype measurement structure is adopted; using the measurement function Calculate the Euclidean distance between the prototype and the query feature under multiple feature domains, and then use the KNN algorithm to generate the final predicted label. The specific formula is as follows: in, Represents the metric function for calculating Euclidean distance, represents the predicted label, and They represent the feature representations of the query set samples in the basic domain and the fusion domain after passing through the backbone network, J represents the number of query set samples in each batch, Indicates that the class prototype with the smallest distance from the query feature is selected; the classification loss function is expressed as follows: in, represents the cross entropy loss function, is the true label, Represents the deep model inference prediction label; In order to ensure the diversity of multi-domain prototype channel groups, the sparsity of the channel group is constrained; the channel group sparse loss is constructed based on the cross-domain channel group weights, and the formula is as follows: , Represents a hybrid domain prototype influenced by the characteristics of the base domain Cross-domain channel group weight, this loss uses cosine similarity to calculate the similarity between multi-domain channel groups. When the similarity of the channel group is large, will be large; this loss is encouraged by backpropagation to minimize To preserve the sparsity between multi-domain prototype channel groups; Combined with the above overall objective loss function is as follows: in, Represents the loss balance coefficient.

9. A small sample anomaly detection system based on multi-domain interactive construction and context-aware optimization, used to implement the small sample anomaly detection method based on multi-domain interactive construction and context-aware optimization according to any one of claims 1 to 8, characterized in that: The small sample anomaly detection system based on multi-domain interaction construction and context-aware optimization specifically includes the following modules: A data conversion module, used to convert traffic anomaly video data in multiple road scenarios into labeled image data; The small sample scenario construction module is used to preprocess image data and construct scenario tasks based on the preprocessed image dataset according to the small sample learning training requirements. The support set and query set for model training and testing are obtained from the image dataset through random sampling. The multi-feature domain construction module uses the backbone network to extract the positive features of the input image data for each scenario task. The image data is mapped to multiple feature domain spaces according to the sequential feature embedding method through the feature output of different layers of the backbone network, and the multi-domain prototype is obtained by calculating the average feature value of similar samples. The multi-domain interaction module inputs the multi-domain prototypes into the multi-domain interaction structure and completes the inter-domain information complementation through the multi-domain interaction guided by the inter-domain related information; The cross-domain channel group perception module adaptively groups multi-domain features into channels based on the current scenario task, and weights each channel group within the domain through cross-domain channel group perception to highlight the key information representation within each domain of the prototype; The context attention adjustment module is used to introduce the context attention mechanism to adjust the weight of the channel group in the domain by utilizing the inter-class relationship under each context task, thereby expanding the difference between class prototypes under each classification context task; Network training module, used to train deep learning networks using loss functions for small-sample learning; The testing module uses the test set data to test the performance of the small sample classification model and retains the model training parameters with the best anomaly detection performance.

Images