Weakly supervised anomaly detection method based on multi-scale feature fusion and multiple attention combinations

Abstract

The present invention relates to a weakly supervised anomaly detection method combining multi-scale feature fusion and multiple attentions. Normal and abnormal videos are divided into T video segments, and the spatiotemporal features in each video segment are extracted through an I3D convolutional network. Then, a multi-scale feature fusion network that is conducive to the expression of spatiotemporal features is used to obtain fused multi-scale features from the spatiotemporal features. Then, a multi-head self-attention mechanism and a channel attention mechanism are added to the temporal dimension and channel dimension of the multi-scale features respectively to realize the global temporal dependency between video segments and the information on different channel features. Then, an anomaly score of each video segment is output through a fully connected layer. Finally, a target loss function is constructed according to the output anomaly score, so that the entire weakly supervised anomaly detection process is more easily converged to the optimal state.

Description

Technical Field

[0001] The present invention relates to the technical field of video anomaly detection, and in particular to a weakly supervised anomaly detection method combining multi-scale feature fusion and multiple attentions. Background Art

[0002] Video anomaly detection aims to automatically detect and locate abnormal behavior events in video data through algorithms. It can be applied to traffic management, public security, environmental monitoring and other fields. Due to the characteristics of video such as complex backgrounds, changing scenes, and large data volumes, video anomaly detection is more complex than general time series anomaly detection and faces the following four challenges: 1) Unclear definition of abnormal events;

[0003] 2) The number of normal and abnormal events is highly unbalanced; 3) Abnormal events are diverse; and 4) Event categories are scene-dependent. Due to the diversity and scene-dependence of abnormal events, obtaining accurate data labels is difficult, especially precise segment-level labels, which involves splitting the video into several segments of equal length and accurately labeling each segment.

[0004] Based on the development stage of video anomaly detection methods, existing approaches can be categorized into three categories: traditional machine learning, hybrid methods, and deep learning. The first two require manual modeling of abnormal events, resulting in complex computational processes and limited application scenarios. Based on the discriminative mechanism, deep learning methods can be further divided into four categories: reconstruction, prediction, single-class neural networks, and multiple-instance learning. Multiple-instance learning, which uses video-level labels to train models, can accurately locate the frame locations where abnormal behavior occurs. It is a weakly supervised approach and offers better performance and greater robustness than the first three unsupervised approaches. Furthermore, video-level labels are easier to obtain and less expensive than segment-level labels. Therefore, multiple-instance learning models are more practical than supervised methods that use detailed segment-level labels for model training and can be used for abnormal behavior detection in real-world scenarios. However, weakly supervised anomaly detection methods primarily rely on multiple-instance learning. The sparse processing of anomaly scores for abnormal videos in these weakly supervised anomaly detection methods is only suitable for cases where the total anomaly score is small. This sparse processing approach is inappropriate for cases with high anomaly rates, and further improvement is needed. Summary of the Invention

[0005] The problem solved by the present invention is how to improve the problem that the sparse loss function of the multi-instance learning method is imperfect due to the high anomaly rate.

[0006] To solve the above problems, the present invention provides a weakly supervised anomaly detection method combining multi-scale feature fusion and multiple attentions, including:

[0007] Step 1: Collect a training video set, which includes normal videos and abnormal videos;

[0008] Step 2: Split the normal video into negative packets containing T non-overlapping video segments The abnormal video is divided into positive packets containing T non-overlapping video segments Each segment contains c consecutive frames;

[0009] Step 3: Use I3D convolutional network to extract positive packets V n and negative package V a The spatiotemporal features X of each video clip in n and X a ; And use multi-scale feature fusion network to align the package V n and negative package V a The spatiotemporal characteristics X n , spatiotemporal characteristics X a Perform feature fusion separately to obtain the multi-scale feature Y of the positive package n and the multi-scale feature Y of the negative bag a ;

[0010] Step 4: Use the multi-head self-attention mechanism to respectively analyze the multi-scale features Y of the positive packet. n and the multi-scale feature Y of the negative bag a Add attention to obtain the multi-scale feature F with attention mechanism n and F a ;

[0011] Step 5: Use the channel attention mechanism to separately add the multi-scale features F n and F a Perform dimensionality reduction, dimensionality increase, sigmoid function to calculate channel attention weights and add multi-scale features F to the attention mechanism n and F a After each channel is weighted, a multi-scale feature Z with multiple attention mechanisms is obtained. n and Z a ;

[0012] Step 6: Use a fully connected layer network to add multiple attention mechanisms to the multi-scale feature Z n and Z a Perform linear regression and get V n and V a The anomaly score of each segment in S n and S a ;

[0013] Step 7: Package S based on the anomaly score nand S a Construct the target loss function to optimize the parameters of steps 4 to 6.

[0014] The beneficial effects of the present invention are as follows: normal videos and abnormal videos are divided into T video clips, the spatiotemporal features in each video clip are extracted through an I3D convolutional network, and then the fused multi-scale features are obtained from the spatiotemporal features through a multi-scale feature fusion network that is conducive to the expression of spatiotemporal features. Then, a multi-head self-attention mechanism and a channel attention mechanism are added to the temporal dimension and channel dimension of the multi-scale features respectively to realize the global temporal dependency between video clips and the information on different channel features. Then, the anomaly score of each video clip is output through a fully connected layer. Finally, a target loss function is constructed according to the output anomaly score, so that the entire weakly supervised anomaly detection process is more easily converged to the optimal.

[0015] As a preference, in step 3, a multi-scale feature fusion network is used to align the envelope V n and negative package V a The spatiotemporal characteristics X n , spatiotemporal characteristics X a Perform feature fusion separately to obtain the multi-scale feature Y of the positive package n and the multi-scale feature Y of the negative bag a Specifically include:

[0016] S301, use m one-dimensional convolution kernels to respectively process the spatiotemporal features X n , spatiotemporal characteristics X a For convolution operation, the size of the one-dimensional convolution kernel is l, and its expressions are:

[0017] Y1 n =conv1(X n , l, m);

[0018] Y1 a =conv1(X a , l, m);

[0019] Step 302: Use m one-dimensional convolution kernels to respectively transform the spatiotemporal features X n , spatiotemporal characteristics X a Perform the convolution operation again, the size of the one-dimensional convolution kernel is l', and its expressions are:

[0020] Y2 n =conv1(X n , l′, m);

[0021] Y2 a =conv1(X a , l′, m);

[0022] The features of the convolution outputs in step 303 and step 301 and the features of the convolution output in step 302 are fused with the spatiotemporal features to obtain multi-scale features, whose expressions are:

[0023] Y n =[X n , Y1 n , Y2 n ];

[0024] Y a =[X a , Y1 a , Y2 a ].

[0025] As a preference, in step 4, a multi-head self-attention mechanism is used to respectively focus on the multi-scale features Y of the positive packet. n and the multi-scale feature Y of the negative bag a Add attention to obtain the multi-scale feature F with attention mechanism n and F a Specifically include:

[0026] Step 401: Use a fully connected layer to transform the multi-scale feature Y n and Y a Converted into shared matrix Y n (c) and Y a (c);

[0027] Step 402: Share the matrix Y n (c) and Y a (c) Generate q respectively n 、k n 、v n and q a 、k a 、v a , the calculation formula is:

[0028] q n =W Q Y n (c);

[0029] k n =W K Y n (c);

[0030] v n =W V Y n (c);

[0031] q a =W Q Y a (c);

[0032] ka =W K Y a (c);

[0033] v a =W V Y a (c);

[0034] Where W Q 、W K 、W V is the weight matrix;

[0035] Step 403: According to the number of heads of multi-head self-attention, n 、k n 、v n and q a 、k a 、v a Split into and

[0036] Step 404: and Transpose and multiply, then use the softmax function to get the self-attention map M n ; Then the self-attention map M n With v n Transpose and multiply to get the self-attention matrix q n (c); Then perform normalization operation to obtain multiple features o n , Finally, multiple features o n Connect them to get the multi-scale feature F n ;

[0037] Similarly, and Transpose and multiply, then use the softmax function to get the self-attention map M a ; Then the self-attention map M a and Transpose and multiply to get the self-attention matrix q a (c); Then perform normalization operation to obtain multiple features o a , Finally, multiple features o n Connect them to get the multi-scale feature F a .

[0038] Preferably, the objective loss function constructed in step 7 is:

[0039] L=L rank +L sparse +L smooth

[0040] Where, L rank is the ranking loss function, L smooth is the smooth loss function, L sparse is the improved sparse loss function, where:

[0041] L rank The calculation formula is:

[0042]

[0043]

[0044] L rank =L(S a )+L(S n )

[0045] Where, Denotes the i-th anomaly score in the positive bag, represents the i-th anomaly score in the negative bag;

[0046] L smooth The calculation formula is:

[0047] L sparse The calculation formula is: L sparse =(mean(S a )-η) 2 , where η is the average score reference value, mean(S a ) is the average score of anomaly scores in the positive bag. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic diagram of the process of the present invention;

[0049] Figure 2 This is a diagram of the multi-scale feature fusion network structure of the present invention;

[0050] Figure 3 This is the flow chart of the channel attention mechanism of the present invention;

[0051] Figure 4 The average loss value in each optimization cycle during the experimental phase of the present invention and the evaluation results on the test video set;

[0052] Figure 5 This is the visualization result on the test video sequence in the experimental stage of the present invention. DETAILED DESCRIPTION

[0053] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0054] Multi-scale feature fusion and multiple attention-based weakly supervised anomaly detection methods, such as Figure 1 Shown, including:

[0055] Step 1: Collect a training video set, which includes normal videos and abnormal videos;

[0056] Step 2: Split the normal video into negative packets containing T non-overlapping video segments The abnormal video is divided into positive packets containing T non-overlapping video segments Each segment contains c consecutive frames; in this embodiment, T=32;

[0057] Step 3: Use I3D convolutional network to extract positive packets V n and negative package V a The spatiotemporal features X of each video clip in n and X a In this embodiment, Here, the I3D convolutional network extracts spatiotemporal features from video clips as an existing technology, so it is not too elaborate here; then a multi-scale feature fusion network is used to align the V n and negative package V a The spatiotemporal characteristics X n , spatiotemporal characteristics X a Perform feature fusion separately to obtain the multi-scale feature Y of the positive package n and the multi-scale feature Y of the negative bag a , including Figure 2 As shown:

[0058] S301, in this embodiment, 1024 one-dimensional convolution kernels are used to respectively n , spatiotemporal characteristics X a For convolution operation, the size of the one-dimensional convolution kernel is 3, and its expressions are:

[0059] Y1 n =conv1(X n , 3, 1024);

[0060] Y1 a =conv1(X a , 3, 1024);

[0061] Step 302: In this embodiment, 1024 one-dimensional convolution kernels are used to respectively convolute the spatiotemporal features X n , spatiotemporal characteristics X a Perform the convolution operation again, the size of the one-dimensional convolution kernel is 7, and its expressions are:

[0062] Y2 n =conv1(X n , 7, 1024);

[0063] Y2 a =conv1(X a , 7, 1024);

[0064] The features of the convolution outputs in step 303 and step 301 and the features of the convolution output in step 302 are fused with the spatiotemporal features to obtain multi-scale features, whose expressions are:

[0065] Y n =[X n , Y1 n , Y2 n ];

[0066] Y a =[X a , Y1 a , Y2 a ];

[0067] In this embodiment, Y n ∈R T×3072 ;

[0068] Step 4: Use the multi-head self-attention mechanism to respectively analyze the multi-scale features Y of the positive packet. n and the multi-scale feature Y of the negative bag a Add attention to obtain the multi-scale feature F with attention mechanism n and F a ; By adding a multi-head self-attention mechanism, the global time dependency is captured in the time dimension, generating an attention graph M n and M a , M n ∈R H×T×T , M a ∈R H×T×T , H represents the number of heads in the multi-head self-attention mechanism. In this embodiment, H=4; specifically, it includes:

[0069] Step 401: Use a fully connected layer to transform the multi-scale feature Y n and Y a Converted into shared matrix Y n (c) and Y a (c), in this embodiment, Y n (c)∈R T×64 , Y a (c)∈R T×64 ; About using the fully connected layer to transform the multi-scale feature Y n and Y a Converted into shared matrix Yn (c) and Y a (c) is a prior art and will not be described in detail here;

[0070] Step 402: Share the matrix Y n (c) and Y a (c) Generate q respectively n 、k n 、v n and q a 、k a 、v a , the calculation formula is:

[0071] q n =W Q Y n (c);

[0072] k n =W K Y n (c);

[0073] v n =W V Y n (c);

[0074] q a =W Q Y a (c);

[0075] k a =W K Y a (c);

[0076] v a =W V Y a (c);

[0077] Where W Q 、W K 、W V is the weight matrix; in this embodiment, q n ∈R T×64 , k n ∈R T×64 , v n ∈R T×64 ,q a ∈R T×64 , k a ∈R T×64 , v a ∈R T×64 ;

[0078] Step 403: According to the number of heads of multi-head self-attention, n 、k n 、vn and q a 、k a 、v a Split into and In this embodiment, after segmentation,

[0079] Among them, R 4×T×16 By R H×T×64 / H get;

[0080] Step 404: and Transpose and multiply, then use the softmax function to get the self-attention map M n ; Then the self-attention map M n With v n Transpose and multiply to get the self-attention matrix q n (c)∈R 4×T×16 ; Then perform normalization operation to obtain multiple features o n ∈R 4×T×16 , Finally, multiple features o n Connect them to get the multi-scale feature F n ;

[0081] Similarly, and Transpose and multiply, then use the softmax function to get the self-attention map M a ; Then the self-attention map M a and Transpose and multiply to get the self-attention matrix q a (c)∈R 4×T×16 ; Then perform normalization operation to obtain multiple features o a ∈R 4×T×16 , Finally, multiple features o n Connect them to get the multi-scale feature F a ∈R T×64 ;

[0082] Step 5: Use the channel attention mechanism to separately add the multi-scale features F n and F a Perform dimensionality reduction, dimensionality increase, sigmoid function to calculate channel attention weights and add multi-scale features F to the attention mechanism n and F a After each channel is weighted, a multi-scale feature Z with multiple attention mechanisms is obtained. n ∈R T×64 and Za ∈R T×64 ,like Figure 3 As shown, in this specific embodiment, the 64-dimensional multi-scale feature F n and F a Perform dimensionality reduction to 16 dimensions, then increase the dimension back to 64 dimensions, and then use the sigmoid function to calculate the channel attention weights. Then, the calculated channel attention weights are compared with the multi-scale feature F n and F a Multiply to get the multi-scale feature Z with multiple attention mechanisms n and Z a ;

[0083] Step 6: Use a fully connected layer network to add multiple attention mechanisms to the multi-scale feature Z n and Z a Perform linear regression and get V n and V a The anomaly score of each segment in S n and S a ;

[0084] Step 7: Package S based on the anomaly score n and S a Construct a target loss function to optimize the parameters of steps 4 to 6; the target loss function is:

[0085] L=L rank +L sparse +L smooth

[0086] Where, L rank is the ranking loss function, L smooth is the smooth loss function, L sparse is the improved sparse loss function, where:

[0087] L rank The calculation formula is:

[0088]

[0089]

[0090] L rank =L(S a )+L(S n )

[0091] Where, Denotes the i-th anomaly score in the positive bag, represents the i-th anomaly score in the negative bag; by approaching the maximum anomaly score in the positive bag to 1 and the maximum anomaly score in the negative bag to 0, we can avoid interfering with the optimization process;

[0092] Furthermore, in order to smooth the abnormal scores of the video clips obtained in the time dimension, a smoothing loss function L is added on the basis of the ranking loss function. smooth , L smooth The calculation formula is:

[0093]

[0094] In addition, since not all video clips in the positive package contain abnormal events, abnormal events only appear in a part of the videos in the positive package, it is necessary to perform sparse processing on the abnormal scores of all video clips in the positive package. The improved sparse loss function L sparse The calculation formula is: L sparse =(mean(S a )-η) 2 , where η is the average score reference value, mean(S a ) is the average score of the abnormal scores in the positive bag; from the improved sparse loss function L sparse It can be seen that the improved sparse loss function can make the average score of the anomaly scores of the video clips in the positive package close to η. In this embodiment, η = 0.5. At the same time, this embodiment takes into account the situation where the average score deviates greatly from η. By adding the square term, it can, to a certain extent, prevent the interference of large deviations or oscillations of the average score, so as not to make it difficult for the entire weakly supervised anomaly detection process to converge to the optimal value.

[0095] Experiment and result analysis

[0096] Dataset

[0097] This experiment uses a large-scale anomaly detection dataset, UCF-Crime, for experiments. The dataset includes 1,900 untrimmed videos with a total length of 128 hours, collected from street and door surveillance cameras. The UCF-Crime dataset consists of complex and diverse scenes. Both the training and test video sets contain an equal number of normal and abnormal videos, and both cover 13 types of abnormal events in daily life. The training video set contains 1,610 training videos, and the test video set contains 290 test videos. The training videos are all labeled at the video level, and the test videos are all labeled at the frame level. Table 1 shows the number of training and test videos for each anomaly in the UCF-Crime dataset:

[0098]

[0099]

[0100] Table 1

[0101] Parameter configuration in weakly supervised anomaly detection from step 1 to step 7

[0102] To facilitate batch model training, each video is evenly divided into 32 video segments. A pre-trained I3D model for each video segment generates features of shape l×1024, where the size of l is proportional to the total number of video segments. The features of each video segment are converted into features of shape 1×1024 using an averaging method. For the convolutional layers described by the multi-scale feature fusion network, this specific embodiment uses a dropout function to randomly inactivate 50% of the neurons to prevent model overfitting. Furthermore, this specific embodiment incorporates an L2 regularization term, setting the weight decay coefficient to 0.001. Furthermore, the Adam optimizer and an RTX3060 graphics card are used for end-to-end training, with an initial learning rate of 0.0001, a learning rate decay period of 10, a batch size of 64, and 60 training cycles. The hyperparameters λ1, η, and H are set to 8e-5, 0.5, and 4, respectively.

[0103] This experiment counts the average loss value in each optimization cycle and its evaluation results on the test set, such as Figure 4 As shown in the figure, the model converges around the 60th epoch, the average loss value stabilizes at the lowest point, and the performance on the test set stabilizes at the highest point.

[0104] Comparison with Similar Anomaly Detection Methods

[0105] The anomaly score of each video clip is calculated. The anomaly scores of all frames in the video clip are copied from the anomaly score of the video clip, so the frame-level AUC is finally obtained. Table 2 shows the experimental results of the proposed method and the weakly supervised anomaly detection method in the prior art on the UCF-Crime test set:

[0106] Compared with the multi-instance method of Sultani et al. using C3D-RGB features, the AUC of the method of the present invention is improved by 8.0%. The method of Sultani et al. is reproduced using I3D-RGB features. Compared with the reproduced results, the method of the present invention is still improved by 5.5%, which proves the effectiveness of the network model designed by the method of the present invention based on Sultani et al.

[0107] Compared with the methods of Zhang et al., Zhu et al., and Zhong et al., the method of the present invention improves by 5.0%, 4.4%, and 2.4%, respectively.

[0108] Compared with the methods of Wu et al. and Feng et al., the method of the present invention improves by 1.0% and 1.1%, respectively.

[0109] Compared with the best method currently available, the method of the present invention only differs by 0.9%. Analyzing the reason, Tian et al. used dilated convolution to fuse deeper temporal features to train a feature magnitude learning function to improve the robustness of the multi-instance learning method for normal segments in abnormal videos and to more effectively detect abnormal video segments.

[0110]

[0111] Table 2

[0112] Visualize the results

[0113] In order to further evaluate the performance of the model, this embodiment shows the visualization results of the time series of the test video. The higher the anomaly score, the higher the possibility of representing an abnormal event, and vice versa, the higher the possibility of representing a normal event. Figure 5 As shown, the method of the present invention accurately locates the abnormal event in the abuse video Abuse030 in terms of time sequence. For the arrest video Arrest001, although the abnormal event is blocked in the video, the method of the present invention also re-locates the abnormal event. For the arson video Arson007, although the method of the present invention accurately locates the abnormal event, the positioning lasts for a short time, and the fire caused by arson should also be an abnormal event. For the attack video Assault006, the fight video Fighting033 and the shooting video Shooting018, the method of the present invention locates the abnormal event throughout the entire process. For the normal videos Normal_Videos_912 and Normal_Videos_892, the present invention detects them as normal events. In addition, in order to improve the subsequent work, a number of use cases where positioning failed were carefully selected. As shown in FIG. Figure 5 As shown in sub-figures (i) to (m) in the figure, for the abnormal video Vandalism028, the method of the present invention locates the abnormal event too early in time sequence; for the abnormal video Robbery048, the method of the present invention does not detect the abnormal event; for the abnormal video Explosion010, the method of the present invention does not generate a high abnormality score when the abnormal event occurs; for the normal video Normal_Videos_885, the method of the present invention mistakenly locates it as an abnormal event.

[0114] Ablation experiments

[0115] 1) Ablation Experiments on Mean Score Reference Value Selection: For an anomalous video, its anomaly rate is uncertain. Here, the anomaly rate refers to the proportion of anomalous frames in the video. Some anomalous videos have an anomaly rate as high as 70%, while others have an anomaly rate below 30%. When thinning out the anomaly scores of anomalous videos, we cannot simply thin out the anomaly scores of a video with a high anomaly rate to a smaller value. Therefore, this experiment theoretically hypothesized that the mean score reference value η should be set to 0.5. To verify this hypothesis, we conducted ablation experiments on the mean score reference value η, setting η to 0.3, 0.4, 0.5, and 0.6, while keeping all other conditions unchanged. Table 3 shows the ablation results of this experiment. When η was set to 0.3, the AUC value only reached 73.4%, indicating very poor model performance. As η increased, the AUC value gradually improved. The model achieved optimal performance when η was set to 0.5. Further increases in η resulted in a continuous decline in model performance. The experimental results show that our conjecture is correct. By setting η to 0.5 as a compromise, we can take into account both abnormal videos with higher anomaly rates and abnormal videos with lower anomaly rates, thereby effectively handling the situation where the video anomaly rate fluctuates around 0.5.

[0116]

[0117] Table 3

[0118] 2) Ablation Experiments with Different Module Selections: This experiment uses the method proposed by Sultani et al. as a baseline. Through multiple experiments, we refine the improved sparse loss function and try various module combinations to improve model performance. Table 4 shows the ablation test results. Without any improvements, the baseline method achieves only an AUC of 77.9%. This specific embodiment improves performance by approximately 2% after modifying the improved sparse loss function, demonstrating the necessity of improving the improved sparse loss function. Adding a multi-scale feature fusion network to the improved sparse loss function improves performance by 1.7%. Regarding the attention mechanism, adding only the multi-head self-attention module improves model performance by 0.8%. However, adding only the SE attention module weakens model performance by 0.3%. When both the multi-head self-attention module and the SE attention module are added simultaneously, model performance improves by 1.8%. Analysis shows that the low-dimensional features, after the multi-head self-attention module compresses the channel dimensions, are more conducive to the SE attention network focusing on global correlation information between channels. After adding all modules, the proposed method improves by 5.5% compared to the baseline method.

[0119]

[0120] Table 4

[0121] Although the present disclosure is disclosed as above, the protection scope of the present disclosure is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present disclosure, and these changes and modifications will fall within the protection scope of the present invention.

Claims

1. A weakly supervised anomaly detection method combining multi-scale feature fusion and multiple attentions, characterized by: include: Step 1: Collect a training video set, which includes normal videos and abnormal videos; Step 2: Split the normal video into Negative bag of non-overlapping video clips , divide the abnormal video into Positive envelope of non-overlapping video segments , each fragment contains Frame continuous picture; Step 3: Use I3D convolutional network to extract positive packets and negative package The spatiotemporal features of each video clip in and ; and use multi-scale feature fusion network to align the packet and negative package The spatiotemporal characteristics of , spatiotemporal characteristics Perform feature fusion separately to obtain multi-scale features of the positive package Multi-scale features of negative bags ; Specifically include: S301, use The one-dimensional convolution kernels are used to analyze the spatiotemporal features , spatiotemporal characteristics Perform convolution operation, the size of the one-dimensional convolution kernel is , whose expressions are: ； ； Step 302: Use The one-dimensional convolution kernels are used to analyze the spatiotemporal features , spatiotemporal characteristics Perform the convolution operation again, and the size of the one-dimensional convolution kernel is , whose expressions are: ； ； The features of the convolution outputs in step 303 and step 301 and the features of the convolution output in step 302 are fused with the spatiotemporal features to obtain multi-scale features, whose expressions are: ； ； Step 4: Use the multi-head self-attention mechanism to analyze the multi-scale features of the positive packet. Multi-scale features of negative bags Add attention to obtain multi-scale features by adding attention mechanism and ; Step 5: Use the channel attention mechanism to separately add the multi-scale features of the attention mechanism and Perform dimensionality reduction, dimensionality increase, sigmoid function calculation of channel attention weights, and multi-scale features that add attention mechanisms and After each channel is weighted, multi-scale features with multiple attention mechanisms are obtained. and ; Step 6: Use a fully connected layer network to add multiple attention mechanisms to the multi-scale features and Perform linear regression and get and The anomaly score for each segment in and ; Step 7: According to the abnormal score package and Construct the target loss function to optimize the parameters of steps 4 to 6.

2. The weakly supervised anomaly detection method combining multi-scale feature fusion and multiple attention according to claim 1 is characterized in that: In step 4, a multi-head self-attention mechanism is used to respectively focus on the multi-scale features of the positive packet. Multi-scale features of negative bags Add attention to obtain multi-scale features by adding attention mechanism and Specifically include: Step 401: Use a fully connected layer to transform the multi-scale features and Converted into shared matrices and ; Step 402: Share the matrix and Generate separately and , the calculation formula is: ； ； ； ； ； ； Where, 、 、 is the weight matrix; Step 403: According to the number of heads of multi-head self-attention, and Split into and ; Step 404: and Transpose and multiply, then use the softmax function to get the self-attention map ; Then the self-attention map and Transpose and multiply to get the self-attention matrix ; Then perform normalization to obtain multiple features , ; Finally, multiple features Connect to get multi-scale features ; Similarly, and Transpose and multiply, then use the softmax function to get the self-attention map ; Then the self-attention map and Transpose and multiply to get the self-attention matrix ; Then perform normalization to obtain multiple features , ; Finally, multiple features Connect to get multi-scale features .

3. The weakly supervised anomaly detection method combining multi-scale feature fusion and multiple attention according to claim 1 is characterized in that: The objective loss function constructed in step 7 is: ； Where, is the ranking loss function, is a smooth loss function, is the improved sparse loss function, where: The calculation formula is: ； ； ； Where, Indicates the first Anomaly score, Indicates the first Anomaly score; The calculation formula is: ; The calculation formula is: , where is the average score reference value, is the average score of anomaly scores in positive bags.

Images