A dual-branch video anomaly detection method combined with a convolutional attention module

Abstract

The application belongs to the technical field of video anomaly detection, and discloses a double-branch video anomaly detection method combined with a convolution attention module, which comprises the following steps: taking original frames in a training set, inputting the original frames into an optical flow extraction network to obtain optical flow frames, then inputting the original frames and the optical flow frames into two branch networks of prediction and reconstruction respectively, training the double-branch network through a joint loss function, thereby obtaining a complete video anomaly detection model, and similarly, in the test stage, taking original frames in a test set, obtaining optical flow frames through the optical flow extraction network, then inputting the original frames and the optical flow frames into the prediction and reconstruction branches respectively, obtaining prediction frames and reconstruction frames respectively, calculating the prediction error between the prediction frames and the real frames, the reconstruction error between the reconstruction frames and the optical flow frames, weighting and balancing the two kinds of errors, obtaining a video frame anomaly score, and realizing the detection of abnormal video frames. The application can realize higher detection accuracy.

Description

Technical Field

[0001] This invention belongs to the field of video anomaly detection technology, specifically relating to a two-branch video anomaly detection method that combines a convolutional attention module. Background Technology

[0002] Video anomaly detection is an important research topic in the field of computer vision, and it has received increasing attention from researchers in recent years. The specific task of video anomaly detection is to detect events in a video that are inconsistent with normal activities, such as fighting, traffic accidents, or objects being thrown. These abnormal events often pose a threat to public safety; therefore, research on video anomaly detection is of great significance for early detection of abnormal events and, consequently, for reducing their harmfulness.

[0003] However, due to the large variety and low frequency of anomalous events, it is impossible to collect all and enough anomalous events to train the model. Conversely, normal events are plentiful and easy to collect. Therefore, the current mainstream approach is to train the model using only normal events. These mainstream methods can be broadly categorized into two types: video frame reconstruction and video frame prediction. Video frame reconstruction involves using an autoencoder to reconstruct the current frame into a reconstructed frame. The reconstruction error between the current and reconstructed frames is calculated and used to represent an anomaly score, thus enabling the detection of anomalous events, as seen in the AE method (Mahmudul Hasan, Jonghyun Choi, Jan Neumann, Amit K Roy-Chowdhury, and Larry S.D.A. Vis. Learning temporal regularity in video sequences. In CVPR, 2016.). However, due to the strong generalization ability of autoencoders, some anomalous events can also be reconstructed well, resulting in low detection accuracy. Video frame prediction, on the other hand, predicts the next video frame from several consecutive frames. The predicted video frame is then input into a discriminator to determine whether it represents an anomalous event, thus enabling the detection of anomalous events, as seen in (Wen Liu, Weixin Luo, DongzeLian, Shenghua Gao. Future Frame Prediction for Anomaly Detection--A New Baseline. In...). (CVPR, 2018.), but because some normal events such as turning on the light or opening the door cannot be predicted well, the detection accuracy is not high. Summary of the Invention

[0004] To address the low detection accuracy of existing video anomaly detection methods, this invention provides a two-branch video anomaly detection method incorporating a convolutional attention module. This method constructs a two-branch network embedded with a convolutional attention module, performing video frame prediction and optical flow frame reconstruction tasks separately. The two-branch network is trained using a joint loss function, thereby building a network model capable of effectively detecting anomalous events. The specific steps of this video anomaly detection method are as follows:

[0005] S1: Collect video data and divide the video data into training set and test set.

[0006] S2 takes t consecutive video frames from the training set S1, i.e., the original frames, and inputs them into the video frame prediction branch of the dual-branch structure to predict the (t+1)th video frame, i.e., the predicted frame.

[0007] S3, input the t original frames from S2 into the optical flow extraction network to extract t optical flow images from the original frames, i.e., optical flow frames y. 1:t Then, the optical flow frames are input into the optical flow frame reconstruction branch in the dual-branch structure, and t new optical flow frames are reconstructed, i.e., reconstructed frames.

[0008] S4, calculate the (t+1)th predicted frame obtained in S2. and the (t+1)th real frame x t+1 The prediction error L between pred Calculate the t reconstructed frames obtained in S3 and the input t optical flow frames y 1:t Reconstruction error L between recon .

[0009] S5, combined with the prediction error L obtained in S4 pred and reconstruction error L recon And the proposed gradient loss L grad Cross-entropy loss L entro A joint loss function L is constructed to train the two-branch network. The final trained network model is capable of effectively detecting anomalous events.

[0010] S6, similarly, take the test set data from S1, i.e., t consecutive video frames, and sequentially pass through steps S2, S3, and S5 to obtain the prediction error S of the test video frames. pred and reconstruction error S recon Then, the two types of errors are weighted and balanced to obtain the anomaly score S of the video frame under test, thereby realizing the detection of abnormal events.

[0011] Furthermore, the training set in S1 consists only of normal event videos and does not contain any abnormal events, while the test set contains both normal and abnormal events.

[0012] Furthermore, the video frame prediction branch proposed in S2 mainly consists of an autoencoder, multiple embedded memory modules, and a convolutional attention module. The output of this branch is the prediction result for the (t+1)th frame. The specific structure is as follows:

[0013] S2-1, the auto encoder mainly consists of an encoder and a decoder, both of which have a three-layer structure. In the encoder, each layer consists of two convolutional layers and one downsampling layer; similarly, in the decoder, each layer consists of two convolutional layers and one upsampling layer.

[0014] S2-2. Since information loss occurs during the encoding and decoding process, skip connections are added between corresponding layers of the encoder and decoder. The feature maps of each layer of the encoder and the corresponding feature maps of the decoder are directly concatenated to reduce information loss.

[0015] S2-3, to enable the network to better learn the feature representations of normal events, multiple multi-scale memory modules are embedded in the autoencoder structure. These modules can obtain a new feature map based on the cosine similarity principle, as shown in the following formula:

[0016]

[0017] Among them, F in For the input feature map, m i To store the feature vector for each dimension in memory, d(F) in ,m i ) represents cosine similarity, and T represents transpose.

[0018]

[0019] Where exp represents taking the exponent of e, N represents the dimension of the memory, and r i This represents the query weight corresponding to the feature vector in memory.

[0020] Based on the above formula, the query weight corresponding to each feature vector in the memory module can be obtained. Multiplying the query weight by the corresponding feature vector and concatenating them yields a new feature map, as shown in the following formula:

[0021]

[0022] Among them, F out This represents the output feature map, where N represents the dimension of the memory, and ri The m represents the query weight corresponding to the feature vector in memory. i This represents the corresponding memory feature vector.

[0023] To learn feature information at different scales, memory modules are embedded in the middle of the autoencoder and between different layers of the decoder.

[0024] In section S2-4, to further enhance the network's learning of semantic information surrounding the input image, a convolutional attention module is embedded after the auto-decoder. This module consists of a masked convolutional layer and channel attention layers. The masked convolutional layer mainly comprises four small convolutions, each operating on the corner regions of the feature map. The results of the four small convolutions are then summed to obtain the output of the convolutional attention module. The channel attention layer consists of three steps: 1. Compression stage: Global average pooling is performed on each channel of the input feature map to obtain a one-dimensional vector with dimensions equal to the number of feature channels; 2. Activation stage: The one-dimensional vector obtained in the previous step is input into two fully connected layers to obtain a one-dimensional weight vector corresponding to each feature channel; 3. Readjustment stage: The one-dimensional weight vector obtained in the previous step is multiplied with the input feature map to obtain the final output. To maintain consistency with the number of channels in the input data, the number of convolutional attention modules used is equal to the number of channels in the input data.

[0025] Furthermore, the optical flow extraction network in S3 adopts the commonly used FlowNet2 network. Similar to the prediction branch in S2, the optical flow frame reconstruction branch proposed in S3 mainly consists of an autoencoder, multiple embedded memory modules, and a convolutional attention module. The difference is that the output of this branch is t optical flow reconstruction frames.

[0026] Furthermore, the prediction error L in S4 pred and reconstruction error L recon They are obtained from the following two formulas respectively:

[0027]

[0028] Where ||||2 represents taking the l2 norm.

[0029]

[0030] Where ||||2 represents taking the l2 norm.

[0031] Furthermore, to make the predicted results closer to the actual results, gradient loss L is added in step S5. grad The formula is as follows:

[0032]

[0033] Where k and l represent the spatiotemporal indices of the video frames, ||||1 indicates taking the l1 norm, and || indicates taking the absolute value.

[0034] To enable the memory module to better learn normal events, we want the query weights to be more sparse. Therefore, we added cross-entropy loss Li. entro The formula for constraining query weights is as follows:

[0035]

[0036] Where O represents the number of memory modules, N represents the dimension of the memory modules, and r i,j This represents the query weight of the corresponding feature vector in memory.

[0037] By weighting and balancing the four loss functions mentioned above, we can obtain the joint loss function L, whose formula is as follows:

[0038]

[0039] Where, λ pred , λ grad , λ recon and λ entro For balancing parameters.

[0040] Furthermore, the anomaly score S in S6 is determined by the prediction error S. pred and reconstruction error S recon The weighted balance yields the following formula:

[0041] S = w p S pred +w r S recon

[0042] in, w p and w r For balancing parameters.

[0043] The beneficial effects of this invention are:

[0044] (1) This invention proposes a dual-branch network structure to perform video frame prediction and reconstruction tasks respectively. The generated features have rich information. Experiments were conducted on multiple public datasets. Compared with previous methods, this invention achieved better detection results.

[0045] (2) The present invention embeds the convolutional attention module into the dual-branch network structure, which is beneficial for the branches to learn the global structure of local features and further improves the performance of the model.

[0046] (3) The present invention embeds multiple multi-scale memory modules in the autoencoder, which is beneficial for the network to learn the features of normal events at different scales and further improves the model’s ability to detect abnormal events.

[0047] (4) This invention belongs to a semi-supervised learning method, which uses normal events as training data, greatly reducing the cost of data collection.

[0048] The dual-branch video anomaly detection method of the present invention designs a dual-branch network structure for two tasks: video frame prediction and video frame reconstruction, and embeds a convolutional attention module therein. The dual-branch network is then trained using a designed joint loss function, and finally a network model that can effectively detect abnormal events is obtained. Attached Figure Description

[0049] Figure 1 This is a flowchart of the anomaly detection method in an embodiment of the present invention.

[0050] Figure 2 This is an overall structural diagram of the dual-branch network in an embodiment of the present invention.

[0051] Figure 3 This is a specific structural diagram of the branch network in an example of the present invention.

[0052] Figure 4 This is a structural diagram of the masked convolutional kernel of the convolutional attention module in an embodiment of the present invention. Detailed Implementation

[0053] The embodiments of the present invention will be disclosed below with reference to the drawings. For clarity, many practical details will be described in the following description. However, it should be understood that these practical details are not intended to limit the invention. That is, in some embodiments of the invention, these practical details are not essential.

[0054] This invention is a two-branch video anomaly detection method combining a convolutional attention module. The anomaly detection method specifically includes the following steps:

[0055] Step 1: Collect video data and divide the video data into a training set and a test set. The training set consists only of videos of normal events and does not contain any abnormal events, while the test set contains both normal and abnormal events.

[0056] Step 2: Take t consecutive video frames from the training set in Step 1, i.e., the original frames, and input them into the video frame prediction branch of the dual-branch structure to predict the (t+1)th video frame, i.e., the predicted frame. The prediction branch mainly consists of an autoencoder, multiple embedded memory modules, and a convolutional attention module. The output of this branch is the prediction result for frame t+1. The specific structure construction process is as follows:

[0057] S2-1, the auto encoder mainly consists of an encoder and a decoder, both of which have a three-layer structure. In the encoder, each layer consists of two convolutional layers and one downsampling layer; similarly, in the decoder, each layer consists of two convolutional layers and one upsampling layer.

[0058] S2-2. Since information loss occurs during the encoding and decoding process, skip connections are added between corresponding layers of the encoder and decoder. The feature maps of each layer of the encoder and the corresponding feature maps of the decoder are directly concatenated to reduce information loss.

[0059] S2-3, to enable the network to better learn the feature representations of normal events, multiple multi-scale memory modules are embedded in the autoencoder structure. These memory modules can obtain a new feature map based on the cosine similarity principle, as shown in the following formula:

[0060]

[0061] Among them, F in For the input feature map, m i To store the feature vector for each dimension in memory, d(F) in ,m i ) represents cosine similarity, and T represents transpose.

[0062]

[0063] Where exp represents taking the exponent of e, N represents the dimension of the memory, and r i This represents the query weight corresponding to the feature vector in memory.

[0064] Based on the above formula, the query weight corresponding to each feature vector in the memory module can be obtained. Multiplying the query weight by the corresponding feature vector and concatenating them yields a new feature map, as shown in the following formula:

[0065]

[0066] Among them, F out This represents the output feature map, where N represents the dimension of the memory, and r i The m represents the query weight corresponding to the feature vector in memory. i This represents the corresponding memory feature vector.

[0067] To learn feature information at different scales, memory modules are embedded in the middle of the autoencoder and between different layers of the decoder, such as... Figure 2 As shown.

[0068] In section S2-4, to further enhance the network's learning of semantic information surrounding the input image, a convolutional attention module is embedded after the auto-decoder. This module consists of a masked convolutional layer and channel attention layers. The masked convolutional layer mainly comprises four small convolutions, each operating on the corner regions of the feature map. The results of the four small convolutions are then summed to obtain the output of the convolutional attention module. The channel attention layer consists of three steps: 1. Compression stage: Global average pooling is performed on each channel of the input feature map to obtain a one-dimensional vector with dimensions equal to the number of feature channels; 2. Activation stage: The one-dimensional vector obtained in the previous step is input into two fully connected layers to obtain a one-dimensional weight vector corresponding to each feature channel; 3. Readjustment stage: The one-dimensional weight vector obtained in the previous step is multiplied with the input feature map to obtain the final output. To maintain consistency with the number of channels in the input data, the number of convolutional attention modules used is equal to the number of channels in the input data.

[0069] Step 3: Input the t original frames from Step 2 into the optical flow extraction network to extract t optical flow images from the original frames, i.e., optical flow frames y. 1:t Then, the optical flow frames are input into the optical flow frame reconstruction branch in the dual-branch structure, and t new optical flow frames are reconstructed, i.e., reconstructed frames. The optical flow extraction network uses the commonly used FlowNet2 network. Similar to the prediction branch in step 2, this optical flow frame reconstruction branch mainly consists of an autoencoder, multiple embedded memory modules, and a convolutional attention module. The difference is that this branch outputs t optical flow reconstruction frames.

[0070] Step 4: Calculate the (t+1)th predicted frame obtained in Step 2. and the (t+1)th real frame x t+1 The prediction error L between pred Calculate the t reconstructed frames obtained in step 3. and the input t optical flow frames y 1:t Reconstruction error L between recon In step 4, the prediction error L pred and reconstruction error L recon They are obtained from the following two formulas respectively:

[0071]

[0072] Where ||||2 represents taking the l2 norm.

[0073]

[0074] Where ||||2 represents taking the l2 norm.

[0075] Step 5, combine the prediction error L obtained in Step 4 pred and reconstruction error L recon And the proposed gradient loss L grad Cross-entropy loss L entro A joint loss function L is constructed to train the two-branch network, ultimately resulting in a network model that can effectively detect abnormal events.

[0076] To make the predicted results closer to the actual results, step 5 incorporates gradient loss L. grad The formula is as follows:

[0077]

[0078] Where k and l represent the spatiotemporal indices of the video frames, ||||1 indicates taking the l1 norm, and || indicates taking the absolute value.

[0079] To enable the memory module to better learn normal events, this invention aims to make the query weights more sparse. Therefore, this invention incorporates cross-entropy loss L. entro The formula for constraining query weights is as follows:

[0080]

[0081] Where O represents the number of memory modules, N represents the dimension of the memory modules, and r i,j This represents the query weight of the corresponding feature vector in memory.

[0082] By weighting and balancing the four loss functions mentioned above, we can obtain the joint loss function L, whose formula is as follows:

[0083]

[0084] Where, λ pred , λ grad , λ recon and λ entro For balancing parameters.

[0085] Step 6, similarly, take the test set data from Step 1, i.e., t consecutive video frames, and sequentially pass through Steps 2, 3, and 5 to obtain the prediction error S of the test video frames. pred and reconstruction error S reconThe two errors are then weighted and balanced to obtain the anomaly score S of the video frame under test, thereby enabling the detection of abnormal events. The anomaly score S in step 6 is determined by the prediction error S0. pred and reconstruction error S recon The weighted balance yields the following formula:

[0086] S = w p S pred +w r S recon

[0087] in, w p and w r For balancing parameters.

[0088] This invention belongs to a semi-supervised learning method, using only normal events as training data, which greatly reduces the cost of data acquisition. This invention employs a dual-branch structure, with each branch learning different tasks, effectively improving the model's learning ability. Furthermore, this invention embeds multiple memory modules and convolutional attention modules into the dual-branch network, further enhancing the model's ability to learn features of different sizes. Therefore, compared to previous methods, this invention significantly improves the accuracy of anomaly detection.

[0089] To make the technical solution of the present invention clearer, the detection accuracy of the present invention will be verified by experiments below.

[0090] Experimental conditions

[0091] 1. Experimental Database

[0092] Training and testing were performed on the Ped2, Avenue, and ShanghaiTech datasets. During anomaly detection, the dataset was divided into a training class (which may be further subdivided into training and validation classes if necessary) and a test set. The division varied across different datasets. The training class plus the validation class represented the known categories, while the test class represented the unknown categories, with no overlap between the two.

[0093] See Table 1 below for details.

[0094] Table 1 provides a detailed description of the dataset.

[0095]

[0096] 2. Experimental parameter settings

[0097] The fixed parameters for the model are set as shown in Table 2 below:

[0098] Table 2 Model Fixed Parameters

[0099] N <![CDATA[λ pred ]]> <![CDATA[λ grad ]]> <![CDATA[λ recon ]]> <![CDATA[λ entro ]]> 2000 1 1 1 0.0002

[0100] The balancing parameter settings for anomaly scores on different datasets are shown in Table 3:

[0101] Table 3. Anomaly Score Balancing Parameters

[0102] Dataset <![CDATA[w p ]]> <![CDATA[w r ]]> Ped2 1.0 0.1 Avenue 0.1 1.0 ShanghaiTech 0.02 1.0

[0103] A. Evaluation Criteria for Experimental Results

[0104] This model is designed for anomaly detection. In anomaly detection, the test class can contain both unknown and known categories. AUC (Average Unknown Value) is used to measure detection performance. A higher AUC value indicates a better model performance. The formula for calculating AUC is as follows:

[0105]

[0106] Where M is the number of positive packet samples, N is the number of negative packet samples, and the numerator is the total number of combinations in which the probability of predicting a positive packet is greater than the probability of predicting a negative packet.

[0107] B. Comparative Test Plan

[0108] This embodiment compares with other cutting-edge anomaly detection methods on the Ped2, Avenue, and ShanghaiTech datasets.

[0109] Table 4 Performance Comparison of Anomaly Detection Methods:

[0110] Methods Ped2 Avenue ShanghaiTech Memae 0.941 0.833 0.712 VEC 0.973 0.902 0.748 MPN 0.969 0.895 0.738 sRNN 0.922 0.835 0.696 Ours 0.994 0.911 0.766

[0111] Ours is the method proposed in this invention. The comparison results in Table 4 with the current cutting-edge anomaly detection methods show that the anomaly detection effect of this method exceeds that of other compared methods, which objectively demonstrates the effectiveness of the method proposed in this paper.

[0112] The above description is only a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. Any equivalent modifications or changes made by those skilled in the art based on the content disclosed in the present invention should be included within the scope of protection set forth in the claims.

Claims

1. A dual-branch video anomaly detection method combined with a convolutional attention module, characterized in that: The method is based on a double-branch network structure, and a convolutional attention module is embedded in the double-branch network structure to strengthen the learning of features, and two branches are respectively used for video frame prediction and optical flow frame reconstruction tasks, and the convolutional attention module is constructed by a mask convolution layer and a channel attention layer, wherein the mask convolution layer mainly consists of four small convolutions, each small convolution operates on the corner area of the feature map, and finally the results of the four small convolutions are added to obtain the output result of the convolutional attention module, and specifically, the double-branch video anomaly detection method comprises the following steps: Step 1: Collect video data, and divide the video data into a training set and a test set; Step 2: input the consecutive t video frames in the training set described in step 1, i.e. the original frames, into the video frame prediction branch in the double-branch structure, to obtain the (t+1)th video frame, i.e. the predicted frame, by prediction Step 3: input the t original frames in step 2 into the optical flow extraction network to extract t optical flow images of the original frames, i.e. optical flow frames y 1:t Then input the optical flow frames y 1:t into the optical flow frame reconstruction branch in the double-branch structure to reconstruct t new optical flow frames, i.e. reconstructed frames Step 4: Compute the prediction error L between the (t+1)th predicted frame obtained in step 2 and the (t+1)th real frame x t+1 pred Step 5: Compute the reconstruction error L between the t reconstructed frames obtained in step 3 and the input t optical flow frames y 1:t recon ;​​ Step 5: combine the prediction error L obtained in step 4 pred and the reconstruction error L recon and the gradient loss L grad , the cross-entropy loss L entro A joint loss function L is constructed to train the dual-branch network, and finally a network model capable of effectively detecting abnormal events is trained. Step 6: Take the data of the to-be-tested set in step 1, that is, the continuous t video frames, and sequentially pass through steps 2, 3 and 5 to obtain the prediction error S of the to-be-tested video frame pred and the reconstruction error S recon , and then balance the prediction error S pred and the reconstruction error S recon by weighting, and finally obtain the anomaly score S of the to-be-tested video frame, thereby realizing the detection of the abnormal event.

2. The dual-branch video anomaly detection method combined with a convolutional attention module according to claim 1, characterized in that: The video frame prediction branch proposed in step 2 is mainly composed of an auto-encoder, multiple memory-in-embedding modules embedded therein, and a convolutional attention module, and the output of the branch is the prediction result of the t+1 frame 3. The dual-branch video anomaly detection method combined with a convolutional attention module according to claim 2, characterized in that: The construction method of the video frame prediction branch is as follows: Step 2-1: Construct an autoencoder: the autoencoder mainly consists of an encoder and a decoder, and both the encoder and the decoder have a three-layer structure, the encoder consists of two convolutional layers and a down-sampling layer, and the decoder consists of two convolutional layers and an up-sampling layer; Step 2-2: Channel splicing is performed on the autoencoder in step 2-1: a jump connection is added between the corresponding layers of the encoder and the decoder, and the channel splicing operation is directly performed on the feature map of each layer of the encoder and the feature map of the corresponding layer of the decoder, thereby reducing the loss of information; Step 2-3: A plurality of multi-scale memory modules are embedded in the autoencoder structure, and according to the cosine similarity principle, a new feature map is obtained, and the formula is as follows: wherein F in is an input feature map, m i is a feature vector of each dimension in the memory, d(F in , m i ) represents cosine similarity, and T represents transposition. wherein exp denotes the exponential of e, N denotes the dimension of the memory, r i denotes the query weight of the memory for the feature vector; According to the above formula, the query weight corresponding to each feature vector in the memory module can be obtained, and then the query weight is multiplied by the corresponding feature vector and spliced to obtain a new feature map, and the formula is as follows: wherein F out represents an output feature map, N represents the dimension of the memory, r i represents a query weight corresponding to the memory feature vector, m i represents the corresponding memory feature vector; Step 2-4: Embed a convolutional attention module: in order to strengthen the learning of the network on the peripheral semantic information of the input image, a convolutional attention module is embedded after the automatic decoder.

4. The dual-branch video anomaly detection method combined with a convolutional attention module according to claim 3, characterized in that: The construction of the channel attention layer in step 2-4 mainly includes the following steps: Step 2-4-1: Compression stage: performing global average pooling on each channel of the input feature map to obtain a one-dimensional vector with a dimension equal to the number of feature channels; Step 2-4-2: Excitation stage: inputting the one-dimensional vector obtained in the compression stage of step 2-4-1 into two fully connected layers to obtain a one-dimensional weight vector corresponding to each feature channel; Step 2-4-3: Re-adjustment stage: multiplying the one-dimensional weight vector obtained in the excitation stage of step 2-4-2 with the input feature map to obtain the final output result.

5. The dual-branch video anomaly detection method combined with a convolutional attention module according to claim 4, characterized in that: The number of convolutional attention modules used is equal to the number of channels of the input data.

6. The dual-branch video anomaly detection method combined with a convolutional attention module according to claim 1, characterized in that: The prediction error L in step 4 pred and the reconstruction error L recon are obtained from the following two equations, respectively: Wherein, || ||2 represents taking l2 norm, Wherein, || ||2 represents taking l2 norm.

7. The dual-branch video anomaly detection method combined with a convolutional attention module according to claim 6, characterized in that: The construction process of the joint loss function in step 5 is as follows: Step 5-1: Gradient loss L is added in the step 5 grad The formula is as follows: Wherein, k, l represent the space-time index of the video frame, || ||1 represents taking l1 norm, and | | represents taking absolute value; Step 5-2: To make the memory module learn normal events better, the query weights are more sparse, so the cross-entropy loss L is added entro The query weights are constrained, and the formula is as follows: where O represents the number of memory modules, N represents the dimension of the memory modules, r i,j represents the query weight of the feature vector in the corresponding memory Step 5-3: The prediction error L pred and the reconstruction error L recon and the gradient loss L grad , the cross-entropy loss L entro The joint loss function L can be obtained by weighting and balancing, and its formula is as follows: where λ pred , λ grad , λ recon and λ entro are balancing parameters.

8. The dual-branch video anomaly detection method combined with a convolutional attention module according to claim 1, characterized in that: The anomaly score S in step 6 is derived from the prediction error S pred and the reconstruction error S recon weighted in balance to give, S = 0.5S + 0.5S S = w p S pred +w r S recon wherein w p and w r are balancing parameters.

Images