A Method and System for Anomaly Detection in Surveillance Video Based on Dynamic Self-Supervised Networks

Abstract

This invention discloses a method and system for anomaly detection in surveillance videos based on a dynamic self-supervised network. The method includes: inputting acquired video samples to be detected into a trained dynamic self-supervised network, and outputting the final detection result. The dynamic self-supervised network includes a feature reconstruction module and a discrimination module. The feature reconstruction module adopts an encoder-decoder architecture, with multi-layer downsampling encoder layers and multi-layer upsampling decoder layers having skip connections within the same layer. Each of the multi-layer downsampling encoder layers employs a hybrid anomaly dynamic convolution, adaptively and dynamically extracting features based on the input video samples to be detected. The extracted features are then passed through multi-layer upsampling decoder layers to output a predicted image of the next frame of the video samples to be detected. Based on the predicted image, the discrimination module outputs the final detection result. This invention designs a hybrid anomaly dynamic convolution to adaptively extract features from diverse anomaly events, thereby improving the accuracy of anomaly detection.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for detecting anomalies in surveillance videos based on dynamic self-supervised networks. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Video anomaly detection, aimed at detecting anomalous events in surveillance videos, is widely used in many real-world scenarios, such as security video surveillance, autonomous vehicles, and industrial robots. As an important and popular research topic in computer vision, video anomaly detection has attracted increasing attention from researchers. However, it is a challenging task because anomalous events are diverse, making it difficult to describe all anomalies with a single, unified definition. Therefore, many researchers, when performing video anomaly detection, only use normal video to train a feature encoder to represent normal video segments. Then, when anomalous video segments are fed into the encoder, the learned distribution of the normal video differs from the distribution of the anomalous video input. Therefore, a network can be used to distinguish anomalous video segments.

[0004] Considering that the differences between normal and abnormal events can be described in terms of both motion and appearance, researchers typically employ reconstruction-based schemes to detect anomalies. This involves reconstructing and encoding features from normal video to detect abnormal videos. Currently, some researchers utilize traditionally handcrafted spatiotemporal local features, such as spatiotemporal gradient information and optical flow histograms, to describe scene features. Other researchers have proposed unsupervised, deep learning-based anomaly detection algorithms that learn a convolutional autoencoder using normal video frames. This convolutional autoencoder exhibits smaller reconstruction errors when reconstructing normal samples, but larger reconstruction errors when reconstructing abnormal samples.

[0005] However, due to the powerful capabilities of deep neural networks, the reconstructed anomalous events or videos can be too "good" to be detected. To alleviate this problem, researchers have used memory-enhanced autoencoders to assist in anomaly detection tasks. Although this improvement has made progress in anomaly detection, relying solely on a single static network cannot explore discriminative information from diverse anomalies, making it difficult to guarantee the effective reconstruction of the "good" of normal events and the "bad" of anomalous events, ultimately resulting in poor accuracy in anomaly detection. Summary of the Invention

[0006] To address the shortcomings of the existing technologies, this invention provides a method and system for anomaly detection in surveillance videos based on a dynamic self-supervised network. A novel dynamic self-supervised network (DSS-Net) is constructed and trained to adaptively select an appropriate network structure, extracting spatially and temporally effective information from diverse abnormal and normal video samples. This avoids the problem that using a single static network cannot explore discriminative information from diverse anomalies, making it impossible to accurately identify anomaly patterns based on reconstructed features.

[0007] Firstly, this disclosure provides a method for detecting anomalies in surveillance videos based on dynamic self-supervised networks.

[0008] A method for anomaly detection in surveillance video based on dynamic self-supervised networks, comprising:

[0009] Obtain video samples to be detected, input the video samples to be detected into the trained dynamic self-supervised network, and output the final detection result;

[0010] The dynamic self-supervised network includes a feature reconstruction module and an discrimination module. The feature reconstruction module adopts an encoder-decoder architecture, with multi-layer downsampling encoder layers and multi-layer upsampling decoder layers skipping connections within the same layer.

[0011] All multi-layer downsampling encoder layers employ hybrid anomalous dynamic convolution to adaptively and dynamically extract features based on the input video sample to be detected. The extracted features are then passed through multi-layer upsampling decoder layers to output the next frame prediction image of the video sample to be detected.

[0012] Based on the predicted image of the next frame of the video sample to be detected, the peak signal-to-noise ratio of the predicted frame is calculated by the discrimination module to determine whether the video sample to be detected is an abnormal sample, and the final detection result is output.

[0013] Secondly, this disclosure provides a surveillance video anomaly detection system based on a dynamic self-supervised network.

[0014] A surveillance video anomaly detection system based on a dynamic self-supervised network includes:

[0015] The video acquisition module is used to acquire video samples to be detected.

[0016] The detection module is used to input the video samples to be detected into the trained dynamic self-supervised network and output the detection results;

[0017] The dynamic self-supervised network includes a feature reconstruction module and an discrimination module. The feature reconstruction module adopts an encoder-decoder architecture, with multi-layer downsampling encoder layers and multi-layer upsampling decoder layers skipping connections within the same layer.

[0018] All multi-layer downsampling encoder layers employ hybrid anomalous dynamic convolution to adaptively and dynamically extract features based on the input video sample to be detected. The extracted features are then passed through multi-layer upsampling decoder layers to output the next frame prediction image of the video sample to be detected.

[0019] Based on the predicted image of the next frame of the video sample to be detected, the peak signal-to-noise ratio of the predicted frame is calculated by the discrimination module to determine whether the video sample to be detected is an abnormal sample, and the final detection result is output.

[0020] Thirdly, this disclosure also provides an electronic device, including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the steps of the method described in the first aspect.

[0021] Fourthly, this disclosure also provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the steps of the method described in the first aspect.

[0022] The above one or more technical solutions have the following beneficial effects:

[0023] 1. This invention provides a method and system for anomaly detection in surveillance videos based on a dynamic self-supervised network. A novel dynamic self-supervised network (DSS-Net) is constructed and trained to adaptively select an appropriate network structure to extract spatially and temporally effective information from diverse abnormal and normal video samples. This avoids the problem that using a single static network cannot explore discriminative information from diverse anomalies, making it impossible to accurately identify anomaly patterns based on reconstructed features.

[0024] 2. In this invention, to enable self-learning, spatially and temporally pseudo-anomaly data is generated as input to a dynamic self-supervised network, and a hybrid anomaly dynamic convolution HAD-Conv is designed to adaptively extract features from diverse anomaly events. Furthermore, a feature separation loss is designed for this network model to maximize the difference between anomalous and normal videos, thus benefiting the model in extracting discriminative information.

[0025] 3. The effectiveness of the proposed DSS-Net on public anomaly detection datasets was evaluated through extensive experiments. Compared with existing methods, the proposed detection scheme has better detection performance, improves detection accuracy, and achieves better detection results. Attached Figure Description

[0026] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0027] Figure 1 This is a schematic diagram of the overall structure of the dynamic self-supervised network in an embodiment of the present invention;

[0028] Figure 2 This is a schematic diagram of the time-space pseudo-anomaly sample generator in an embodiment of the present invention;

[0029] Figure 3 This is a schematic diagram of hybrid abnormal dynamic convolution in an embodiment of the present invention. Detailed Implementation

[0030] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0031] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0032] Example 1

[0033] This embodiment provides a method for anomaly detection in surveillance videos based on a dynamic self-supervised network (DSS-Net). By constructing a novel DSS-Net, it adaptively selects an appropriate network structure to extract spatial and temporal information from diverse abnormal and normal video samples. This avoids the problem that a single static network cannot explore discriminative information from diverse anomalies, making it difficult to accurately identify abnormal patterns based on reconstructed features. Specifically, this embodiment uses generated spatial and temporal pseudo-anomaly data as input and sends it to the DSS-Net to mine discriminative information. Simultaneously, a feature separation loss is constructed for this network to maximize the difference between abnormal and normal patterns. Through this DSS-Net, discriminative information can be mined from different abnormal and normal patterns. During anomaly detection, when no abnormal event exists, the network reconstructs a "good" result; conversely, when an abnormal event exists, the network reconstructs a "bad" result, thereby distinguishing between abnormal and normal patterns / events and improving detection accuracy.

[0034] The anomaly detection method for surveillance videos based on dynamic self-supervised networks proposed in this embodiment specifically includes the following steps:

[0035] Obtain video samples to be detected, input the video samples to be detected into the trained dynamic self-supervised network, and output the final detection result;

[0036] The dynamic self-supervised network includes a feature reconstruction module and an discrimination module. The feature reconstruction module adopts an encoder-decoder architecture, with multi-layer downsampling encoder layers and multi-layer upsampling decoder layers skipping connections within the same layer.

[0037] All multi-layer downsampling encoder layers employ hybrid anomalous dynamic convolution to adaptively and dynamically extract features based on the input video sample to be detected. The extracted features are then passed through multi-layer upsampling decoder layers to output the next frame prediction image of the video sample to be detected.

[0038] Based on the predicted image of the next frame of the video sample to be detected, the peak signal-to-noise ratio of the predicted frame is calculated by the discrimination module to determine whether the video sample to be detected is an abnormal sample, and the final detection result is output.

[0039] The scheme described in this embodiment constructs and trains a dynamic self-supervised network. This trained network adaptively selects a suitable network structure to extract spatial and temporal information from diverse abnormal and normal video samples, reconstructing and encoding video features to improve anomaly detection accuracy. In this embodiment, the training process of the dynamic self-supervised network is as follows: First, normal video samples are input into a temporal-spatial pseudo-anomaly sample generator to obtain spatial and temporal pseudo-anomaly video samples. Second, the spatial pseudo-anomaly video samples, temporal pseudo-anomaly video samples, and normal video samples are used as training set samples and input into the dynamic self-supervised network to train it. The network learns the differences between normal and abnormal video samples and outputs the predicted next frame image of the input video sample. Then, the test set is input into the network. The predicted frame for normal video samples is very similar to the real image, while the predicted frame for abnormal video samples differs significantly from the real image. The peak signal-to-noise ratio (PSNR) of the normal and abnormal video samples is calculated to determine whether the current video sample is an abnormal sample.

[0040] Specifically, such as Figure 1 As shown, the training process of the dynamic self-supervised network described in this embodiment mainly includes the following steps:

[0041] Step S1: Input normal video samples into the temporal-spatial pseudo-anomaly sample generator to generate pseudo-anomaly video samples, and generate and obtain spatial pseudo-anomaly video samples and temporal pseudo-anomaly video samples.

[0042] In this embodiment, a time-space pseudo-anomaly sample generator is constructed, such as... Figure 2As shown, it includes a temporal pseudo-anomaly sample generator and a spatial pseudo-anomaly sample generator, which can explore the abnormal features in time and space. Specifically, given a sequence of normal video samples {I1, I2,..., I t ,..., I 2t+1}, which comes from a 2t + 1-frame normal video, it is input into the pseudo-anomaly sample generator.

[0043] For the temporal pseudo-anomaly sample generator, considering the first 2t frames in the given sequence of normal video samples, that is, {I1, I2,..., I 2t}, the goal of generating pseudo-temporal samples is to generate the target future frame I 2t+1 . Therefore, the generation process of the temporal pseudo-anomaly sample generator is as follows:

[0044] First, randomly select an integer n (note that n < t), and start from the end of the entire video sample to select t - n or t + n frames to form the video sample sequence or the video sample sequence

[0045]

[0046] Secondly, for the t - n-frame video sample sequence randomly copy n frames; for the t + n-frame video sample sequence randomly delete n frames. At this time, a temporal pseudo-anomaly video sample sequence with t frames is generated, denoted as

[0047] After the above process, the number of pseudo-anomaly frames in time is the same as that of the normal sequence, but the time information is distorted. The generated temporal pseudo-anomaly video samples are input into the dynamic self-supervised network to simulate the abnormal information in time.

[0048] It should be noted that in the training stage, the future target frame of the temporal pseudo-anomaly needs to be consistent with the real frame, which enables the dynamic self-supervised network to utilize the difference in the temporal feature distribution between normal events and abnormal events.

[0049] For the spatial pseudo-anomaly sample generator, first, select the last t-frame video sample sequence {I t+1 ,..., I 2t}, and then perform a random rectangular masking operation on it using Gaussian noise. The formula is:

[0050]

[0051]

[0052] μ is set to 0.6. The size of the mask remains constant within the same sequence, but the mask is random for different video sequences.

[0053] Through the above scheme, in the target frame An all-zero mask is added to the network for spatial pseudo-anomalies, enabling the dynamic self-supervised network to clearly define the boundaries between normal and anomalous events. In this way, predictions of normal data are satisfactory during inference, while predictions of spatial anomalies are closer to the all-zero mask.

[0054] Step S2: Input the spatial pseudo-anomaly video samples, temporal pseudo-anomaly video samples, and normal video samples into the dynamic self-supervised network to train the dynamic self-supervised network. The dynamic self-supervised network learns the difference between normal video samples and anomalous video samples and outputs the predicted next frame image of the input video sample.

[0055] Specifically, the dynamic self-supervised network includes a feature reconstruction module and a discrimination module connected sequentially. The feature reconstruction module serves as the backbone network, employing an encoder-decoder architecture, with multi-layer downsampling encoder layers and multi-layer upsampling decoder layers connected in skip connections within the same layer. The multi-layer downsampling encoder layers utilize Hybrid Anomaly Dynamic Convolution (HAD-Conv). Unlike traditional static networks, the weights of HAD-Conv can be adjusted according to the input samples. Therefore, HAD-Conv can adaptively and dynamically extract different features from different anomalies. Feature extraction is performed by replacing all three-dimensional convolution kernels in the encoder with HAD-Conv. The input of each HAD-Conv layer is represented as X, and the output is Y, i.e., the input frame image X and the output feature map Y. C in C represents the number of channels in the input image. out H represents the number of channels in the output image, and H×W represents the length and width of the image.

[0056] The aforementioned HAD-Conv consists of four different modules, mainly including an aggregation dynamic weight module, a dynamic weight redistribution module, a first-channel dynamic weight module, and a second-channel dynamic weight module. For example... Figure 3As shown, the input frame image sample enters the hybrid anomaly dynamic convolution. First, image features are extracted through the average pooling layer, convolutional layer, and activation layer in the feature compression module to generate the original feature map. Then, the original feature map is input to the aggregation dynamic weight module and the dynamic weight redistribution module, respectively, to dynamically adjust and determine the final convolution kernel weights according to the input samples. At the same time, the original feature map and the original input frame image are input to the first channel dynamic weight module to generate the first feature map after dynamic feature update. After that, the first feature map is input to the generation dynamic convolution module, and combined with the determined convolution kernel weights, to generate the second feature map after dynamic feature update. Finally, the second feature map and the original feature map are input to the second channel dynamic weight module to generate the third feature map after dynamic feature update. This third feature map is the final feature map output by the hybrid anomaly dynamic convolution.

[0057] The aggregation dynamic weighting module works as follows: While a conventional convolutional layer uses a static kernel applied to all input samples, in this embodiment, the aggregation dynamic weighting module uses a linear combination of n dynamically weighted convolutional kernels, making the convolution operation dependent on the input samples. Dynamic convolution operations can be defined as follows:

[0058] α w =f AW (v s ),α w ∈R n (3)

[0059] W p =α w1 W1+...+α wn W n (4)

[0060] In the above formula, f AW Indicates a fully connected operation, α w W is an n-dimensional vector; in this embodiment, n is the default value of 4. p W represents the convolution kernel formed by the aggregation of n convolution kernels. i This represents the kernel parameters of the i-th convolution, specifically the 27 values ​​of a 3x3x3 convolution kernel; α w For W i Dynamic weights.

[0061] The dynamic weight reassignment module addresses the issue that traditional 3D convolutional kernels can simultaneously focus on both spatial and temporal information. However, this embodiment aims to enable the network to adaptively focus on either the temporal or spatial information of the input image samples. To alleviate this problem, a dynamic weight reassignment module is proposed, which dynamically adjusts the weights of the convolutional kernels based on the input samples. The formula is as follows:

[0062] α s =f WR (vs ),α s ∈R k (5)

[0063] W d =α s1 w1+...+α sk w k (6)

[0064] In the above formula, f WR W represents a fully connected operation. d This represents the dynamic convolution kernel output by the dynamic weight redistribution module; w is the dynamic convolution kernel W. p Each convolution kernel has k parameters, in total. In this embodiment, k represents the 27 numerical parameters of a single 3*3*3 convolution kernel; α s It is each parameter w i The dynamic weights, α s It is a k-dimensional vector.

[0065] The first-channel dynamic weighting module works as follows: the original features are processed through convolutional layers and softmax layers (i.e., through fully connected layers) in the first-channel dynamic weighting module to generate dynamically weighted features v. in The formula is:

[0066]

[0067] Combined with input frame image samples Where x i =R 1×H×W Using a channel dynamic weighting mechanism, the first feature map after dynamic weighting and updating is obtained as follows:

[0068]

[0069] Then, the dynamic convolutional kernel W generated by the dynamic weight reallocation module is used. d The first feature map X' is processed by performing a convolution operation on the first feature map using the dynamic convolution kernel obtained above, thereby generating the second feature map. The generated second feature map Y' is input into the second channel dynamic weighting module.

[0070] The second-channel dynamic weighting module works as follows: the original features are processed through convolutional layers and softmax layers (i.e., through fully connected layers) in the second-channel dynamic weighting module to generate dynamically weighted features v. out The formula is:

[0071]

[0072] In the dynamically weighted feature v outBased on this, and combined with the input second feature map Y', the final feature Y is obtained, as shown in the formula:

[0073]

[0074] By utilizing a multi-layer HAD-Conv to construct a dynamic self-supervised network, this dynamic self-supervised network can dynamically extract different spatiotemporal information from anomalous video samples.

[0075] The aforementioned multi-layer downsampling encoder layer adaptively and dynamically extracts features based on the input video sample to be detected. The extracted features are then upsampled through a multi-layer upsampling 2D decoder layer, and finally the next frame prediction image of the video sample is output through a 2D conversion layer.

[0076] During the prediction inference process, a dynamic self-supervised network extracts features from normal and abnormal frame images, and uses a decoder to predict future frames. The predicted frames for normal frames are close to the actual future frames, while the predicted frames for abnormal frames are close to the predicted frames for pseudo-abnormal frames. Finally, based on the predicted image of the next frame, the final detection result is output through an discrimination module. In the discrimination module, the peak signal-to-noise ratio (PSNR) of the predicted frame and the actual future frame is calculated, and the PSNR is used to distinguish between normal and abnormal frames.

[0077] In this embodiment, backward gradient updates are performed using loss functions, and distance loss function, reconstruction loss function, gradient loss function, and adversarial loss function are constructed respectively to train the above dynamic self-supervised network.

[0078] The distance loss is defined as follows: Considering the inconsistency between the features of the extracted normal samples and the pseudo-anomaly samples, distance loss is used in the encoder stage to ensure that the network focuses on the parts changed by the pseudo-anomalies. The distance loss function is:

[0079]

[0080] in, These are anomalous regions generated by a pseudo-anomaly sample generator. It is the corresponding region in the normal sample.

[0081] The reconstruction loss is defined as follows: it penalizes the difference in intensity to make the reconstructed video frames from the decoder similar to their corresponding ground truth images. Specifically, it minimizes the L2 distance between the decoder output and the ground truth image. The formula for the reconstruction loss function is:

[0082]

[0083] The gradient loss is used to sharpen the image in order to ensure that the video frames predicted by the decoder are similar to their corresponding real images. The formula for the gradient loss function is:

[0084]

[0085] The adversarial loss is as follows: The goal of training G is to generate a framework for D to classify it into class 1 (representing the true label). When training G, the weights of D are fixed. The loss function is constructed based on the imposed mean absolute error (MAE), and the formula for the adversarial loss function is:

[0086]

[0087] Combining these four loss functions, the final loss function is obtained as follows:

[0088]

[0089] Where, λ int , λ gd , λ dis , λ adv These are the weights for the reconstruction loss, gradient loss, distance loss, and adversarial loss, respectively.

[0090] Through the above training, a trained dynamic self-supervised network is obtained. The obtained video samples to be detected are input into the trained dynamic self-supervised network, and the final normal or abnormal detection result is output.

[0091] To further verify the superiority of the scheme described in this embodiment, experiments were conducted on three benchmark datasets: the CUHK Avenue dataset from the Chinese University of Hong Kong, the pedestrian dataset from UCSD (University of California, San Diego), and the Shanghai Tech dataset. A frame-level evaluation criterion was adopted, and ROC (Receiver Operator Characteristic) curves were plotted. Two important quantitative evaluation metrics, AUC (Area Under Curve) and EER (Equal Error Rate), were calculated from the ROC curves to evaluate the algorithm's performance. A higher AUC value and a lower EER value indicate better algorithm performance.

[0092] The CUHK Avenue dataset contains 16 training videos and 21 test videos, with 15,328 frames in the training videos and 15,324 frames in the test videos. Each video frame has a resolution of 360×640 pixels, and there are a total of 47 anomalous events in the test videos. The test results are shown in Table 1 below.

[0093] Table 1 Comparison Results of the CUHK Avenue Dataset

[0094]

[0095] The UCSD pedestrian dataset consists of two parts: Ped1 and Ped2. Ped1 contains 34 training videos and 36 test videos, with 6800 frames in the training videos and 7200 frames in the test videos. Each video has a resolution of 238×158 pixels, and 40 anomalous events are found in the test videos. Ped2 contains 16 training videos and 12 test videos, with 2550 frames in the training videos and 2010 frames in the test videos. Each video has a resolution of 240×360 pixels. Ped2 is identical to Ped1, and 12 anomalous events were found in the test videos. The test results are shown in Tables 2 and 3 below.

[0096] Table 2 Comparison results of Ped1 dataset

[0097]

[0098]

[0099] Table 3 Comparison results of the Ped2 dataset

[0100]

[0101] The Shanghai Tech dataset contains 330 training videos and 107 test videos, each with a resolution of 480×856 pixels and a total of 316,154 frames. The test videos contain 130 anomalous events, comprising 13 scenes and various anomalous types. The test results are shown in Table 4 below.

[0102] Table 4 Comparison Results of the ShanghaiTech Dataset

[0103]

[0104] Clearly, compared to existing methods, the video anomaly detection method based on dynamic self-supervised networks proposed in this embodiment can achieve better detection performance and results.

[0105] Example 2

[0106] This embodiment provides a surveillance video anomaly detection system based on a dynamic self-supervised network, including:

[0107] The video acquisition module is used to acquire video samples to be detected.

[0108] The detection module is used to input the video samples to be detected into the trained dynamic self-supervised network and output the detection results;

[0109] The dynamic self-supervised network includes a feature reconstruction module and an discrimination module. The feature reconstruction module adopts an encoder-decoder architecture, with multi-layer downsampling encoder layers and multi-layer upsampling decoder layers skipping connections within the same layer.

[0110] All multi-layer downsampling encoder layers employ hybrid anomalous dynamic convolution to adaptively and dynamically extract features based on the input video sample to be detected. The extracted features are then passed through multi-layer upsampling decoder layers to output the next frame prediction image of the video sample to be detected.

[0111] Based on the predicted image of the next frame of the video sample to be detected, the peak signal-to-noise ratio of the predicted frame is calculated by the discrimination module to determine whether the video sample to be detected is an abnormal sample, and the final detection result is output.

[0112] Example 3

[0113] This embodiment provides an electronic device, including a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When the processor executes the computer instructions, it completes the steps in the surveillance video anomaly detection method based on a dynamic self-supervised network as described above.

[0114] Example 4

[0115] This embodiment also provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the steps in the surveillance video anomaly detection method based on a dynamic self-supervised network as described above.

[0116] The steps and methods involved in Embodiments 2 to 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0117] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0118] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for monitoring video anomaly detection based on a dynamic self-supervised network, characterized in that, The method comprises the following steps: acquiring a video sample to be detected, inputting the video sample to be detected into a trained dynamic self-supervised network, and outputting a final detection result; The dynamic self-supervised network comprises a feature reconstruction module and a discrimination module, the feature reconstruction module adopts an encoder-decoder architecture, and the multi-layer down-sampling encoder layer and the multi-layer up-sampling decoder layer are connected by the same layer. The multi-layer down-sampling encoder layer adopts a hybrid abnormal dynamic convolution, dynamically extracts features from the input video sample to be detected, and the extracted features are outputted to the next frame of the video sample to be detected through the multi-layer up-sampling decoder layer; the hybrid abnormal dynamic convolution comprises an aggregated dynamic weight module, a dynamic weight redistribution module, a first channel dynamic weight module and a second channel dynamic weight module, and a feature compression module and a generated dynamic convolution module; the input frame image sample enters the hybrid abnormal dynamic convolution, and image features are extracted through the average pooling layer, the convolution layer and the activation layer in the feature compression module to generate an original feature map; The original feature map is inputted into the aggregated dynamic weight module and the dynamic weight redistribution module, respectively, and the final convolution kernel weight is dynamically adjusted and determined according to the input sample; at the same time, the original feature map and the original input frame image are inputted into the first channel dynamic weight module to generate a first feature map after dynamic feature update; The first feature map is inputted into the generated dynamic convolution module to generate a second feature map after dynamic feature update in combination with the determined convolution kernel weight; The second feature map and the original feature map are inputted into the second channel dynamic weight module to generate a third feature map after dynamic feature update, which is the final feature map outputted by the hybrid abnormal dynamic convolution; Based on the next frame prediction image of the video sample to be detected, the discrimination module calculates the peak signal-to-noise ratio of the prediction frame to determine whether the video sample to be detected is an abnormal sample, and outputs the final detection result. 2.The method of claim 1, wherein the method further comprises, The training process of the dynamic self-supervised network comprises the following steps: inputting a normal video sample into a time-space pseudo abnormal sample generator to obtain a spatial pseudo abnormal video sample and a temporal pseudo abnormal video sample; inputting the spatial pseudo abnormal video sample, the temporal pseudo abnormal video sample and the normal video sample into the dynamic self-supervised network as training set samples, training the dynamic self-supervised network, and enabling the dynamic self-supervised network to learn the difference between the normal video sample and the abnormal video sample, and outputting the next frame image of the input video sample. 3.The method of claim 2, wherein the method further comprises: The training process of the dynamic self-supervised network comprises the following steps: A normal video sample sequence is input into a temporal pseudo abnormal sample generator, and a random integer is extracted n , from the end of the whole video sample t - n frame or t + n frame, for t - n frame video sample sequence, a random copy n frame; for t + n frame video sample sequence, a random deletion n frame, a temporal pseudo abnormal video sample sequence with t frames is generated; A normal video sample sequence is input into the spatial pseudo abnormal sample generator, the last t frame video sample sequence is selected, and a random rectangular mask operation is performed on the sequence using Gaussian noise to generate a spatial pseudo abnormal video sample sequence with one t frame.

4. The method for detecting anomalies in surveillance videos based on dynamic self-supervised networks as described in claim 1, characterized in that, using a loss function to perform reverse gradient update, respectively constructing a distance loss function, a reconstruction loss function, a gradient loss function and an adversarial loss function to train the dynamic self-supervised network. 5.A system for monitoring video anomaly detection based on dynamic self-supervised network, characterized in that, The method comprises the following steps: a video to be detected acquisition module for acquiring a video sample to be detected; a detection module for inputting the video sample to be detected into a trained dynamic self-supervised network, and outputting a detection result. The dynamic self-supervised network comprises a feature reconstruction module and a discrimination module, the feature reconstruction module adopts an encoder-decoder architecture, and a plurality of layers of down-sampling encoder layers and a plurality of layers of up-sampling decoder layers are connected by layer skipping; The plurality of layers of down-sampling encoder layers each adopt a hybrid abnormal dynamic convolution, features are adaptively extracted dynamically according to the input video sample to be detected, and the extracted features are output to the next frame of the video sample to be detected through the plurality of layers of up-sampling decoder layers; the hybrid abnormal dynamic convolution comprises an aggregated dynamic weight module, a dynamic weight redistribution module, a first channel dynamic weight module and a second channel dynamic weight module, and a feature compression module and a generated dynamic convolution module; the input frame image sample enters the hybrid abnormal dynamic convolution, image features are extracted through an average pooling layer, a convolution layer and an activation layer in the feature compression module, and an original feature map is generated; The original feature map is input to the aggregated dynamic weight module and the dynamic weight redistribution module respectively, and the final convolution kernel weight is dynamically adjusted and determined according to the input sample; at the same time, the original feature map and the original input frame image are input to the first channel dynamic weight module to generate a first feature map after dynamic feature update; The first feature map is input to the generated dynamic convolution module, and a second feature map after dynamic feature update is generated in combination with the determined convolution kernel weight; The second feature map and the original feature map are input to the second channel dynamic weight module to generate a third feature map after dynamic feature update, which is the final feature map output by the hybrid abnormal dynamic convolution; Based on the next frame of the predicted image of the video sample to be detected, the peak signal-to-noise ratio of the predicted frame is calculated through the discrimination module to determine whether the video sample to be detected is an abnormal sample, and the final detection result is output.

6. The dynamic self-supervised network based surveillance video anomaly detection system of claim 5, wherein, The training process of the dynamic self-supervised network comprises: inputting normal video samples into a time-space pseudo-abnormal sample generator to obtain spatial pseudo-abnormal video samples and time pseudo-abnormal video samples; inputting the spatial pseudo-abnormal video samples, the time pseudo-abnormal video samples and the normal video samples as training set samples into the dynamic self-supervised network, training the dynamic self-supervised network, making the dynamic self-supervised network learn the difference between the normal video samples and the abnormal video samples, and outputting the next frame of the predicted image of the input video sample.

7. An electronic device, characterized by comprising: A memory and a processor, and computer instructions stored in the memory and running on the processor, when the computer instructions are run by the processor, complete the steps of the monitoring video abnormal detection method based on the dynamic self-supervised network in any one of claims 1-4.

8. A computer readable storage medium characterized by, A computer instruction storage device, when the computer instruction is executed by the processor, completes the steps of the monitoring video abnormal detection method based on the dynamic self-supervised network in any one of claims 1-4.

Images