Video anomaly detection method based on wavelet transform and multi-scale feature fusion

Abstract

The invention discloses a video anomaly detection method based on wavelet transform and multi-scale feature fusion, which comprises the following steps: acquiring a video data set, splitting the video data set into a video frame data set, performing size adjustment and pixel value normalization processing on the video frame data set, and dividing the video frame data set into a training set and a test set; a U-Net video anomaly detection network model is constructed, the U-Net video anomaly detection network model comprises an encoder, a decoder and a memory module, a hierarchical offset connection structure is added between the encoder and the decoder, a comprehensive loss function is constructed to train the model, the U-Net video anomaly detection network model is deployed, and anomaly detection is performed on a real-time video. And the U-Net video anomaly detection network model judges whether the video has an anomaly phenomenon in real time. The method can effectively improve the detection precision and robustness in a complex dynamic scene.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a video anomaly detection method based on wavelet transform and multi-scale feature fusion. Background Art

[0002] Detecting anomalies in video sequences (such as vehicles intruding on sidewalks) has become a hot topic in the field of surveillance systems and fault detection over the past decade. The challenges of this problem are mainly reflected in the following: first, the definition of abnormal events is highly context-dependent—for example, a car driving on a motor vehicle lane is normal behavior, while driving on a non-motor vehicle lane or sidewalk is considered abnormal; second, the cost of constructing anomaly datasets is high because the frequency of abnormal events in real-world scenarios is extremely low. Therefore, anomaly detection is often regarded as an unsupervised (or semi-supervised) learning task. Its core logic is to construct a normality description model based only on a training set of normal samples. During the testing phase, events that the model cannot effectively describe are classified as anomalies.

[0003] In unsupervised learning tasks, the U-Net network model was initially applied to semantic segmentation of medical images. This U-Net model, an encoder-decoder (a symmetric U-shaped architecture) that uses skip connections to achieve multi-scale feature fusion, has achieved excellent performance in segmentation tasks. In recent years, this architecture has been gradually introduced into the field of video anomaly detection.

[0004] FuturePred is the first study to use a U-Net network to predict future frames for anomaly detection. This method feeds the previous t frames of a video into a U-Net network, predicts the content of frame t+1, and calculates the difference between the predicted frame and the actual frame to generate an anomaly score, thereby determining whether the frame is anomalous.

[0005] Mem-AE builds on U-Net by introducing a memory module. This module, located between the encoder and decoder, plays a key role in storing and managing normal feature patterns. Architecturally, it comprises a memory bank that stores normal features as key-value pairs. After the encoder converts video frame data into feature vectors (i.e., queries), the memory module begins its operation. It compares these query items with the memory items in the memory bank one by one using methods such as Euclidean distance and cosine similarity. While this approach enhances the model's ability to memorize normal features, it also reduces overfitting and improves generalization performance. However, because it still uses the same-layer skip connection mechanism of U-Net, it only enables feature fusion between corresponding encoder and decoder layers, making it difficult to effectively integrate fine-grained and coarse-grained features across layers. This results in the model failing to adequately capture the diversity of normal behavior, such as at different viewpoints and scales.

[0006] In summary, existing U-Net-style video anomaly detection models mainly focus on the generalization ability of suppressing abnormal events, but ignore the problem of insufficient fusion of multi-scale feature information, resulting in the model failing to fully capture the diversity and differences of normal patterns. Summary of the Invention

[0007] In order to overcome the problems in the prior art, the purpose of the present invention is to provide a video anomaly detection method based on wavelet transform and multi-scale feature fusion, which can effectively improve the detection accuracy and robustness in complex dynamic scenes.

[0008] To achieve the above object, the present invention provides a video anomaly detection method based on wavelet transform and multi-scale feature fusion, comprising the following steps:

[0009] S1: Obtain a video dataset, split the video dataset into a video frame dataset, resize the video frame dataset and normalize the pixel values, and divide the video frame dataset into a training set and a test set;

[0010] S2: Construct a U-Net video anomaly detection network model, which includes an encoder, a decoder, and a memory module. The encoder and decoder include a multi-level structure. Each level of the encoder includes a convolution block and a Haar wavelet transform downsampling module. The output end of the convolution block is connected to the input end of the Haar wavelet transform downsampling module. The Haar wavelet transform module performs multi-scale feature extraction on the input video frame.

[0011] Each level of the decoder includes a multi-scale feature fusion module, a discrete wavelet transform feature fusion module, a convolution block and an upsampling module connected in sequence. The multi-scale feature fusion module is used to receive the feature map output by the encoder at the same level and the feature map output by the upsampling module, and perform feature fusion on the two to output a large-scale feature map. The discrete wavelet transform feature fusion module is used to receive the large-scale feature map and the small-scale feature map output by the encoder of the next layer, and perform wavelet domain feature decomposition and fusion on the two to generate a reconstructed feature map and output it to the convolution block.

[0012] The memory module is deployed between the encoder and the decoder and includes a dynamic memory bank for filtering and enhancing normal feature representations and suppressing the propagation of abnormal features by storing normal feature patterns and calculating the similarity between input features and stored items;

[0013] S3: Constructing a comprehensive loss function L total , train the U-Net video anomaly detection network model through the training set;

[0014] S4: Use the test set to calculate the anomaly score of the predicted frame output by the U-Net video anomaly detection network model, and determine the optimal anomaly score threshold τ by constructing the ROC curve * , get the optimal U-Net video anomaly detection network model;

[0015] S5: Deploy the optimal U-Net video anomaly detection network model to perform anomaly detection on real-time videos. The optimal U-Net video anomaly detection network model determines in real time whether there are any anomalies in the video.

[0016] Furthermore, in step S1, the method of resizing and normalizing the pixel values ​​of the video frame dataset and dividing the video frame dataset into a training set and a test set is as follows:

[0017] The sizes of all data frames in the video frame dataset are adjusted to a uniform size, and the pixel values ​​are normalized. The video frame dataset is divided into a training set containing only normal phenomena and a test set containing abnormal phenomena. The test set uses binary labels to mark whether each frame has abnormal phenomena.

[0018] Furthermore, in step S2, the Haar wavelet transform downsampling module includes a lossless feature encoding block and a spatial domain feature representation block connected in sequence, and the frequency domain lossless feature encoding block is used to Figure X 1 Perform multi-scale decomposition to generate intermediate features containing three high-frequency detail components and one low-frequency approximate component Figure X 2; among them, R represents the real tensor element, C represents the number of channels, H represents the vertical pixel value of the feature map, and W represents the horizontal pixel value of the feature map;

[0019] The spatial domain feature representation block includes a convolution layer, a normalization layer, and an activation layer arranged in sequence, which is used to perform channel compression and nonlinear transformation on the intermediate feature X2 to obtain the output feature Figure X 3, among which,

[0020] Furthermore, the frequency domain lossless feature encoding block is used to Figure X 1 Perform multi-scale decomposition to generate intermediate features containing high-frequency detail components and low-frequency approximate components Figure X The specific method of 2 is:

[0021] Construct a low-pass filter H0 in the horizontal direction and a high-pass filter H1 in the vertical direction, and use the low-pass filter H0 and the high-pass filter H1 to filter the input features. Figure X 1. Convolution operations are performed in the horizontal and vertical directions respectively. Each convolution output is downsampled by 2 times to generate four orthogonal sub-band components, which are low-frequency approximation components and high-frequency detail components representing the horizontal direction Characterizes the high-frequency detail components in the vertical direction and the high-frequency detail components representing the diagonal direction Concatenate the channel dimensions to generate intermediate feature maps The mathematical expressions for generating the low-frequency approximation component LL and the high-frequency detail component LH representing the horizontal direction, the high-frequency detail component HL representing the vertical direction, and the high-frequency detail component HH representing the diagonal direction are as follows:

[0022]

[0023] in, Represents a tensor outer product operation, and * represents a two-dimensional convolution operation.

[0024] Furthermore, in step S2, the discrete wavelet transform feature fusion module includes a multi-scale decomposition module, a low-frequency feature enhancement module, a high-frequency feature fusion module and a multi-scale reconstruction module.

[0025] The input end of the multi-scale decomposition module receives the large-scale feature map output by the multi-scale feature fusion module The multi-scale decomposition module is used to decompose large-scale features Figure X 4. Perform discrete wavelet transform decomposition to transform the input large-scale features Figure X 4 is decomposed into a low-frequency approximation component and three high-frequency detail components representing the diagonal, vertical, and horizontal directions; the formula is expressed as:

[0026] {HH,HL,LH,LL}=DWT(X1)

[0027] in, is the low-frequency approximation component, HH is the high-frequency detail component representing the diagonal direction, HL is the high-frequency detail component representing the vertical direction, LH is the high-frequency detail component representing the horizontal direction, and DWT is discrete wavelet transform;

[0028] The input end of the low-frequency feature enhancement module is connected to the small-scale feature Figure X 5 and the low-frequency approximate component LL output by the multi-scale decomposition module, the low-frequency feature enhancement module is used to convert the small-scale features output by the encoder Figure X 5 is concatenated with the low-frequency approximate component LL, and then the features are enhanced through convolution and residual connection to obtain the low-frequency enhanced feature map E2;

[0029] The high-frequency feature fusion module receives the high-frequency detail component HH representing the diagonal direction, the high-frequency detail component HL in the vertical direction, and the high-frequency detail component LH in the horizontal direction output by the low-frequency feature enhancement module. The high-frequency feature fusion module is used to perform channel compression and nonlinear transformation on the high-frequency detail component to generate a high-frequency discriminant feature map F3;

[0030] The multi-scale reconstruction module receives the low-frequency enhancement feature map E2 output by the low-frequency feature enhancement module and the high-frequency discriminant feature map F3 output by the high-frequency feature fusion module. The multi-scale reconstruction module is used to splice the low-frequency enhancement feature map E2 and the high-frequency discriminant feature map F3 to generate a reconstructed feature map Y1.

[0031] Furthermore, the low-frequency feature enhancement includes a first splicing layer, a first convolutional layer and a first residual layer connected in sequence, wherein the first splicing layer is used to convert the small-scale features Figure X 5 is spliced ​​with the low-frequency approximate component LL along the channel dimension to generate a low-frequency splicing feature map E0

[0032] E0=Concat(X5,LL);

[0033] The first convolutional layer is used to extract features from the low-frequency splicing feature map E0 through 3×3 convolution to generate a convolution feature map E1. The formula is expressed as:

[0034] E1 = Conv3 × 3 (E0);

[0035] The first residual layer is used to perform feature enhancement on the convolution feature map E1 to generate a low-frequency enhanced feature map E2, which is expressed as: E2=E1+Conv3×3(E1).

[0036] Furthermore, the high-frequency feature fusion module is constructed, which includes the second splicing layer, the second convolutional layer, the second residual layer and the third convolutional layer connected in sequence. The second splicing layer is used to splice the high-frequency detail components {HH, HL, LH} along the channel dimension to generate a high-frequency splicing feature map F0, which is expressed as follows:

[0037] F0=Concat(HH, HL, LH);

[0038] The second convolutional layer is used to perform channel compression on the high-frequency splicing feature map F0 through 1×1 convolution to generate a high-frequency compressed feature map F1. The formula is expressed as:

[0039] F1 = Conv1 × 1 (F0);

[0040] The second residual layer is used to perform nonlinear transformation on the high-frequency compression feature map F1 by introducing a residual connection to generate a high-frequency residual feature map F2. The formula is expressed as:

[0041] F2 = F1 + Conv3 × 3 (F1);

[0042] The third convolutional layer is used to extract discriminative features from the high-frequency residual feature map F2 through 1×1 convolution to generate a high-frequency discriminative feature map F3. The formula is expressed as:

[0043] F3=F2+Conv1×1(F2).

[0044] Furthermore, the multi-scale reconstruction module includes a feature integration layer and a scale recovery layer connected in sequence. The feature integration layer is used to splice the low-frequency enhancement feature map E2 and the high-frequency discrimination feature map F3 along the channel dimension to generate an integrated feature map Y0, which is expressed as follows:

[0045] Y0=Concat(E2,F3);

[0046] The scale recovery layer is used to perform inverse discrete wavelet transform on the integrated feature map Y0 to restore the large-scale features. Figure X 4The same spatial resolution is obtained to reconstruct the feature map Y1, the formula is expressed as:

[0047]

[0048] Among them, IDWT is inverse discrete wavelet transform.

[0049] Further, in step S3, the comprehensive loss function L is constructed total The following steps are involved:

[0050] S31: Build strength loss L int Measuring the pixel-level difference between the predicted frame and the real frame, the formula is expressed as:

[0051]

[0052] Among them, ||·||2 represents the Euclidean distance, is the predicted frame at time t, I t is the real frame at time t;

[0053] S32: Construct feature compactness loss L compact Force query items Minimize the feature distance to the nearest neighbor storage item and enhance the compact clustering of normal patterns. The formula is expressed as:

[0054]

[0055] The index q is determined by:

[0056]

[0057] Where t is the time step index, T is the time step set, t∈T; k is the query item index, K is the number of query items in a single time step, k∈K; is the kth query item at time t, p q For memory The storage item with the closest Euclidean distance, the index q of the nearest neighbor storage item is calculated by weight Determine, M is the set of memory storage items;

[0058] S33: Constructing feature separation loss L separate Constrain similar query items to be mapped to the same storage item, suppressing the generation of redundant storage items. The formula is expressed as:

[0059]

[0060] The index n is determined by:

[0061]

[0062] Where t is the time step index, T is the time step set, t∈T; k is the query item index, K is the number of query items in a single time step, k∈K; is the kth query item at time t, p n For memory The Euclidean distance is the next closest storage item feature, and m≠q, α is the interval threshold, is the set of items in the memory except q, [x] + is the RELU function, taking the maximum number between x and 0;

[0063] S34: Loss of strength L int , feature compactness loss L compact and feature separation loss L separate Combined to form a comprehensive loss function L total , the formula is:

[0064]

[0065] Among them, L total is the comprehensive loss, L int is the strength loss, L compact is the feature compactness loss, L separate is the feature separation loss; int ,λ c ,λ s Represents the weight of each loss.

[0066] Furthermore, in step S4, the method of calculating the anomaly score of the predicted frame output by the U-Net video anomaly detection network model using the test set and determining the optimal anomaly score threshold by constructing a ROC curve includes the following steps:

[0067] S41: Calculate the Euclidean distance between the query item and the nearest stored item. The formula is:

[0068]

[0069] in, is the kth query item at time t, p q For memory The stored item with the closest Euclidean distance, K is the total number of query items in a single time step,

[0070] S42: Calculate the peak signal-to-noise ratio (PSNR) between the real frame and the predicted frame. The peak signal-to-noise ratio (PSNR) formula is expressed as:

[0071]

[0072] in, is the predicted frame at time t, I t is the real frame at time t; is the maximum pixel value of the predicted frame, and N is the total number of pixels per frame;

[0073] S43: D(q t ,p) and The value of is normalized to the range of [0,1], and the formula is expressed as:

[0074]

[0075] Among them, g(·) represents normalization, mapping the value to the interval [0,1];

[0076] S44: Final use formula g(D(q t ,p)) and g(PSNR(I t ,I t The sum of the two metrics is the final anomaly score S t , the formula is:

[0077] S t =λ(1-g(PSNR(I t ,I t )))+(1-λ)g(D(q t ,p)),

[0078] Among them, λ is a hyperparameter that balances the peak signal-to-noise ratio PSNR and the L2 distance weight, S tis the final anomaly score at time t;

[0079] S45: Sort the final anomaly scores τ∈[0,1] of all samples in the test set, traverse all anomaly score thresholds τ∈[0,1], calculate the true positive rate TPR and false positive rate FPR, and draw the ROC curve with FPR on the horizontal axis and TPR on the vertical axis. The formula is:

[0080]

[0081] Among them, TP is a true positive example, FN is a false negative example, FP is a false positive example, and TN is a true negative example.

[0082] S46: Calculate the area under the ROC curve (AUC) value using the trapezoidal method, and select the model with the largest AUC value as the optimal U-Net video anomaly detection network model.

[0083] The present invention embeds the Haar wavelet transform into the downsampling of the decoder, effectively retaining high-frequency detail information while reducing feature redundancy, overcoming the problem of key feature loss caused by traditional convolution operations; by designing a cross-layer discrete wavelet feature fusion module, low-frequency components are used to construct feature transfer channels and high-frequency components to enhance detail expression, thereby achieving collaborative optimization of multi-scale structural information. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1 This is a flow chart of a video anomaly detection method based on wavelet transform and multi-scale feature fusion according to an embodiment of the present invention;

[0085] Figure 2 This is a structural diagram of the U-Net video anomaly detection network model according to an embodiment of the present invention;

[0086] Figure 3 This is a structural diagram of a Haar wavelet transform downsampling module HWD in an embodiment of the present invention;

[0087] Figure 4 A structural comparison diagram of the skip connection of the encoder in the embodiment of the present invention and the skip connection in the prior art;

[0088] Figure 5 This is a structural diagram of a discrete wavelet transform feature fusion module DWTF in an embodiment of the present invention;

[0089] Figure 6 Comparison of the AUC% results of the method described in the examples of the present invention with other methods on different data sets;

[0090] Figure 7 This is the impact of the Haar wavelet transform downsampling module HWD on the accuracy of the U-Net video anomaly detection network model in an embodiment of the present invention;

[0091] Figure 8 The effect of the discrete wavelet transform feature fusion module DWTF on the accuracy of the U-Net video anomaly detection network model in the embodiment of the present invention;

[0092] Figure 9 The impact of the discrete wavelet transform feature fusion module (DWTF) and the Haar wavelet transform downsampling module HWD on the accuracy of the U-Net video anomaly detection network model in the embodiment of the present invention;

[0093] Figure 10 is the error heat map of the real frame and the predicted frame in the embodiment of the present invention;

[0094] Figure 11 This is a graph showing the anomaly scores for the first four hundred frames of the UCSD Ped2 and CUHK Avenue test sets in an embodiment of the present invention. DETAILED DESCRIPTION

[0095] The present invention provides a video anomaly detection method based on wavelet transform and multi-scale feature fusion, such as Figure 1 As shown, the following steps are included:

[0096] S1: Obtain a video dataset, split the video dataset into a video frame dataset, resize and normalize the pixel values ​​of the video frame dataset, and divide the video frame dataset into a training set and a test set.

[0097] S2: Constructing a U-Net video anomaly detection network model, the U-Net video anomaly detection network model includes an encoder, a decoder, and a memory module. The encoder and decoder have a multi-level structure. Each level of the encoder includes a convolution block and a Haar Wavelet Downsampling (HWD) module. The output of the convolution block is connected to the input of the Haar Wavelet Downsampling module. The Haar Wavelet Downsampling module performs multi-scale feature extraction on the input video frame.

[0098] Each level of the decoder includes a multi-scale feature fusion module, a discrete wavelet transform feature fusion module (DWTF), a convolution block and an upsampling module connected in sequence. The multi-scale feature fusion module is used to receive the feature map output by the encoder at the same level and the feature map output by the upsampling module, and perform feature fusion on the two to output a large-scale feature map. The discrete wavelet transform feature fusion module is used to receive the large-scale feature map and the small-scale feature map output by the encoder of the next layer, and perform wavelet domain feature decomposition and fusion on the two to generate a reconstructed feature map and output it to the convolution block.

[0099] The memory module is deployed between the encoder and the decoder, and includes a dynamic memory library for screening and enhancing normal feature representations and suppressing the propagation of abnormal features by storing normal feature patterns and calculating the similarity between input features and stored items.

[0100] In the structural description of the U-Net network model encoder and decoder, the lower layer refers to the deeper network layer, close to the bottleneck layer; the upper layer refers to the shallower network layer, close to the input / output end of the network model.

[0101] S3: Construct a loss function and train the U-Net video anomaly detection network model using the training set;

[0102] The inclusion of strength loss L int , feature compactness loss L compact and feature separation loss L separate The comprehensive loss function optimizes the U-Net video anomaly detection network model;

[0103] S4: Use the test set to calculate the anomaly score of the predicted frame output by the U-Net video anomaly detection network model, and determine the optimal anomaly score threshold τ by constructing the ROC curve * , get the optimal U-Net video anomaly detection network model;

[0104] S5: Deploy the optimal U-Net video anomaly detection network model to perform anomaly detection on real-time videos. The optimal U-Net video anomaly detection network model determines in real time whether there are any anomalies in the video.

[0105] In step S1, the method of resizing and normalizing the pixel values ​​of the video frame dataset and dividing the video frame dataset into a training set and a test set is as follows:

[0106] All video frames in the video frame dataset were resized to 256×256, and pixel values ​​were normalized to the range of [-1, 1]. The video frame dataset was divided into a training set containing only normal events and a test set containing abnormal events. The test set used a binary label (0 / 1) to indicate whether each frame had an abnormal event. In this embodiment, the UCSD Ped2 and Avenue datasets were preferably used. The UCSD Ped2 data was collected from a campus sidewalk surveillance scene. The camera was fixed at a high altitude overlooking the sidewalk, with the view perpendicular to the ground, and pedestrian movement was generally parallel to the camera plane. The crowd density in the scene ranged from sparse to highly crowded, simulating a real-world pedestrian flow environment. The Avenue dataset was collected from a campus corridor surveillance scene. The camera was fixed, and pedestrians moved primarily parallel to the camera plane, with their size dynamically changing depending on their distance from the camera. The UCSD Ped2 dataset was divided into 16 training videos (approximately 3,200 frames, all normal events) and 12 test videos (approximately 2,400 frames). The Avenue dataset is divided into 16 training videos (about 15,328 frames) and 21 test videos (about 15,324 frames). All test set video frames are annotated with binary labels (0 / 1) to indicate abnormal phenomena.

[0107] In step S2, the Haar wavelet transform downsampling module includes a lossless feature encoding block and a spatial domain feature representation block connected in sequence. Its structure is as follows Figure 3 shown.

[0108] The frequency domain lossless feature coding block (HWT) is used to Figure X 1 Perform multi-scale decomposition to generate intermediate features containing three high-frequency detail components and one low-frequency approximate component Figure X 2; among them, Here, R represents a real tensor element, C represents the number of channels, H represents the vertical pixel value of the feature map, and W represents the horizontal pixel value of the feature map. R, C, H, and W are used repeatedly in the following text to represent the same meanings as here and are not repeated here.

[0109] The spatial domain feature representation block (FR) includes a convolution layer, a normalization layer and an activation layer arranged in sequence, which are used to represent the intermediate features. Figure X 2 Perform channel compression and nonlinear transformation to obtain output features Figure X 3, among which,

[0110] The method of constructing a frequency domain lossless feature coding block (HWT) for performing multi-scale decomposition on the input feature map X1 to generate an intermediate feature map X2 containing high-frequency detail components and low-frequency approximation components is specifically as follows:

[0111] Construct a low-pass filter H0 in the horizontal direction and a high-pass filter H1 in the vertical direction, and use the low-pass filter H0 and the high-pass filter H1 to filter the input features. Figure X 1. Convolution operations are performed in the horizontal and vertical directions respectively. Each convolution output is downsampled by 2 times to generate four orthogonal sub-band components, which are low-frequency approximation components and high-frequency detail components representing the horizontal direction Characterizes the high-frequency detail components in the vertical direction and the high-frequency detail components representing the diagonal direction Concatenate the channel dimensions to generate intermediate feature maps The mathematical expressions for generating the low-frequency approximation component LL and the high-frequency detail component LH representing the horizontal direction, the high-frequency detail component HL representing the vertical direction, and the high-frequency detail component HH representing the diagonal direction are as follows:

[0112]

[0113] in, Represents a tensor outer product operation, and * represents a two-dimensional convolution operation.

[0114] In step S2, a cross-layer skip connection is added based on the same-layer skip connection of the benchmark U-Net network model. The structure of the skip connection of the encoder in this embodiment is compared with the structure of the skip connection in the prior art. Figure 4 As shown, Figure 4 In the basic U-Net structure of the existing technology on the left, the encoder (C1-C4) and decoder (D1-D4) form a strict correspondence through the same-level jump connections established by the red dotted arrows (such as C1→D1, C2→D2). This single-scale feature interaction mechanism leads to the separation of deep semantic information and shallow detailed features.

[0115] Figure 4 The right side shows the innovative architecture of the present invention. By introducing the blue-labeled discrete wavelet transform fusion (DWTF) module and the cross-level connection formed by the red diagonal arrows, it breaks through the limitations of the traditional structure: at the decoding layer D1, in addition to receiving the same-level D1 features (red solid arrow), a new cross-level feature flow from the C2 layer is added (red C2→D1 solid arrow), forming a hierarchical offset connection structure between the encoder and decoder.

[0116] The discrete wavelet transform feature fusion module DWTF includes a multi-scale decomposition module, a low-frequency feature enhancement module, a high-frequency feature fusion module and a multi-scale reconstruction module. Its structure is as follows: Figure 5 shown.

[0117] Specifically, the input end of the multi-scale decomposition module receives the large-scale feature map output by the multi-scale feature fusion module. The multi-scale decomposition module is used to decompose the large-scale features of the input Figure X 4. Perform discrete wavelet transform decomposition to transform the input large-scale features Figure X 4 is decomposed into a low-frequency approximation component and three high-frequency detail components representing the diagonal, vertical, and horizontal directions; the formula is expressed as:

[0118] {HH,HL,LH,LL}=DWT(X1)

[0119] in, is the low-frequency approximation component, HH is the high-frequency detail component representing the diagonal direction, HL is the high-frequency detail component representing the vertical direction, LH is the high-frequency detail component representing the horizontal direction, and DWT is discrete wavelet transform;

[0120] The input end of the low-frequency feature enhancement module is connected to the small-scale feature Figure X 5 and the low-frequency approximate component LL output by the multi-scale decomposition module, the low-frequency feature enhancement module includes a first splicing layer, a first convolution layer and a first residual layer in sequence, for converting the small-scale features output by the encoder into Figure X 5 is concatenated with the low-frequency approximate component LL, and then the features are enhanced through convolution and residual connection to obtain the low-frequency enhanced feature map E2.

[0121] The first splicing layer is used to convert small-scale features Figure X 5 is spliced ​​with the low-frequency approximate component LL along the channel dimension to generate a low-frequency splicing feature map E0

[0122] E0=Concat(X5,LL).

[0123] The first convolutional layer extracts features from the low-frequency splicing feature map E0 through 3×3 convolution to generate a convolution feature map E1. The formula is expressed as:

[0124] E1=Conv3×3(E0).

[0125] The first residual layer is used to enhance the convolution feature map E1 to generate a low-frequency enhanced feature map E2. The formula is expressed as:

[0126] E2=E1+Conv3×3(E1).

[0127] The high-frequency feature fusion module receives the high-frequency detail component HH representing the diagonal direction, the high-frequency detail component HL in the vertical direction, and the high-frequency detail component LH in the horizontal direction output by the low-frequency feature enhancement module. The high-frequency feature fusion module includes a second splicing layer, a second convolution layer, a second residual layer, and a third convolution layer connected in sequence, which are used to perform channel compression and nonlinear transformation on the high-frequency detail components to generate a high-frequency discriminant feature map F3.

[0128] The second splicing layer is used to splice the high-frequency detail components {HH, HL, LH} along the channel dimension to generate a high-frequency splicing feature map F0, which is expressed as:

[0129] F0 = Concat(HH, HL, LH).

[0130] The second convolutional layer is used to perform channel compression on the high-frequency splicing feature map F0 through 1×1 convolution to generate a high-frequency compressed feature map F1. The formula is expressed as:

[0131] F1=Conv1×1(F0).

[0132] The second residual layer is used to introduce a residual connection to perform nonlinear transformation on the high-frequency compression feature map F1 to generate a high-frequency residual feature map F2. The formula is expressed as:

[0133] F2=F1+Conv3×3(F1).

[0134] The third convolutional layer is used to extract discriminative features from the high-frequency residual feature map F2 through 1×1 convolution to generate a high-frequency discriminative feature map F3. The formula is expressed as:

[0135] F3=F2+Conv1×1(F2).

[0136] The multi-scale reconstruction module receives the low-frequency enhancement feature map E2 output by the low-frequency feature enhancement module and the high-frequency discriminant feature map F3 output by the high-frequency feature fusion module. The multi-scale reconstruction module includes a third splicing layer and a scale recovery layer connected in sequence. The multi-scale reconstruction module is used to splice the low-frequency enhancement feature map E2 and the high-frequency discriminant feature map F3, and generate a reconstructed feature map Y1.

[0137] The third splicing layer is used to splice the low-frequency enhancement feature map E2 and the high-frequency discrimination feature map F3 along the channel dimension to generate an integrated feature map Y0, which is expressed as follows:

[0138] Y0=Concat(E2, F3).

[0139] The scale recovery layer is used to perform inverse discrete wavelet transform on the integrated feature map Y0 to restore the large-scale features. Figure X 4The same spatial resolution is obtained to reconstruct the feature map Y1, the formula is expressed as:

[0140]

[0141] Among them, IDWT is inverse discrete wavelet transform.

[0142] In step S3, the construction includes the strength loss L int , feature compactness loss Lcompact and feature separation loss L separate The comprehensive loss function L total The method comprises the following steps:

[0143] S31: Build strength loss L int Measuring the pixel-level difference between the predicted frame and the real frame, the formula is expressed as:

[0144]

[0145] Among them, ||·||2 represents the Euclidean distance, is the predicted frame at time t, I t is the real frame at time t;

[0146] S32: Construct feature compactness loss L compact Force query items Minimize the feature distance to the nearest neighbor storage item and enhance the compact clustering of normal patterns. The formula is expressed as:

[0147]

[0148] The index q is determined by:

[0149]

[0150] Where t is the time step index, T is the time step set, t∈T; k is the query item index, K is the number of query items in a single time step, k∈K; is the kth query item at time t, p q For memory The storage item with the closest Euclidean distance, the index q of the nearest neighbor storage item is calculated by weight Determine that M is a set of memory storage items.

[0151] S33: Constructing feature separation loss L separate Constrain similar query items to be mapped to the same storage item, suppressing the generation of redundant storage items. The formula is expressed as:

[0152]

[0153] The index n is determined by:

[0154]

[0155] Where t is the time step index, T is the time step set, t∈T; k is the query item index, K is the number of query items in a single time step, k∈K; is the kth query item at time t, p n For memory The Euclidean distance is the next closest storage item feature, and m≠q, α is the interval threshold, which is used to control the boundary of feature separation. is the set of items in the memory except q, [x] + is the RELU function, taking the maximum number between x and 0.

[0156] S34: Loss of strength L int , feature compactness loss L compact and feature separation loss L separate Combined to get the comprehensive loss function L total , the formula is:

[0157]

[0158] Among them, L total is the comprehensive loss, L int is the strength loss, L compact is the feature compactness loss, L separate is the feature separation loss; int ,λ c ,λ s In step S4, the method of calculating the anomaly score of the predicted frame output by the U-Net video anomaly detection network model using the test set and determining the optimal anomaly score threshold by constructing a ROC curve includes the following steps:

[0159] S41: Calculate the Euclidean distance between the query item and the nearest stored item. The formula is:

[0160]

[0161] in, is the kth query item at time t, p q For memory The stored item with the closest Euclidean distance, K is the total number of query items in a single time step.

[0162] S42: Calculate the peak signal to noise ratio (PSNR) between the real frame and the predicted frame to measure the image reconstruction quality. The peak signal to noise ratio (PSNR) formula is expressed as:

[0163]

[0164] in, is the predicted frame at time t, I t is the real frame at time t; is the maximum pixel value of the predicted frame, N is the total number of pixels in each frame, and the smaller the PSNR, the greater the probability that the predicted frame is abnormal.

[0165] S43: D(q t ,p) and The value of is normalized to the range of [0,1], and the formula is expressed as:

[0166]

[0167] Among them, g(·) represents normalization, which maps the value to the interval [0,1].

[0168] S44: Final use formula g(D(q t ,p)) and g(PSNR(I t ,I t The sum of the two metrics is the final anomaly score S t , the formula is:

[0169] S t =λ(1-g(PSNR(I t ,I t )))+(1-λ)g(D(q t ,p)),

[0170] Among them, λ is a hyperparameter that balances the peak signal-to-noise ratio PSNR and the L2 distance weight, S t is the final anomaly score at time t.

[0171] S45: Final anomaly score S for all samples in the test set t Sort and traverse all anomaly score thresholds S t , calculate the true positive rate TPR and false positive rate FPR, and draw the ROC curve, with the horizontal axis being FPR and the vertical axis being TPR. The formula is:

[0172]

[0173] Among them, TP is a true positive example, FN is a false negative example, FP is a false positive example, and TN is a true negative example.

[0174] S46: Calculate the area under the ROC curve (AUC) value using the trapezoidal method, and select the model with the largest AUC value as the optimal U-Net video anomaly detection network model.

[0175] The comparative analysis results of the present invention and the existing video anomaly detection method are as follows: Figure 8As shown. Compared to MemAE's 2,000 256-dimensional static memory items using 3D convolution operations, this method utilizes a lightweight dynamic memory library based on 2D convolutions, achieving superior performance by maintaining only 10 512-dimensional prototype items. Compared to methods such as Frame-Pred, which rely on complex optical flow estimation or adversarial training frameworks, this method adopts a pure autoencoder architecture and introduces a multi-scale wavelet transform feature fusion mechanism, achieving AUC improvements of 1.6 percentage points and 2.9 percentage points on the Ped2 and CUHK Avenue datasets, respectively, compared to Frame-Pred. Both DGGAN and this method achieve an AUC of 97% on the Ped2 dataset, but the pseudo-anomaly generation strategy and GAN network complexity used by the former are higher. Furthermore, the pseudo-anomaly generation strategy used by DGGAN may have problems such as insufficient sample diversity, resulting in an AUC value 2.1 percentage points lower than this method on the Avenue dataset, where anomalies are more complex. This demonstrates that this method is more robust than DGGAN in anomaly detection under complex dynamic backgrounds. Experimental results show that the dynamic memory mechanism and wavelet transform fusion strategy proposed in this paper achieves a balanced improvement in detection accuracy and computational efficiency while maintaining the scale of model parameters.

[0176] To verify the effectiveness of key modules, we conducted ablation analysis on the UCSD Ped2 and CUHK Avenue datasets. Based on the performance data of the U-Net network model (AUC: Ped2 94.3%, Avenue 84.3%), we gradually introduced the Haar Wavelet Transform Downsampling Module (HWD) and the Discrete Wavelet Transform Feature Fusion Module (DWTF), hereinafter referred to as the HWD module and the DWTF module, respectively, to explore their contributions to performance.

[0177] The feature preservation capability of the HWD module is as follows: Figure 7 As shown in the figure, after introducing the HWD module, the model's AUC on the CUHK Avenue dataset increased by 0.9% (84.3% → 85.2%), significantly higher than the 0.5% gain on the Ped2 dataset (95.3% → 95.8%). This difference stems from the frequency domain characteristics of the module: traditional downsampling methods lose approximately 15% of high-frequency details when splicing the memory module and encoder features (verified by feature energy spectrum analysis), while the HWD module encodes spatial redundant information into the channel dimension through multi-scale decomposition of Haar wavelets, retaining more than 92% of high-frequency components while reducing resolution. This feature significantly enhances the model's adaptability to complex dynamic backgrounds in CUHK Avenue (such as crowd occlusion and lighting changes).

[0178] The cross-layer feature fusion mechanism of the DWTF module can effectively improve performance. Figure 6As shown in the figure, the addition of the DWTF module improves the AUC on the CUHK Avenue dataset by 2.8% (84.3% → 87.1%), a gain twice that of the Ped2 dataset (1.4%), demonstrating that the DWTF module is more adept at handling complex datasets. This phenomenon is closely related to the characteristics of the data: in the Ped2 dataset, the visual patterns of normal samples (such as pedestrians walking regularly) and abnormal samples (such as vehicles intruding into a crowd) differ significantly. In the CUHK Avenue dataset, the spatiotemporal features of normal samples (pedestrians walking normally) and abnormal samples (such as sudden running and throwing objects) are more similar, making them more difficult to distinguish. To address this challenge, the DWTF module enhances the representation of multi-scale details through wavelet reconstruction. The deep features of the encoder are enhanced by high-frequency components and then adaptively weighted fused with the features of the same layer of the decoder, significantly improving the model's ability to recognize highly similar abnormal patterns.

[0179] Further analysis shows that there is a synergistic optimization effect between HWD and DWTF modules. Figure 9 As shown in the CUHK Avenue dataset, the combined effect of the two modules improves AUC by 3.9%, exceeding the linear sum of the gains of the individual modules (0.9% + 2.8%). This demonstrates that downsampling in the wavelet domain and feature fusion strategies complement each other: the detail-preserving properties of HWD lay the foundation for high-quality features in cross-layer fusion of DWTF, while the temporal modeling capabilities of DWTF compensate for the representational limitations of HWD in dynamic scenes. This synergistic mechanism provides a new approach to multi-scale feature learning for video anomaly detection.

[0180] Figure 10 The qualitative results of the model prediction on the UCSD Ped2 and Avenue datasets are shown, showing normal frames, abnormal frames, prediction errors, and abnormal regions covered on the frames. In order to visualize the abnormal regions, this method calculates the pixel-by-pixel abnormality score and uses heat map technology to highlight the regions where the abnormality score is higher than the average value of the pixels in the same frame. Figure 10 It can be seen that the prediction error of the abnormal behavior area is much larger than that of the normal behavior area, and it can well mark abnormal behaviors such as the appearance of cars and pedestrians running.

[0181] Figure 11A dual-coordinate comparison demonstrates the model's dynamic response to anomaly detection on two test sets: the top half shows the anomaly score curve for the Ped2 test set, while the bottom half corresponds to the monitoring results for the Avenue test set. The experimental results show that: 1) in the Ped2 scenario, when a spatial semantic anomaly (such as a vehicle intruding on a sidewalk) occurs, the anomaly score continuously climbs from the baseline to 0.91; 2) for temporal behavioral anomalies in the Avenue dataset (such as a pedestrian suddenly running), the system outputs a peak anomaly score of 0.96 after the event is triggered.

Claims

1. A video anomaly detection method based on wavelet transform and multi-scale feature fusion, characterized in that: The steps include: S1: Obtain a video dataset, split the video dataset into a video frame dataset, resize the video frame dataset and normalize the pixel values, and divide the video frame dataset into a training set and a test set; S2: Construct a U-Net video anomaly detection network model, which includes an encoder, a decoder, and a memory module. The encoder and decoder include a multi-level structure. Each level of the encoder includes a convolution block and a Haar wavelet transform downsampling module. The output end of the convolution block is connected to the input end of the Haar wavelet transform downsampling module. The Haar wavelet transform module performs multi-scale feature extraction on the input video frame. Each level of the decoder includes a multi-scale feature fusion module, a discrete wavelet transform feature fusion module, a convolution block and an upsampling module connected in sequence. The multi-scale feature fusion module is used to receive the feature map output by the encoder at the same level and the feature map output by the upsampling module, and perform feature fusion on the two to output a large-scale feature map. The discrete wavelet transform feature fusion module is used to receive the large-scale feature map and the small-scale feature map output by the encoder of the next layer, and perform feature decomposition and fusion on the two in the wavelet domain to generate a reconstructed feature map and output it to the convolution block. The memory module is deployed between the encoder and the decoder and includes a memory bank for filtering and enhancing normal feature representations and suppressing the propagation of abnormal features by storing normal feature patterns and calculating the similarity between input features and stored items; S3: Constructing a comprehensive loss function L total , train the U-Net video anomaly detection network model through the training set; S4: Use the test set to calculate the anomaly score of the predicted frame output by the U-Net video anomaly detection network model, and determine the optimal anomaly score threshold τ by constructing the ROC curve * , get the optimal U-Net video anomaly detection network model; S5: Deploy the optimal U-Net video anomaly detection network model to perform anomaly detection on real-time videos. The optimal U-Net video anomaly detection network model determines in real time whether there are any anomalies in the video.

2. The video anomaly detection method based on wavelet transform and multi-scale feature fusion according to claim 1 is characterized in that: In step S1, the method of resizing and normalizing the pixel values ​​of the video frame dataset and dividing the video frame dataset into a training set and a test set is as follows: The sizes of all data frames in the video frame dataset are adjusted to a uniform size, and the pixel values ​​are normalized. The video frame dataset is divided into a training set containing only normal phenomena and a test set containing abnormal phenomena. The test set uses binary labels to mark whether each frame has abnormal phenomena.

3. The video anomaly detection method based on wavelet transform and multi-scale feature fusion according to claim 1 is characterized in that: In step S2, the Haar wavelet transform downsampling module includes a lossless feature encoding block and a spatial domain feature representation block connected in sequence; The frequency domain lossless feature coding block is used to perform multi-scale decomposition on the input feature map X1 to generate an intermediate feature map X2 containing three high-frequency detail components and one low-frequency approximation component; wherein, R represents the real tensor element, C represents the number of channels, H represents the vertical pixel value of the feature map, and W represents the horizontal pixel value of the feature map; The spatial domain feature representation block includes a convolution layer, a normalization layer, and an activation layer arranged in sequence, which is used to perform channel compression and nonlinear transformation on the intermediate feature map X2 to obtain an output feature map X3, wherein, 4. The video anomaly detection method based on wavelet transform and multi-scale feature fusion according to claim 3 is characterized in that: The frequency domain lossless feature coding block is used to perform multi-scale decomposition on the input feature map X1 to generate an intermediate feature map X2 containing high-frequency detail components and low-frequency approximation components. Specifically, the method is as follows: A low-pass filter H0 is set in the horizontal direction, and a high-pass filter H1 is set in the vertical direction. The low-pass filter H0 and the high-pass filter H1 are used to perform convolution operations on the input feature map X1 in the horizontal and vertical directions respectively. Each convolution output is downsampled by 2 times to generate four orthogonal sub-band components, which are low-frequency approximate components and high-frequency detail components representing the horizontal direction Characterizes the high-frequency detail components in the vertical direction and the high-frequency detail components representing the diagonal direction Concatenate the channel dimensions to generate intermediate feature maps The mathematical expressions for generating the low-frequency approximation component LL and the high-frequency detail component LH representing the horizontal direction, the high-frequency detail component HL representing the vertical direction, and the high-frequency detail component HH representing the diagonal direction are as follows: in, Represents a tensor outer product operation, and * represents a two-dimensional convolution operation.

5. The video anomaly detection method based on wavelet transform and multi-scale feature fusion according to claim 1 is characterized in that: In step S2, the discrete wavelet transform feature fusion module includes a multi-scale decomposition module, a low-frequency feature enhancement module, a high-frequency feature fusion module and a multi-scale reconstruction module. The input end of the multi-scale decomposition module receives the large-scale feature map output by the multi-scale feature fusion module The multi-scale decomposition module is used to perform discrete wavelet transform decomposition on the large-scale feature map X4, decomposing the input large-scale feature map X4 into a low-frequency approximate component and three high-frequency detail components representing the diagonal, vertical, and horizontal directions; the formula is expressed as: {HH,HL,LH,LL}=DWT(X1) in, is the low-frequency approximation component, HH is the high-frequency detail component representing the diagonal direction, HL is the high-frequency detail component representing the vertical direction, LH is the high-frequency detail component representing the horizontal direction, and DWT is discrete wavelet transform; The input end of the low-frequency feature enhancement module is connected to the small-scale feature map X5 output by the encoder and the low-frequency approximate component LL output by the multi-scale decomposition module. The low-frequency feature enhancement module is used to splice the small-scale feature map X5 output by the encoder with the low-frequency approximate component LL, and then enhance the features through convolution and residual connection to obtain a low-frequency enhanced feature map E2; The high-frequency feature fusion module receives the high-frequency detail component HH representing the diagonal direction, the high-frequency detail component HL in the vertical direction, and the high-frequency detail component LH in the horizontal direction output by the low-frequency feature enhancement module. The high-frequency feature fusion module is used to perform channel compression and nonlinear transformation on the high-frequency detail component to generate a high-frequency discriminant feature map F3; The multi-scale reconstruction module receives the low-frequency enhancement feature map E2 output by the low-frequency feature enhancement module and the high-frequency discriminant feature map F3 output by the high-frequency feature fusion module. The multi-scale reconstruction module is used to splice the low-frequency enhancement feature map E2 and the high-frequency discriminant feature map F3 to generate a reconstructed feature map Y1.

6. The video anomaly detection method based on wavelet transform and multi-scale feature fusion according to claim 5 is characterized in that: The low-frequency feature enhancement includes a first splicing layer, a first convolutional layer and a first residual layer connected in sequence. The first splicing layer is used to splice the small-scale feature map X5 and the low-frequency approximate component LL along the channel dimension to generate a low-frequency splicing feature map E0 E0=Concat(X5,LL); The first convolutional layer is used to extract features from the low-frequency splicing feature map E0 through 3×3 convolution to generate a convolution feature map E1. The formula is expressed as: E1 = Conv3 × 3 (E0); The first residual layer is used to perform feature enhancement on the convolution feature map E1 to generate a low-frequency enhanced feature map E2, which is expressed as: E2=E1+Conv3×3(E1).

7. The video anomaly detection method based on wavelet transform and multi-scale feature fusion according to claim 5 is characterized in that: The high-frequency feature fusion module includes the second concatenation layer, the second convolution layer, the second residual layer and the third convolution layer connected in sequence. The second splicing layer is used to splice the high-frequency detail components {HH, HL, LH} along the channel dimension to generate a high-frequency splicing feature map F0, which is expressed as: F0=Concat(HH, HL, LH); The second convolutional layer is used to perform channel compression on the high-frequency splicing feature map F0 through 1×1 convolution to generate a high-frequency compressed feature map F1. The formula is expressed as: F1 = Conv1 × 1 (F0); The second residual layer is used to perform nonlinear transformation on the high-frequency compression feature map F1 by introducing a residual connection to generate a high-frequency residual feature map F2. The formula is expressed as: F2 = F1 + Conv3 × 3 (F1); The third convolutional layer is used to extract discriminative features from the high-frequency residual feature map F2 through 1×1 convolution to generate a high-frequency discriminative feature map F3. The formula is expressed as: F3=F2+Conv1×1(F2).

8. The video anomaly detection method based on wavelet transform and multi-scale feature fusion according to claim 5 is characterized in that: The multi-scale reconstruction module includes a feature integration layer and a scale recovery layer connected in sequence. The feature integration layer is used to splice the low-frequency enhancement feature map E2 and the high-frequency discrimination feature map F3 along the channel dimension to generate an integrated feature map Y0, which is expressed as follows: Y0=Concat(E2,F3); The scale recovery layer is used to perform an inverse discrete wavelet transform on the integrated feature map Y0 to restore the same spatial resolution as the large-scale feature map X4 to obtain a reconstructed feature map Y1. The formula is: Among them, IDWT is inverse discrete wavelet transform.

9. The video anomaly detection method based on wavelet transform and multi-scale feature fusion according to claim 1 is characterized in that: In step S3, the comprehensive loss function L is constructed total The following steps are involved: S31: Build strength loss L int Measuring the pixel-level difference between the predicted frame and the real frame, the formula is expressed as: Among them, ||·||2 represents the Euclidean distance, is the predicted frame at time t, I t is the real frame at time t; S32: Construct feature compactness loss L compact Force query items Minimize the feature distance to the nearest neighbor storage item and enhance the compact clustering of normal patterns. The formula is expressed as: The index q is determined by: Where t is the time step index, T is the time step set, t∈T; k is the query item index, K is the number of query items in a single time step, k∈K; is the kth query item at time t, p q For memory The storage item with the closest Euclidean distance, the index q of the nearest neighbor storage item is calculated by weight Determine, M is the set of memory storage items; S33: Constructing feature separation loss L separate Constrain similar query items to be mapped to the same storage item, suppressing the generation of redundant storage items. The formula is expressed as: The index n is determined by: Where t is the time step index, T is the time step set, t∈T; k is the query item index, K is the number of query items in a single time step, k∈K; is the kth query item at time t, p n For memory The Euclidean distance is the next closest storage item feature, and m≠q, α is the interval threshold, is the set of items in the memory except q, [x] + is the RELU function, taking the maximum number between x and 0; S34: Loss of strength L int , feature compactness loss L compact and feature separation loss L separate Combined to form a comprehensive loss function L total , the formula is: Among them, L total is the comprehensive loss, L int is the strength loss, L compact is the feature compactness loss, L separate is the feature separation loss; int ,λ c ,λ s Represents the weight of each loss.

10. The video anomaly detection method based on wavelet transform and multi-scale feature fusion according to claim 1, characterized in that: In step S4, the method of calculating the anomaly score of the predicted frame output by the U-Net video anomaly detection network model using the test set and determining the optimal anomaly score threshold by constructing a ROC curve includes the following steps: S41: Calculate the Euclidean distance between the query item and the nearest stored item. The formula is: in, is the kth query item at time t, p q For memory The stored item with the closest Euclidean distance, K is the total number of query items in a single time step, S42: Calculate the peak signal-to-noise ratio (PSNR) between the real frame and the predicted frame. The peak signal-to-noise ratio (PSNR) formula is expressed as: in, is the predicted frame at time t, I t is the real frame at time t; is the maximum pixel value of the predicted frame, and N is the total number of pixels per frame; S43: D(q t ,p) and The value of is normalized to the range of [0,1], and the formula is expressed as: Among them, g(·) represents normalization, mapping the value to the interval [0,1]; S44: Final use formula g(D(q t ,p)) and g(PSNR(I t ,I t The sum of the two metrics is the final anomaly score S t , the formula is: S t =λ(1-g(PSNR(I t ,I t )))+(1-λ)g(D(q t ,p)), Among them, λ is a hyperparameter that balances the peak signal-to-noise ratio PSNR and the L2 distance weight, S t is the final anomaly score at time t; S45: Sort the final anomaly scores τ∈[0,1] of all samples in the test set, traverse all anomaly score thresholds τ∈[0,1], calculate the true positive rate TPR and false positive rate FPR, and draw the ROC curve with FPR on the horizontal axis and TPR on the vertical axis. The formula is: Among them, TP is a true positive example, FN is a false negative example, FP is a false positive example, and TN is a true negative example. S46: Calculate the area under the ROC curve (AUC) value using the trapezoidal method, and select the model with the largest AUC value as the optimal U-Net video anomaly detection network model.

Images