Video abnormal behavior detection method, system and device fusing wavelet transform and depth separable convolution, and medium

Abstract

The invention relates to a video abnormal behavior detection method, system and device fusing wavelet transform and depth separable convolution, and a medium, and the method comprises the steps: introducing 3D depth separable convolution, a 3D attention module and autonomously constructed 3D wavelet convolution to improve an existing auto-encoder structure; the 3D wavelet depth separable attention auto-encoder with light weight and high detection precision is obtained; and the generated pseudo-abnormal image and the normal behavior image with the adjusted size are jointly input into the 3D wavelet depth separable attention auto-encoder in a continuous frame form for training and evaluation, so that the problem of over-fitting of the 3D wavelet depth separable attention auto-encoder caused by scarcity of real abnormal samples in anomaly detection is solved. The 3D wavelet depth separable attention auto-encoder can better distinguish a normal behavior image and an abnormal behavior image; the method has the advantages of low requirement on model parameter quantity and high detection accuracy; the system, the equipment and the medium are used for realizing the method.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, system, device and medium for detecting abnormal behavior in video by integrating wavelet transform and depthwise separable convolution. Background Art

[0002] With the development of the information age, the application scope of surveillance video continues to expand, and public safety issues are receiving increasing attention. Detecting abnormal crowd behavior is a challenging task in video surveillance. The goal is to identify unusual phenomena in videos, such as fighting and vandalism. Researchers generally define abnormal behavior as behavior that is different from existing patterns, unusual, unexpected, and unpredictable. Capturing video data through surveillance cameras and detecting abnormal behavior in real time can effectively prevent dangerous incidents.

[0003] Due to the unpredictability of abnormal behaviors and the large amount of video data, supervised video anomaly detection methods that require labeling all abnormal behaviors are not applicable. Therefore, research has turned to unsupervised video anomaly detection methods that do not rely on large amounts of labeled data. Existing unsupervised video anomaly detection methods are divided into prediction-based methods and reconstruction-based methods.

[0004] Prediction-based methods primarily learn the temporal characteristics of normal input data and compare the predicted results with the actual input video to determine whether the current input video is abnormal. While they can effectively capture dynamic changes at the video frame level, they rely on historical data and are prone to cumulative errors in historical data over short time windows, making it difficult to accurately detect persistent abnormal behavior.

[0005] Reconstruction-based methods play a certain role in improving model performance by adding memory modules or introducing pseudo-anomalies. However, these methods do not optimize the structure of the autoencoder itself, resulting in redundant model structures and a large number of parameters, making these methods difficult to deploy on embedded devices with limited computing resources.

[0006] Wang (Wang Chenxu, Yao Yanxin, Yao Han. Video anomaly detection methodbased on future frame prediction and attention mechanism[C] / / Proc of IEEE11th Annual Computing and Communicati on Workshop and Conference(CCWC).Piscataway:IEEE,2021:405-407.) et al. proposed a video anomaly detection method based on future frame prediction and attention mechanism. By combining generative adversarial networks and attention mechanisms, the generator's generation quality of future frames was improved and the network's prediction of future frames was enhanced; however, the scheme relied too much on historical data, resulting in the inability to accurately detect persistent abnormal behavior.

[0007] Dang (Dang Yuanjie, Chen Jiangyun, Chen Peng, et al. Generate anomalies from normal: a partial pseudo-anomaly augmented appr oach for video anomaly detection[J]. The Vision Computer, 2024: 1-10.) et al. proposed a partial pseudo-anomaly enhancement method for video anomaly detection. By generating pseudo-anomaly images, the autoencoder's reconstruction effect on normal images was improved, enabling the network to better distinguish between abnormal and normal behaviors; however, the autoencoder has a large number of parameters and is difficult to deploy on edge devices with low computing resources. Summary of the Invention

[0008] In order to overcome the defects of the above-mentioned prior art, the purpose of the present invention is to provide a video abnormal behavior detection method, system, device and medium that integrates wavelet transform and depthwise separable convolution. The method improves the existing autoencoder structure by introducing 3D depthwise separable convolution, 3D attention module and independently constructed 3D wavelet convolution, thereby obtaining a 3D wavelet depthwise separable attention autoencoder that is both lightweight and has high detection accuracy; the generated pseudo-abnormal image and the resized normal behavior image are then input into the 3D wavelet depthwise separable attention autoencoder in the form of continuous frames for training and evaluation, thereby solving the problem of overfitting of the 3D wavelet depthwise separable attention autoencoder caused by the scarcity of real abnormal samples in anomaly detection, so that the 3D wavelet depthwise separable attention autoencoder can better distinguish between normal behavior images and abnormal behavior images; the present invention has the advantages of small requirements on the number of model parameters and high detection accuracy.

[0009] In order to achieve the above object, the technical solution adopted by the present invention is:

[0010] A method for detecting abnormal behavior in video by integrating wavelet transform and depthwise separable convolution, comprising:

[0011] Label the images in the crowd abnormal behavior dataset as normal behavior and abnormal behavior, and adjust the image size;

[0012] A portion of resized normal behavior images is used to generate pseudo-abnormal images. The pseudo-abnormal images and another portion of resized normal behavior images are input into a 3D wavelet deep separable attention autoencoder in the form of consecutive frames for training and evaluation, resulting in a 3D wavelet deep separable attention autoencoder that meets the expected detection accuracy.

[0013] The image to be detected is input into a 3D wavelet deep separable attention autoencoder that meets the expected detection accuracy for detection, and the abnormal behavior detection result is obtained.

[0014] Furthermore, the 3D wavelet deep separable attention autoencoder includes an improved encoder and an improved decoder;

[0015] The improved encoder has four layers; the first layer structure is 3D convolution, batch normalization layer, and Leaky-Relu activation function in sequence; the second layer structure is 3D convolution, batch normalization layer, and Leaky-Relu activation function in sequence; the third layer structure is 3D depth-separable convolution, 3D attention module, batch normalization layer, and Leaky-Relu activation function in sequence; the fourth layer structure is 3D wavelet convolution, 3D attention module, batch normalization layer, and Leaky-Relu activation function in sequence; the first and second layer structures of the improved encoder are both used to extract low-level features of the data and perform dimensionality reduction processing; the third layer structure of the improved encoder is used to perform deeper feature extraction on the data; the fourth layer of the improved encoder is used to map high-dimensional data to a latent space;

[0016] The improved decoder has four layers; the first layer structure is 3D depth-separable convolution, 3D attention module, batch normalization layer, and Leaky-Relu activation function; the second layer structure is 3D depth-separable convolution, 3D attention module, batch normalization layer, and Leaky-Relu activation function; the third layer structure is 3D deconvolution, batch normalization layer, and Leaky-Relu activation function; the fourth layer structure is 3D deconvolution and Than activation function; the first layer structure and the second layer structure of the improved decoder are both used to gradually map the latent space representation back to the high-dimensional space of the input data and restore the structure of the data; the third layer structure of the improved decoder is used to map the data back to the high-dimensional space and refine the reconstruction process; the fourth layer structure of the improved decoder is used to restore the data to the original high-dimensional space and generate data as close to the original input as possible.

[0017] Furthermore, the construction of the 3D wavelet convolution specifically includes:

[0018] Use wavelet transform to decompose the input feature map V in three dimensions D, W, and H and perform convolution operations to obtain low-frequency information and high-frequency information The high-frequency information include Intermediate feature maps of seven different high-frequency bands;

[0019] For low-frequency information and high-frequency information Perform inverse wavelet transform to obtain the output feature map That is 3D wavelet convolution; the relevant calculation formula is as follows:

[0020]

[0021] Among them, W is the convolution kernel weight tensor of size K×K×K; 3DWT is wavelet transform; 3DIWT is inverse wavelet transform.

[0022] Furthermore, the generating of the pseudo abnormal image using a portion of the resized normal behavior image specifically includes:

[0023] A portion of the resized normal behavior images are first used in the form of continuous frames to detect normal areas through YoLoV3, and pseudo-abnormal data are added to the detected normal areas to generate pseudo-abnormal images.

[0024] Furthermore, the pseudo-abnormal image and another portion of resized normal behavior images are inputted into a 3D wavelet deep separable attention autoencoder in the form of continuous frames for training and evaluation, and a 3D wavelet deep separable attention autoencoder that meets the expected detection accuracy is obtained. Specifically, the method includes:

[0025] The pseudo-abnormal image and another portion of resized normal behavior images are input into the 3D wavelet deep separable attention autoencoder in the form of continuous frames. The operating parameters of the 3D wavelet deep separable attention autoencoder are set and the 3D wavelet deep separable attention autoencoder is trained on the experimental operation platform.

[0026] The detection accuracy of the currently trained 3D wavelet deep separable attention autoencoder is evaluated using the area under the ROC curve and the coordinate axis (AreaUnder Curve), parameters, and floating point operations.

[0027] If the detection accuracy reaches the expected level, stop the training and evaluation process of the 3D wavelet deep separable attention autoencoder;

[0028] If the detection accuracy does not meet the expectations, adjust the operating parameters of the 3D wavelet deep separable attention autoencoder, and repeat the training and evaluation process of the 3D wavelet deep separable attention autoencoder until the detection accuracy meets the expectations, and obtain a 3D wavelet deep separable attention autoencoder that meets the expected detection accuracy.

[0029] Furthermore, the operating parameters of the 3D wavelet deep separable attention autoencoder include initial learning rate, learning momentum, and weight decay rate, and the adaptive matrix estimation method (Adaptive Moment Estimation, Adam) is used to update the network parameters.

[0030] Furthermore, the expected detection accuracy of the 3D wavelet deep separable attention autoencoder is set to 94%-96%.

[0031] A video abnormal behavior detection system integrating wavelet transform and depthwise separable convolution, comprising:

[0032] Image processing module: labels the images in the crowd abnormal behavior dataset as normal behavior and abnormal behavior, and adjusts the image size;

[0033] Model training module: Use a portion of resized normal behavior images to generate pseudo-abnormal images. The pseudo-abnormal images and another portion of resized normal behavior images are input into a 3D wavelet deep separable attention autoencoder in the form of consecutive frames for training and evaluation, resulting in a 3D wavelet deep separable attention autoencoder that meets the expected detection accuracy.

[0034] Abnormal behavior detection module: The image to be detected is input into a 3D wavelet deep separable attention autoencoder that meets the expected detection accuracy for detection to obtain the abnormal behavior detection result.

[0035] A method and device for detecting abnormal behavior in video by integrating wavelet transform and depthwise separable convolution, comprising:

[0036] Memory: used to store a computer program to implement the above-mentioned method for detecting abnormal behavior in video by integrating wavelet transform and depthwise separable convolution;

[0037] Processor: used to implement the above-mentioned video abnormal behavior detection method integrating wavelet transform and depthwise separable convolution when executing the computer program.

[0038] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for detecting abnormal video behavior by fusing wavelet transform and depthwise separable convolution.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. This paper reconstructs the decoder and encoder in the existing autoencoder. First, the standard convolutions in the third and fourth layers of the encoder and the standard convolutions in the first and second layers of the decoder are replaced by 3D depthwise separable convolutions. The depthwise convolution and pointwise convolution of the 3D depthwise separable convolutions reduce the number of parameters and computational complexity of the 3D wavelet depthwise separable attention autoencoder.

[0041] Secondly, a 3D attention module is added after the 3D depthwise separable convolution in the third and fourth layers of the encoder and the first and second layers of the decoder. This enables the 3D wavelet depthwise separable attention autoencoder to focus more on important spatial regions and channels, avoiding the performance degradation caused by the reduction of parameters.

[0042] Finally, a 3D wavelet convolution is designed to replace the fourth layer 3D depthwise separable convolution of the encoder. While increasing the receptive field of the convolution operation, it improves the fine-grained feature extraction capability of the 3D wavelet depthwise separable attention autoencoder and further improves the detection accuracy of the 3D wavelet depthwise separable attention autoencoder.

[0043] 2. The present invention uses a portion of resized normal behavior images to generate pseudo-abnormal images, and inputs the pseudo-abnormal images and another portion of resized normal behavior images in the form of continuous frames into a 3D wavelet deep separable attention autoencoder for training and evaluation. This solves the problem of overfitting of the 3D wavelet deep separable attention autoencoder caused by the scarcity of real abnormal samples in anomaly detection, enabling the 3D wavelet deep separable attention autoencoder to better distinguish between normal behavior images and abnormal behavior images, with higher detection accuracy.

[0044] In summary, the present invention improves the existing autoencoder structure by introducing 3D depth-separable convolution, 3D attention module and independently constructed 3D wavelet convolution, and obtains a 3D wavelet depth-separable attention autoencoder with both lightweight and high detection accuracy; then the generated pseudo-abnormal image and the normal behavior image with adjusted image size are input into the 3D wavelet depth-separable attention autoencoder in the form of continuous frames for training and evaluation, which solves the problem of overfitting of the 3D wavelet depth-separable attention autoencoder caused by the scarcity of real abnormal samples in anomaly detection, so that the 3D wavelet depth-separable attention autoencoder can better distinguish between normal behavior images and abnormal behavior images; the present invention has the advantages of small requirements on the number of model parameters and high detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a flow chart of the video abnormal behavior detection method based on the fusion of wavelet transform and depthwise separable convolution.

[0046] Figure 2 This is the overall framework of the video anomaly detection model.

[0047] Figure 3 Schematic diagram of the 3D wavelet deep separable attention autoencoder structure.

[0048] Figure 4 Schematic diagram of the structure of 3D depth-wise separable convolution.

[0049] Figure 5 Schematic diagram of the structure of the 3D attention module.

[0050] Figure 6 Schematic diagram of the structure of 3D wavelet convolution. DETAILED DESCRIPTION

[0051] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0052] See also Figure 1 , a video abnormal behavior detection method integrating wavelet transform and depthwise separable convolution, comprising:

[0053] The images in the crowd abnormal behavior dataset are labeled as normal and abnormal behaviors, and the image size is adjusted. The crowd abnormal behavior dataset in this embodiment is derived from the public UCSD Ped2 dataset (UCSD Ped2Dataset) and Avenue dataset (Avenue Dataset). At the same time, the image size is adjusted to 256×256 to meet the requirements of subsequent network input.

[0054] A portion of resized normal behavior images is used to generate pseudo-abnormal images. The pseudo-abnormal images and another portion of resized normal behavior images are input into a 3D wavelet deep separable attention autoencoder in the form of consecutive frames for training and evaluation, resulting in a 3D wavelet deep separable attention autoencoder that meets the expected detection accuracy.

[0055] Furthermore, the generating of the pseudo abnormal image using a portion of the resized normal behavior image specifically includes:

[0056] A portion of resized normal behavior images is first run through YoLoV3 in the form of continuous frames to detect normal regions. Pseudo-abnormal data is then added to the detected normal regions to generate pseudo-abnormal images. The pseudo-abnormal data is derived from the CIFAR-100 dataset. By using a portion of resized normal behavior images to generate pseudo-abnormal images, the problem of overfitting of the 3D wavelet deep separable attention autoencoder caused by the scarcity of true abnormal samples in anomaly detection is solved.

[0057] like Figure 2 and Figure 3 As shown, this embodiment uses GAFN (Generate Anomalies from Normal) as a baseline and designs a 3D Wavelet Depthwise Separable Convolution Attention Autoencoder (3D Wavelet DSC-AAE) as a video abnormal behavior detection network.

[0058] The construction process of the 3D wavelet deep separable attention autoencoder is as follows:

[0059] (a) First, the standard convolutions in the third and fourth layers of the existing encoder and the first and second layers of the decoder are replaced with 3D depthwise separable convolutions;

[0060] like Figure 4 As shown in the figure, the 3D depth-wise separable convolution is composed of 3D depth-wise convolution and 3D point-wise convolution in sequence; the 3D depth-wise separable convolution can perform depth-wise convolution on each channel separately and aggregate information between channels through point-by-point convolution to reduce the number of parameters and accelerate the inference of the 3D wavelet depth-wise separable attention autoencoder; the formula of the 3D depth-wise separable convolution is defined as follows:

[0061] 3D_DSC(V)=3D_PointC(3D_DepthC(V)) (1)

[0062]

[0063] in, is the input feature map; C is the number of channels; D, H, and W are the dimensions in the depth, height, and width directions respectively; 3D_DSC is 3D depth-separable convolution; 3D_PointC is 3D point-by-point convolution; 3D_DepthC is 3D depth-wise convolution; W1 represents the convolution kernel weight of 3D depth-separable convolution, k, l, m are the convolution kernel sizes; i, j, u are position indices; ⊙ is element-by-element multiplication; W2 is the convolution kernel weight of 3D point-by-point convolution; n is the convolution kernel size; is the eigenvalue of the nth channel at position (i, j, u);

[0064] (b) Based on the above step (a), a 3D attention module (3DA) is added after the 3D depthwise separable convolution in the third and fourth layers of the encoder and the first and second layers of the decoder. By using the 3D attention module after the 3D depthwise separable convolution, the 3D wavelet depthwise separable attention autoencoder can focus more on important spatial regions and channels, avoiding the problem of model performance degradation caused by the reduction of parameters.

[0065] like Figure 5 As shown in Figure 2, the 3D attention module consists of a cascade of a channel attention module and a spatial depth attention module. The relevant construction principles are as follows:

[0066] First, input feature map Different features are extracted through maximum pooling and average pooling; then the channel attention weight is obtained through a multilayer perceptron (MLP) and a Sigmoid activation function. Then Multiply with V to get the intermediate feature map Secondly, The maximum pooling and average pooling are used to further enrich the channel information, spatial features are extracted through spatial convolution, and depth convolution is used to aggregate depth information; finally, the spatial and depth information are fused using comprehensive convolution, and the spatial depth attention weight is obtained through the Sigmoid activation function. and Multiply to get the final output feature map The implementation of the 3D attention module is shown in Equations (4)-(7):

[0067]

[0068]

[0069] Among them, σ is the Sigmoid activation function; MaxPool is the maximum pooling operation; AvgPool is the average pooling operation; f 7×7×7 is the comprehensive convolution; f 1×7×7 is the spatial convolution; f 7×7×1 is the depth convolution;

[0070] (c) Based on the above step (b), a 3D wavelet convolution (3DWTC) is designed to replace the fourth layer of the encoder 3D depthwise separable convolution; while increasing the receptive field of the convolution operation, the 3D wavelet convolution improves the fine-grained feature extraction capability of the 3D wavelet depthwise separable attention autoencoder, further improving the detection accuracy of the 3D wavelet depthwise separable attention autoencoder.

[0071] like Figure 6 As described above, based on the existing 2D wavelet convolution, the present invention constructs a 3D wavelet convolution; the construction of the 3D wavelet convolution specifically includes:

[0072] Use wavelet transform to decompose the input feature map V in three dimensions D, W, and H and perform convolution operations to obtain low-frequency information and high-frequency information The high-frequency information include Intermediate feature maps of seven different high-frequency bands;

[0073] For low frequency information and high-frequency information Perform inverse wavelet transform to obtain the output feature map That is 3D wavelet convolution; the relevant calculation formula is as follows:

[0074]

[0075] Among them, W is the convolution kernel weight tensor of size K×K×K; 3DWT is wavelet transform; 3DIWT is inverse wavelet transform.

[0076] Through the above operations, a 3D wavelet deep separable attention autoencoder is constructed. Compared with the existing autoencoders, it maintains the lightweightness of the 3D wavelet deep separable attention autoencoder while maintaining the detection accuracy of the 3D wavelet deep separable attention autoencoder, so that the detection accuracy and computational efficiency of the 3D wavelet deep separable attention autoencoder are better balanced.

[0077] The 3D wavelet deep separable attention autoencoder includes an improved encoder and an improved decoder;

[0078] like Figure 3 E1 to E4, the improved encoder has four layers, the first layer structure is 3D convolution, batch normalization layer, Leaky-Relu activation function; the second layer structure is 3D convolution, batch normalization layer, Leaky-Relu activation function; the third layer structure is 3D depth separable convolution, 3D attention module, batch normalization layer, Leaky-Relu activation function; the fourth layer structure is 3D wavelet convolution, 3D attention module, batch normalization layer, Leaky-Relu activation function; the first layer structure and the second layer structure of the improved encoder are both used to extract low-level features of the data and perform dimensionality reduction processing; the third layer structure of the improved encoder is used to perform deeper feature extraction on the data; the fourth layer of the improved encoder is used to map high-dimensional data to a latent space;

[0079] like Figure 3 D1 to D4, the improved decoder has four layers, the first layer structure is 3D depth separable convolution, 3D attention module, batch normalization layer, Leaky-Relu activation function; the second layer structure is 3D depth separable convolution, 3D attention module, batch normalization layer, Leaky-Relu activation function; the third layer structure is 3D deconvolution, batch normalization layer, Leaky-Relu activation function; the fourth layer structure is 3D deconvolution, Than activation function; the first layer structure and the second layer structure of the improved decoder are used to gradually map the latent space representation back to the high-dimensional space of the input data and restore the structure of the data; the third layer structure of the improved decoder is used to map the data back to the high-dimensional space and refine the reconstruction process; the fourth layer structure of the improved decoder is used to restore the data to the original high-dimensional space and generate data as close to the original input as possible.

[0080] The pseudo-abnormal image and another part of the normal behavior image with adjusted size are inputted into the 3D wavelet deep separable attention autoencoder in the form of continuous frames for training and evaluation, and the 3D wavelet deep separable attention autoencoder with expected detection accuracy is obtained. Specifically, the following steps are performed:

[0081] The pseudo-abnormal image and another portion of resized normal behavior images are input into the 3D wavelet deep separable attention autoencoder in the form of continuous frames, and the operating parameters of the 3D wavelet deep separable attention autoencoder are set. The 3D wavelet deep separable attention autoencoder is trained on the experimental operation platform. In this embodiment, 16 continuous frames of normal behavior images are used as input;

[0082] The operating parameters of the 3D wavelet deep separable attention autoencoder include initial learning rate, learning momentum, and weight decay rate, and the adaptive matrix estimation method (Adaptive Moment Estimation, Adam) is used to update the network parameters.

[0083] The method of updating network parameters by using the adaptive matrix estimation method specifically includes:

[0084] Calculate the current gradient; update the first-order matrix by performing exponentially weighted moving average on the current gradient; update the second-order matrix by performing exponentially weighted squared gradient update; correct the deviation between the first-order matrix and the second-order matrix, and update the network parameters in combination with the learning momentum and initial learning rate;

[0085] The detection accuracy of the currently trained 3D wavelet deep separable attention autoencoder is evaluated using the area under the ROC curve (AUC), parameters, and floating point operations, which are commonly used evaluation criteria in the field of video anomaly detection. The calculation formula and principle of the relevant detection accuracy evaluation are as follows:

[0086] In the area under the ROC curve (Area Under Curve), the horizontal axis of the ROC curve is the probability of misclassifying negative samples (False Positive Rate, FPR), and the vertical axis is the probability of correctly classifying positive samples (True Positive Rate, TPR); the calculation formulas for FPR and TPR are as follows:

[0087]

[0088] Among them, FP is the number of false positive samples; TN is the number of true negative samples; TP is the number of true positive samples; FN is the number of false negative samples;

[0089] The calculation formula of the parameters is as follows:

[0090] Params=C0×(K d ×K w ×K h ×C i +1) (12)

[0091] Among them, C0 is the number of output channels; K d , K w , K h are the depth, width, and height of the convolution kernel respectively; C i is the number of input channels;

[0092] The calculation formula of the floating point operations is as follows:

[0093] FLOPs = 2 × K d ×K w ×K h ×C i ×C O ×d×w×h (13)

[0094] Where: d, w, h are the depth, width, and height of the feature map respectively;

[0095] The experimental operation platform of this embodiment is shown in Table 1:

[0096] Table 1 Experimental operation platform

[0097] parameter Configuration operating system Ubuntu 20.04 system CPU Intel Xeon Gold 6330 GPU NVIDIA GeForce RTX 3090 Graphics card memory 24GB programming language Python Deep Learning Framework Pytorch 1.12.0 deep learning framework

[0098] If the detection accuracy reaches the expected level, stop the training and evaluation process of the 3D wavelet deep separable attention autoencoder;

[0099] If the detection accuracy does not meet the expectations, adjust the operating parameters of the 3D wavelet deep separable attention autoencoder, and repeat the training and evaluation process of the 3D wavelet deep separable attention autoencoder until the detection accuracy meets the expectations, and obtain a 3D wavelet deep separable attention autoencoder that meets the expected detection accuracy.

[0100] Furthermore, the expected detection accuracy of the 3D wavelet deep separable attention autoencoder is set to 94%-96%.

[0101] The image to be detected is input into a 3D wavelet deep separable attention autoencoder that meets the expected detection accuracy for detection, and the abnormal behavior detection result is obtained.

[0102] A video abnormal behavior detection system integrating wavelet transform and depthwise separable convolution, comprising:

[0103] Image processing module: labels the images in the crowd abnormal behavior dataset as normal behavior and abnormal behavior, and adjusts the image size;

[0104] Model training module: Use a portion of resized normal behavior images to generate pseudo-abnormal images. The pseudo-abnormal images and another portion of resized normal behavior images are input into a 3D wavelet deep separable attention autoencoder in the form of consecutive frames for training and evaluation, resulting in a 3D wavelet deep separable attention autoencoder that meets the expected detection accuracy.

[0105] Abnormal behavior detection module: The image to be detected is input into a 3D wavelet deep separable attention autoencoder that meets the expected detection accuracy for detection to obtain the abnormal behavior detection result.

[0106] A method and device for detecting abnormal behavior in video by integrating wavelet transform and depthwise separable convolution, comprising:

[0107] Memory: used to store a computer program to implement the above-mentioned method for detecting abnormal behavior in video by integrating wavelet transform and depthwise separable convolution;

[0108] Processor: used to implement the above-mentioned video abnormal behavior detection method integrating wavelet transform and depthwise separable convolution when executing the computer program.

[0109] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for detecting abnormal video behavior by fusing wavelet transform and depthwise separable convolution.

[0110] The application effect of the present invention is described in detail below in conjunction with simulation experiments.

[0111] The 3D wavelet deep separable attention autoencoder of the present invention and the nine mainstream models (AE-CONV3D (2017), Mem-AE (2019), Frame-pred (2018), MNAD-Recon (2020), Patch-based (2021), Backward-frame (2022), VAD-ViT (2023), RTAD-CGAN (2024), GAFN (2024)) in Table 2 below are all carried out under the experimental operation platform built in Table 1 above;

[0112] During the test, the public UCSD Ped2 dataset and Avenue dataset were respectively input into the 3D wavelet deep separable attention autoencoder of the present invention and the nine mainstream models in Table 2 for testing. The experimental parameters were set as follows: total number of iterations 400, initial learning rate 0.0001, first-order exponential decay rate 0.9, and second-order exponential decay rate 0.999.

[0113] Table 2 Experimental comparison of the 3D wavelet deep separable attention autoencoder of the present invention and 9 mainstream models

[0114]

[0115] The parameter size of the 3D wavelet deep separable attention autoencoder of the present invention is only 1.0M, which is significantly lower than other mainstream models. Compared with GAFN (2024), the parameter size is reduced by 83.3%, which greatly reduces the model's computational and storage overhead and greatly improves the model's computational efficiency.

[0116] Compared with RTAD-CGAN (2024) and Mem-AE (2019) with similar AUC values of the 3D wavelet deep separable attention autoencoder of the present invention, the number of parameters of the 3D wavelet deep separable attention autoencoder of the present invention is reduced by 85.7%, which shows that the introduction of 3D wavelet convolution effectively compensates for the accuracy loss caused by lightweighting and achieves a balance between accuracy and efficiency.

[0117] The AUC value of Frame-pred (2018) on the Avenue dataset reached 85.1%, but the number of parameters of Frame-pred (2018) is 7.7 times that of the 3D wavelet deep separable attention autoencoder of the present invention. In comparison, the AUC value of the 3D wavelet deep separable attention autoencoder of the present invention on the Avenue dataset only dropped by 1.6%, providing a better solution for video abnormal behavior detection in resource-constrained scenarios.

[0118] In summary, the 3D wavelet deep separable attention autoencoder of the present invention significantly reduces the number of model parameters while maintaining a high AUC value, achieving a better balance between detection accuracy and computational efficiency, and fully demonstrating its potential in practical applications.

[0119] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with this technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered by the scope of protection of the present invention.

Claims

1. A method for detecting abnormal behavior in video by integrating wavelet transform and depthwise separable convolution, characterized by: include: Label the images in the crowd abnormal behavior dataset as normal behavior and abnormal behavior, and adjust the image size; A portion of resized normal behavior images is used to generate pseudo-abnormal images. The pseudo-abnormal images and another portion of resized normal behavior images are input into a 3D wavelet deep separable attention autoencoder in the form of consecutive frames for training and evaluation, resulting in a 3D wavelet deep separable attention autoencoder that meets the expected detection accuracy. The image to be detected is input into a 3D wavelet deep separable attention autoencoder that meets the expected detection accuracy for detection, and the abnormal behavior detection result is obtained.

2. The method for detecting abnormal video behavior by integrating wavelet transform and depthwise separable convolution according to claim 1, characterized in that: The 3D wavelet deep separable attention autoencoder includes an improved encoder and an improved decoder; The improved encoder has four layers; The first layer structure is 3D convolution, batch normalization layer, and Leaky-Relu activation function in sequence; the second layer structure is 3D convolution, batch normalization layer, and Leaky-Relu activation function in sequence; the third layer structure is 3D depth-separable convolution, 3D attention module, batch normalization layer, and Leaky-Relu activation function in sequence; the fourth layer structure is 3D wavelet convolution, 3D attention module, batch normalization layer, and Leaky-Relu activation function in sequence; the first layer structure and the second layer structure of the improved encoder are both used to extract low-level features of the data and perform dimensionality reduction processing; the third layer structure of the improved encoder is used to perform deeper feature extraction on the data; the fourth layer of the improved encoder is used to map high-dimensional data to a latent space; The improved decoder has four layers; the first layer structure is 3D depth-separable convolution, 3D attention module, batch normalization layer, and Leaky-Relu activation function; the second layer structure is 3D depth-separable convolution, 3D attention module, batch normalization layer, and Leaky-Relu activation function; the third layer structure is 3D deconvolution, batch normalization layer, and Leaky-Relu activation function; the fourth layer structure is 3D deconvolution and Than activation function; the first layer structure and the second layer structure of the improved decoder are both used to gradually map the latent space representation back to the high-dimensional space of the input data and restore the structure of the data; the third layer structure of the improved decoder is used to map the data back to the high-dimensional space and refine the reconstruction process; the fourth layer structure of the improved decoder is used to restore the data to the original high-dimensional space and generate data as close to the original input as possible.

3. The method for detecting abnormal behavior in video by integrating wavelet transform and depthwise separable convolution according to claim 2, characterized in that: The construction of the 3D wavelet convolution specifically includes: Use wavelet transform to decompose the input feature map V in three dimensions D, W, and H and perform convolution operations to obtain low-frequency information and high-frequency information The high-frequency information include Intermediate feature maps of seven different high-frequency bands; For low frequency information and high-frequency information Perform inverse wavelet transform to obtain the output feature map That is 3D wavelet convolution; the relevant calculation formula is as follows: Among them, W is the convolution kernel weight tensor of size K×K×K; 3DWT is wavelet transform; 3DIWT is inverse wavelet transform.

4. The method for detecting abnormal video behavior by integrating wavelet transform and depthwise separable convolution according to claim 1, characterized in that: The method of using a portion of the normal behavior image with a resized size to generate a pseudo abnormal image specifically includes: A portion of the resized normal behavior images are first used in the form of continuous frames to detect normal areas through YoLoV3, and pseudo-abnormal data are added to the detected normal areas to generate pseudo-abnormal images.

5. The method for detecting abnormal video behavior by integrating wavelet transform and depthwise separable convolution according to claim 1 or 2, characterized in that: The pseudo-abnormal image and another part of the normal behavior image with adjusted size are inputted into the 3D wavelet deep separable attention autoencoder in the form of continuous frames for training and evaluation, and the 3D wavelet deep separable attention autoencoder with expected detection accuracy is obtained. Specifically, the following steps are performed: The pseudo-abnormal image and another portion of resized normal behavior images are input into the 3D wavelet deep separable attention autoencoder in the form of continuous frames. The operating parameters of the 3D wavelet deep separable attention autoencoder are set and the 3D wavelet deep separable attention autoencoder is trained on the experimental operation platform. The detection accuracy of the currently trained 3D wavelet deep separable attention autoencoder is evaluated using the area under the ROC curve and the coordinate axis (AreaUnder Curve), parameters, and floating point operations. If the detection accuracy reaches the expected level, stop the training and evaluation process of the 3D wavelet deep separable attention autoencoder; If the detection accuracy does not meet the expectations, adjust the operating parameters of the 3D wavelet deep separable attention autoencoder, and repeat the training and evaluation process of the 3D wavelet deep separable attention autoencoder until the detection accuracy meets the expectations, and obtain a 3D wavelet deep separable attention autoencoder that meets the expected detection accuracy.

6. The method for detecting abnormal video behavior by integrating wavelet transform and depthwise separable convolution according to claim 5, characterized in that: The operating parameters of the 3D wavelet deep separable attention autoencoder include initial learning rate, learning momentum, and weight decay rate, and the adaptive matrix estimation method (Adaptive Moment Estimation, Adam) is used to update the network parameters.

7. The method for detecting abnormal video behavior by integrating wavelet transform and depthwise separable convolution according to claim 5, characterized in that: The expected detection accuracy of the 3D wavelet deep separable attention autoencoder is set to 94%-96%.

8. A video abnormal behavior detection system integrating wavelet transform and depthwise separable convolution, characterized by: include: Image processing module: labels the images in the crowd abnormal behavior dataset as normal behavior and abnormal behavior, and adjusts the image size; Model training module: Use a portion of resized normal behavior images to generate pseudo-abnormal images. The pseudo-abnormal images and another portion of resized normal behavior images are input into a 3D wavelet deep separable attention autoencoder in the form of consecutive frames for training and evaluation, resulting in a 3D wavelet deep separable attention autoencoder that meets the expected detection accuracy. Abnormal behavior detection module: The image to be detected is input into a 3D wavelet deep separable attention autoencoder that meets the expected detection accuracy for detection to obtain the abnormal behavior detection result.

9. A method and device for detecting abnormal video behavior by integrating wavelet transform and depthwise separable convolution, characterized by: include: Memory: used for storing a computer program to implement the method for detecting abnormal behavior in video by integrating wavelet transform and depthwise separable convolution as described in any one of claims 1 to 7; Processor: configured to implement the video abnormal behavior detection method integrating wavelet transform and depthwise separable convolution as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for detecting abnormal behavior in video by fusing wavelet transform and depthwise separable convolution are implemented.

Images