A Real-Time Filtering Method for Surveillance Video Data Based on Anomaly Detection

Abstract

This invention discloses a real-time filtering method for surveillance video data oriented towards anomaly detection, comprising: adding mixed distortion to the surveillance image, analyzing the quality differences between data of different distortion levels, and determining the filtering threshold corresponding to each distortion type; determining the distortion type of the input video frame through a pre-established distortion classification network module; obtaining the quality score of the input video frame through a pre-established image quality evaluation module; filtering and splitting the input video frames, excluding and storing video frames with quality scores higher than the filtering threshold, and inputting video frames with quality scores lower than the filtering threshold into an anomaly detection network for anomaly detection. This invention can accurately filter and store low-quality video frames and apply the filtering method to anomaly detection, possessing advantages of objectivity, good real-time performance, and strong stability, effectively improving the performance of video anomaly detection algorithms in practical application scenarios where data quality fluctuates.

Description

Technical Field

[0001] This invention belongs to the field of image processing and relates to image usability evaluation, specifically to a real-time filtering method for surveillance video data oriented towards anomaly detection. Background Technology

[0002] With the development of artificial intelligence technology, smart security and intelligent monitoring have become important research directions in the field of computer vision, with high application value in industry. Currently, video surveillance has been widely applied in security, transportation, and other fields, generating a large amount of valuable video image data. Faced with this massive amount of data, traditional manual monitoring methods suffer from low efficiency and high labor costs. Therefore, video anomaly detection algorithms are typically used to automatically detect abnormal behavior in the input surveillance video images. Most existing video anomaly detection algorithms are based on clear video data, and the datasets used in this field are clear images without distortion. However, in actual monitoring scenarios, environmental factors such as rain, snow, fog, haze, and changes in lighting inevitably affect the recognizability of surveillance videos. Furthermore, during video storage and transmission, various types of distortion can occur due to unstable physical equipment, limited storage resources, and transmission channel interference, resulting in video quality loss and a decrease in the accuracy of video anomaly detection. Therefore, overcoming data quality fluctuations and improving the robustness of algorithms is a key challenge in the field of video anomaly detection. Designing an effective real-time data filtering and anomaly detection method is of significant research importance.

[0003] Currently, video anomaly detection methods are broadly categorized into two types: frame reconstruction-based and frame prediction-based methods. Reconstruction-based algorithms typically train an autoencoder on normal data to encode high-dimensional input into low-dimensional latent variables, allowing the network to learn the most informative features. During testing, anomalous data exhibits significant reconstruction errors and can be detected by the algorithm. Prediction algorithms leverage the context of the video, assuming that normal data can be predicted well, while anomalous data cannot. They predict the appearance and behavior of the next frame based on previous video frames. If there is a significant prediction error between the actual next frame and the predicted frame, the video frame can be identified as an anomalous frame. However, due to the strong generalization ability of autoencoders, some anomalous behaviors can also be reconstructed well, leading to missed anomalies. Meanwhile, frame prediction-based methods have a relatively simple structure and fast execution speed, making them more suitable for projects with high real-time requirements. Summary of the Invention

[0004] Purpose of the invention: To overcome the shortcomings of existing technologies, this invention provides a real-time filtering method for surveillance video data oriented towards anomaly detection. It adopts an image availability evaluation method based on image distortion classification and no-reference image quality assessment, which can accurately filter and store low-quality video frames. Furthermore, the filtering method is applied to video anomaly detection to uncover abnormal behavior. It has the advantages of objectivity, good real-time performance, and strong stability, effectively improving the performance of video anomaly detection algorithms in practical application scenarios where data quality fluctuates.

[0005] Technical Solution: To achieve the above objectives, this invention provides a real-time filtering method for surveillance video data oriented towards anomaly detection, comprising the following steps:

[0006] S1: Add mixed distortion to the monitoring image, analyze the quality difference between data of different distortion levels, and determine the screening threshold corresponding to each distortion type;

[0007] S2: Determine the distortion type of the input video frame by establishing a distortion classification network module;

[0008] The quality score of the input video frame is obtained by establishing a good image quality evaluation module;

[0009] S3: Based on the quality score and the filtering threshold corresponding to the distortion type in step S2, the input video frames are filtered and split. For video frames with a quality score higher than the filtering threshold, they are excluded and stored in a local folder. For video frames with a quality score lower than the filtering threshold, they are input into the anomaly detection network for anomaly detection.

[0010] Furthermore, the mixed distortion in step S1 consists of a combination of six types of distortion, including JPEG compression distortion (JPEG), JPEG2000 compression distortion (JP2K), Gaussian noise (Gau), motion blur (Mot), overexposure distortion (Bri), and underexposure distortion (Dark). By adding these six distortions, common distortions in actual monitoring scenarios are simulated. The specific distortion simulation process is as follows:

[0011] A1: JPEG compression distortion transforms the original image from RGB space to YCbCr space and performs a discrete cosine transform. Then, it obtains a distorted image through compression and encoding. The distortion level is controlled by adjusting the compression ratio; as the compression ratio increases, the image gradually exhibits a mosaic effect. The specific process of the discrete cosine transform is as follows:

[0012]

[0013]

[0014] In this context, the image signal is a two-dimensional function F(x, y), where x and y are the coordinates of a certain value in the index matrix, u and v are the coordinates of a certain value in the matrix after transformation, G(u, v) represents the data value of the point after transformation, and F(x, y) represents the data value of the point in the data matrix.

[0015] A2: JPEG2000 compression distortion uses a method similar to JPEG compression distortion. First, the image is preprocessed, and then the image is distorted through discrete wavelet transform, quantization, and encoding. As the compression ratio increases, the image gradually becomes blurred.

[0016] A3: Gaussian noise refers to noise whose probability distribution conforms to a Gaussian distribution. The probability density function is:

[0017]

[0018] Where z represents the gray value of Gaussian noise, μ is the mean of z, and σ is the standard deviation of z. In the simulation process, the degree of distortion is controlled by adjusting the variance v and the mean μ. The variance v affects the density of noise. The larger the variance v is, the denser the noise and the more noise there is in the image. The mean μ affects the brightness of the image. The larger the mean μ is, the brighter the image.

[0019] A4: The process of simulating motion blur involves convolving a clear image with a degraded model, then subjecting it to external noise to generate a motion-blurred image. The distortion level is controlled by adjusting the motion angle and motion length; the higher the value, the more blurred the image.

[0020] A5: Overbrightness and Underbrightness Distortion: Image brightness is changed by adjusting the grayscale values ​​of the RGB channels. When the grayscale values ​​of all three channels are increased simultaneously, the image becomes brighter, and conversely, when the grayscale values ​​are decreased, the image appears darker.

[0021] Furthermore, among the six types of image distortion in step S1, the compression ratio of JPEG compression distortion ranges from [1, 20]; the compression ratio of JPEG2000 compression distortion ranges from [50, 500]; the mean of Gaussian noise ranges from [0.05, 0.50], and the variance ranges from [0.05 to 0.40]; the motion length of motion blur ranges from [2, 25], and the motion angle ranges from [2, 25]; the grayscale change value of overly bright distortion ranges from [20, 140]; and the grayscale change value of overly dark distortion ranges from [10, 140]. Since image quality does not change uniformly with the uniform change of parameter values, each distortion parameter is set in a non-equal interval manner.

[0022] In step S1, an incremental method is used to add distortion. First, JPEG compression distortion or JPEG2000 compression distortion and Gaussian noise, which are common in monitoring scenes, are added to the monitoring image. Then, motion blur is added. Finally, over-brightness distortion and over-darkness distortion are added respectively. Specifically, there are 12 distortion combinations: (1) JPEG+Gau, (2) JP2K+Gau, (3) JPEG+Gau+Bri, (4) JP2K+Gau+Bri, (5) JPEG+Gau+Dark, (6) JP2K+Gau+Dark, (7) JPEG+Gau+Mot, (8) JP2K+Gau+Mot, (9) JPEG+Gau+Mot+Bri, (10) JP2K+Gau+Mot+Bri, (11) JPEG+Gau+Mot+Dark, and (12) JP2K+Gau+Mot+Dark.

[0023] Furthermore, in step S1, an existing video anomaly detection algorithm is used to train and test on a mixed distortion dataset to determine the impact of data quality fluctuations on algorithm performance. It is determined that the accuracy of the algorithm drops significantly when the distortion level reaches 4 or above. Therefore, video frames with distortion levels of 4 and 5 are selected as data to be excluded.

[0024] Furthermore, in step S2, the distortion classification network module is a distortion classification network based on ResNet-18. Specifically, the distortion classification is as follows: first, the image is input into a convolutional layer and a max pooling layer, then through three residual blocks, and finally through a fully connected layer to obtain a 1*12 vector. The distortion type corresponding to the maximum value is taken as the result. The training data and test data of the distortion classification network are both taken from a mixed distortion dataset.

[0025] Furthermore, in step S2, the image quality assessment module employs a no-reference image quality assessment method based on convolutional neural networks. The specific process is as follows:

[0026] B1: Perform local normalization on the image. Assuming the pixel intensity at position (i, j) is I(i, j), calculate its normalized value.

[0027]

[0028]

[0029]

[0030] Where: C is a positive constant, and P and Q are the normalized window sizes;

[0031] B2: Convolve the locally normalized image patches using 50 convolutional kernels, with each kernel generating a feature map. Perform max pooling and min pooling on each feature map; assume... This represents the response of the k-th convolutional kernel at (i, j) in the feature map:

[0032]

[0033]

[0034] Where: u k v k These are the maximum and minimum values, respectively.

[0035] B3: Input the pooling result into two fully connected layers, and use a non-linear form of ReLU in the fully connected layers to obtain the final quality score:

[0036] g = max(0, ∑ i w i a i )

[0037] Where: g is the output of the fully connected layer, w i For ReLU weights, a i This is the output of the previous layer.

[0038] Further, in step S2, a no-reference image quality evaluation method based on convolutional neural networks is used to evaluate the mixed distortion dataset, obtaining quality scores for data of different distortion levels for each distortion type. The quality score ranges from [1, 5], with low to high indicating a gradual decrease in image quality. A quality score of 1 indicates good image quality with almost no distortion, while a quality score of 5 indicates extremely poor image quality where the target in the image is unidentifiable. Each distortion combination is calculated separately, and the test results of the video anomaly detection algorithm on data of different distortion combinations and distortion levels are combined. The threshold settings for different distortion types are different, and the average quality score of distortion level 4 data and distortion level 3 data, which have a greater impact on algorithm performance, is used as the screening threshold for this distortion combination.

[0039] Furthermore, in step S2, the surveillance video frames captured by the camera are classified for distortion and the image quality is evaluated to obtain the distortion type and quality score of the video frame. Based on the quality score, a threshold corresponding to the distortion type is used for filtering. If the quality score is higher than the threshold, the frame is considered a low-quality frame. If the quality score is lower than the threshold, the frame is considered a high-quality frame.

[0040] Furthermore, in step S3, the filtered video frames are split into streams to exclude low-quality video frames from the entire video segment, thereby improving the overall data quality. The low-quality frames are also stored in the local folder corresponding to the distortion type they contain, and data on the real-world scene with each distortion is collected.

[0041] Furthermore, in step S3, a video anomaly detection network based on the U-net architecture is used to generate the predicted next frame. Anomalies in the video are detected by comparing the difference between the predicted frame and the actual frame. The specific process is as follows:

[0042] C1: Input consecutive video frames I1, I2, I3, ..., I t U-Net is used as the generator to predict the next frame. Constraints are imposed on the predicted frame in terms of appearance and motion information to make the predicted frame closer to the actual next frame. t+1 :

[0043]

[0044]

[0045]

[0046] Where: L int For strength loss, L gd For gradient loss, L op For optical flow loss, i and j represent the spatial position of the pixel in the frame, and f is the pre-trained FlowNet used to extract optical flow information;

[0047] C2: Calculate the peak signal-to-noise ratio between the actual frame and the predicted frame to measure the degree of difference between the two frames:

[0048]

[0049] C3: Normalize the PSNR of all frames to the range [0, 1], and calculate the anomaly score for each frame:

[0050]

[0051] Based on the anomaly score S(t), a threshold is set according to the application scenario to determine whether anomalies exist.

[0052] In this invention, a self-attention mechanism is added to the video anomaly detection network based on the U-net architecture to obtain better feature representation. The video anomaly detection network includes three downsampling processes and three upsampling processes, and the self-attention mechanism is mapped in each sampling process.

[0053] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0054] 1. Compared with commonly used datasets in the field of video anomaly detection, this invention constructs a hybrid distortion dataset, which fully considers the data quality fluctuation problem caused by image distortion, is more in line with the characteristics of surveillance videos, and has a wider range of application scenarios.

[0055] 2. Compared to directly applying existing video anomaly detection algorithms to real-world scenarios, this invention considers multiple sources of image distortion that cause data quality fluctuations, quantitatively and qualitatively analyzes the characteristics of surveillance images, and obtains more accurate prediction results by first screening low-quality video frames and then performing anomaly detection.

[0056] 3. Compared with existing methods for image restoration of low-quality images, the screening method of this invention is more in line with the characteristics of large data volume and high real-time requirements in monitoring scenarios, and has higher computational efficiency.

[0057] 4. This invention stores low-quality images from the actual detection process locally, providing a large amount of real-world data for research on low-quality images and offering a valuable reference for image usability evaluation tasks, thus possessing high practical value. Attached Figure Description

[0058] Figure 1 : Flowchart of the method of the present invention;

[0059] Figure 2 The six types of distortion images added in this embodiment of the invention are: (a) the original image, (b) the JPEG compression distortion image, (c) the JPEG2000 compression distortion image, (d) the Gaussian noise image, (e) the motion blur image, (f) the overly bright distortion image, and (g) the overly dark distortion image.

[0060] Figure 3 The mixed distortion dataset of this embodiment of the invention includes the following data: the first row adds JPEG compression distortion and Gaussian noise, the second row adds JPEG2000 compression distortion, Gaussian noise and over-darkness distortion, and the third row adds JPEG compression distortion, Gaussian noise, motion blur and over-brightness distortion, with levels 1 to 5 from left to right.

[0061] Figure 4 The embodiments of this invention employ the full-reference image quality assessment method SSIM to verify the usability of the mixed distortion dataset;

[0062] Figure 5 The first row, from left to right, shows the image distortion at levels one to five, including JPEG compression distortion, Gaussian noise, motion blur, and overexposure distortion. The second row is a heatmap showing the difference between the reconstructed frame and the actual frame.

[0063] Figure 6 The present invention includes a video anomaly detection module with a self-attention mechanism. Detailed Implementation

[0064] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading this invention, any modifications of the invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0065] This invention provides a real-time filtering method for surveillance video data oriented towards anomaly detection, such as... Figure 1 As shown, it includes the following steps:

[0066] S1: Add a mixed distortion consisting of 6 types of distortion to the monitoring image, construct a dataset containing 5 levels of mixed distortion, analyze the quality difference between data of different distortion levels, and determine the screening range corresponding to each distortion type;

[0067] S2: Real-time filtering of input video frames. First, a distortion classification network based on ResNet-18 is used to determine the distortion type contained in the video frame. Then, no-reference image quality evaluation is performed. If the quality score is higher, it means that the image quality of the frame is worse. Then, the video frames are filtered according to the threshold corresponding to the distortion type contained in the frame.

[0068] S3: The filtered data is split into streams. Video frames with quality scores higher than the threshold are excluded and stored in a local folder. For video frames with quality scores lower than the threshold, a video anomaly detection network based on the U-net architecture is used to generate the predicted next frame. Anomalies in the video are detected by comparing the difference between the predicted frame and the actual frame.

[0069] In step S1:

[0070] Six common image distortions in surveillance scenarios include JPEG compression distortion (JPEG), JPEG2000 compression distortion (JP2K), Gaussian noise (Gau), motion blur (Mot), overexposure distortion (Bri), and underexposure distortion (Dark). By adding these six distortions, common distortions in real-world surveillance scenarios are simulated. Figure 2 The effects of adding the above six distortions are shown. The specific distortion simulation process is as follows:

[0071] A1: JPEG compression distortion transforms the original image from RGB space to YCbCr space and performs a discrete cosine transform. Then, it obtains a distorted image through compression and encoding. The distortion level is controlled by adjusting the compression ratio; as the compression ratio increases, the image gradually exhibits a mosaic effect. The specific process of the discrete cosine transform is as follows:

[0072]

[0073]

[0074] In this context, the image signal is a two-dimensional function F(x, y), where x and y are the coordinates of a certain value in the index matrix, u and v are the coordinates of a certain value in the matrix after transformation, G(u, v) represents the data value of the point after transformation, and F(x, y) represents the data value of the point in the data matrix.

[0075] A2: JPEG2000 compression distortion uses a method similar to JPEG compression distortion. First, the image is preprocessed, and then the image is distorted through discrete wavelet transform, quantization, and encoding. As the compression ratio increases, the image gradually becomes blurred.

[0076] A3: Gaussian noise refers to noise whose probability distribution conforms to a Gaussian distribution. The probability density function is:

[0077]

[0078] Where z represents the gray value of Gaussian noise, μ is the mean of z, and σ is the standard deviation of z. In the simulation process, the degree of distortion is controlled by adjusting the variance v and the mean μ. The variance v affects the density of noise. The larger the variance v is, the denser the noise and the more noise there is in the image. The mean μ affects the brightness of the image. The larger the mean μ is, the brighter the image.

[0079] A4: The process of simulating motion blur involves convolving a clear image with a degraded model, then subjecting it to external noise to generate a motion-blurred image. The distortion level is controlled by adjusting the motion angle and motion length; the higher the value, the more blurred the image.

[0080] A5: Overbrightness and Underbrightness Distortion: Image brightness is changed by adjusting the grayscale values ​​of the RGB channels. When the grayscale values ​​of all three channels are increased simultaneously, the image becomes brighter, and conversely, when the grayscale values ​​are decreased, the image appears darker.

[0081] The compression ratio range for JPEG compression distortion is [1, 20]; the compression ratio range for JPEG2000 compression distortion is [50, 500]; the mean value range for Gaussian noise is [0.05, 0.50], and the variance range is [0.05~0.40]; the motion length range for motion blur is [2, 25], and the motion angle range is [2, 25]; the grayscale variation value range for overly bright distortion is [20, 140]; the grayscale variation value range for overly dark distortion is [10, 140]; since image quality does not change uniformly with uniform parameter values, non-uniform intervals are used to set the distortion parameters.

[0082] The incremental method is used to add distortion. First, JPEG compression distortion or JPEG2000 compression distortion and Gaussian noise, which are common in monitoring scenes, are added to the monitoring image. Then, motion blur is added. Finally, over-brightness distortion and over-darkness distortion are added respectively. Specifically, there are 12 distortion combinations: (1) JPEG+Gau, (2) JP2K+Gau, (3) JPEG+Gau+Bri, (4) JP2K+Gau+Bri, (5) JPEG+Gau+Dark, (6) JP2K+Gau+Dark, (7) JPEG+Gau+Mot, (8) JP2K+Gau+Mot, (9) JPEG+Gau+Mot+Bri, (10) JP2K+Gau+Mot+Bri, (11) JPEG+Gau+Mot+Dark, and (12) JP2K+Gau+Mot+Dark. Figure 3 This shows a portion of the mixed distortion dataset. The credibility of the mixed distortion dataset is verified using the full-reference image quality assessment method SSIM. Figure 4 The verification results show that the objective quality of each distorted combination of data decreases in a stepwise manner, which is consistent with the subjective level classification, proving that the dataset is reasonable and credible.

[0083] Existing video anomaly detection algorithms were trained and tested on a mixed distortion dataset to determine the impact of data quality fluctuations on algorithm performance. It was found that the algorithm's accuracy significantly decreased when the distortion level reached 4 or higher; therefore, video frames with distortion levels of 4 and 5 were excluded. Figure 5 The visualization results show that distortion amplifies the difference between the reconstructed frame and the actual frame in the background, negatively impacting the calculation of the reconstruction loss and consequently reducing the accuracy of the algorithm.

[0084] In step S2:

[0085] The ResNet-18 network is used for distortion classification. Specifically, the image is first input into a convolutional layer and a max pooling layer, then passed through three residual blocks, and finally passed through a fully connected layer to obtain a 1*12 vector. The distortion type corresponding to the maximum value is taken as the result. The training and testing data of the distortion classification network are both taken from a mixed distortion dataset.

[0086] In addition, the image quality assessment module adopts a no-reference image quality assessment method based on convolutional neural networks, the specific process of which is as follows:

[0087] B1: Perform local normalization on the image. Assuming the pixel intensity at position (i, j) is I(i, j), calculate its normalized value.

[0088]

[0089]

[0090]

[0091] Where: C is a positive constant, and P and Q are the normalized window sizes;

[0092] B2: Convolve the locally normalized image patches using 50 convolutional kernels, with each kernel generating a feature map. Perform max pooling and min pooling on each feature map; assume... This represents the response of the k-th convolutional kernel at (i, j) in the feature map:

[0093]

[0094]

[0095] Where: u k v k These are the maximum and minimum values, respectively.

[0096] B3: Input the pooling result into two fully connected layers, and use a non-linear form of ReLU in the fully connected layers to obtain the final quality score:

[0097] g = max(0, ∑ i w i a i )

[0098] Where: g is the output of the fully connected layer, w i For ReLU weights, a i This is the output of the previous layer.

[0099] A no-reference image quality assessment method based on convolutional neural networks was used to evaluate the mixed distortion dataset, obtaining quality scores for different distortion levels for each distortion type. The quality scores range from [1, 5], with low to high indicating a gradual decrease in image quality. A quality score of 1 indicates good image quality with almost no distortion, while a quality score of 5 indicates extremely poor image quality where the target in the image is unidentifiable. Each distortion combination was calculated separately, and the test results of the video anomaly detection algorithm on data of different distortion combinations and distortion levels were combined. The threshold settings for different distortion types are different. The average quality score of distortion level 4 data and distortion level 3 data, which have a greater impact on algorithm performance, was used as the screening threshold for this distortion combination.

[0100] Distortion classification and image quality evaluation are performed on the surveillance video frames captured by the camera to obtain the distortion type and quality score of the video frame. The frame is then filtered using the threshold corresponding to the distortion type. If the quality score is higher than the threshold, the frame is considered a low-quality frame, and if the quality score is lower than the threshold, the frame is considered a high-quality frame.

[0101] In step S3:

[0102] The filtered video frames are split into streams to exclude low-quality video frames from the entire video segment, thereby improving the overall data quality. The low-quality frames are then stored in the local folder corresponding to the distortion type they contain, and data on the real-world scene with each distortion is collected.

[0103] For high-quality video frames, a frame prediction-based video anomaly detection algorithm is used to discover anomalous behavior. A video anomaly detection network based on the U-net architecture is used to generate the predicted next frame. Anomalies in the video are discovered by comparing the difference between the predicted frame and the actual frame. The specific process is as follows:

[0104] C1: Input consecutive video frames I1, I2, I3, ..., I t U-Net is used as the generator to predict the next frame. Constraints are imposed on the predicted frame in terms of appearance and motion information to make the predicted frame closer to the actual next frame. t+1 :

[0105]

[0106]

[0107]

[0108] Where: L int For strength loss, L gd For gradient loss, L opFor optical flow loss, i and j represent the spatial position of the pixel in the frame, and f is the pre-trained FlowNet used to extract optical flow information;

[0109] C2: Calculate the peak signal-to-noise ratio between the actual frame and the predicted frame to measure the degree of difference between the two frames:

[0110]

[0111] C3: Normalize the PSNR of all frames to the range [0, 1], and calculate the anomaly score for each frame:

[0112]

[0113] Based on the anomaly score S(t), a threshold is set according to the application scenario to determine whether anomalies exist.

[0114] Figure 6 The structure of the video anomaly detection module is shown. A self-attention module is added to the frame prediction-based video anomaly detection method to obtain better feature representations. The attention mechanism can be viewed as a dynamic feature selection process, enhancing the model's ability to process the spatiotemporal features of video data and ensuring that the model can better locate the regions where anomalies occur. The network includes three downsampling processes and three upsampling processes, with each sampling process involving a self-attention mechanism.

[0115] Based on the above scheme, in order to verify the effectiveness and practical effect of the method of the present invention, this embodiment adopts the reconstruction-based methods MNAD and STEAL, and the prediction-based methods ATTFU and HF, which have relatively good performance among existing video anomaly algorithms. 2 VAD was applied to the UCSD Ped2 and CUHK Avenue datasets with added mixed distortion using both the method of this invention and the original method. The specific comparison results are shown in Table 1.

[0116] Table 1 - Performance Improvement of the Algorithm by the Method of the Invention

[0117]

[0118] As can be seen from Table 1, the accuracy of the algorithm is significantly improved after using the method of the present invention to filter low-quality video frames containing distortion.

[0119] To define the scope of low-quality data, we employ two of the better-performing existing video anomaly algorithms: MNAD and STEAL (based on reconstruction), and ATTFU and HF (based on prediction). 2 Table 2 shows the performance comparison results of VAD on mixed distortion datasets:

[0120] Table 2 - Accuracy of Video Anomaly Detection Algorithms on Mixed Distortion Data

[0121]

[0122] As can be seen from Table 2, the accuracy of the video anomaly detection algorithm gradually decreases as the distortion level increases, and the accuracy drops sharply when the distortion level reaches a certain level. Therefore, the method of this invention uses data with distortion level 4 and distortion level 5 as data to be excluded.

[0123] The objective quality assessment results of data for each distortion type using the no-reference image quality assessment method are shown in Table 3:

[0124] Table 3 - Objective Image Quality Evaluation Results for the Mixed Distortion Dataset

[0125]

[0126] As can be seen from Table 3, the image quality scores of each level of data in the mixed distortion dataset are significantly different, so low-quality data can be filtered out based on the quality scores.

[0127] The experimental data above show that the method of the present invention can effectively improve the performance of video anomaly detection algorithms when data quality fluctuates, and improve the accuracy of the algorithm in practical application scenarios.

Claims

1. A real-time filtering method for surveillance video data for anomaly detection, characterized in that, Includes the following steps: S1: Add mixed distortion to the monitoring image, analyze the quality difference between data of different distortion levels, and determine the screening threshold corresponding to each distortion type; S2: Determine the distortion type of the input video frame by establishing a distortion classification network module; The quality score of the input video frame is obtained by establishing a good image quality evaluation module; S3: Based on the quality score and the filtering threshold corresponding to the distortion type in step S2, the input video frames are filtered and split. For video frames with a quality score higher than the filtering threshold, they are excluded and stored in a local folder. For video frames with a quality score lower than the filtering threshold, they are input into the anomaly detection network for anomaly detection. In step S2, the image quality assessment module adopts a no-reference image quality assessment method based on convolutional neural networks. The specific process is as follows: B1: Perform local normalization on the image. Assuming the pixel intensity at position (i,j) is I(i,j), calculate its normalized value. Where: C is a positive constant, and P and Q are the normalized window sizes; B2: Convolve the locally normalized image patches using 50 convolutional kernels, with each kernel generating a feature map. Perform max pooling and min pooling on each feature map; assume... This represents the response of the k-th convolutional kernel at (i,j) in the feature map: Where: u k v k These are the maximum and minimum values, respectively. B3: Input the pooling result into two fully connected layers, and use a non-linear form of ReLU in the fully connected layers to obtain the final quality score: g=max(0,∑ i w i a i ) Where: g is the output of the fully connected layer, w i For ReLU weights, a i This is the output of the previous layer; In step S2, a no-reference image quality assessment method based on convolutional neural networks is used to evaluate the mixed distortion dataset, obtaining quality scores for data of different distortion levels for each distortion type. The quality score ranges from [1, 5], with low to high indicating a gradual decrease in image quality. A quality score of 1 indicates good image quality with almost no distortion, while a quality score of 5 indicates extremely poor image quality where the target cannot be identified. Each distortion combination is calculated separately, and the test results of the video anomaly detection algorithm on data of various distortion combinations and different distortion levels are combined. The average quality score of the distortion level 4 data and the distortion level 3 data, which have a greater impact on algorithm performance, is used as the screening threshold for this distortion combination.

2. The real-time filtering method for surveillance video data oriented towards anomaly detection according to claim 1, characterized in that, In step S1, the mixed distortion is a combination of six types of distortion, including JPEG compression distortion, JPEG2000 compression distortion, Gaussian noise, motion blur, overexposure distortion, and underexposure distortion. By adding these six types of distortion, common distortions in actual monitoring scenarios are simulated. The specific distortion simulation process is as follows: A1: JPEG compression distortion transforms the original image from RGB space to YCbCr space and performs a discrete cosine transform. Then, it obtains a distorted image through compression and encoding. The distortion level is controlled by adjusting the compression ratio; as the compression ratio increases, the image gradually exhibits a mosaic effect. The specific process of the discrete cosine transform is as follows: Wherein, the image signal is a two-dimensional function F(x,y), where x and y are the coordinates of a certain value in the index matrix, u and v are the coordinates of a certain value in the matrix after transformation, G(u,v) represents the data value of the point after transformation, and F(x,y) represents the data value of the point in the data matrix. A2: JPEG2000 compression distortion: First, the image is preprocessed, then distorted images are obtained through discrete wavelet transform, quantization, and encoding. As the compression ratio increases, the image gradually becomes blurred. A3: Gaussian noise refers to noise whose probability distribution conforms to a Gaussian distribution. The probability density function is: Where z represents the gray value of Gaussian noise, μ is the mean of z, and σ is the standard deviation of z. In the simulation process, the degree of distortion is controlled by adjusting the variance v and the mean μ. The variance v affects the density of noise. The larger the variance v is, the denser the noise and the more noise there is in the image. The mean μ affects the brightness of the image. The larger the mean μ is, the brighter the image. A4: The process of simulating motion blur involves convolving a clear image with a degraded model, then subjecting it to external noise to generate a motion-blurred image. The distortion level is controlled by adjusting the motion angle and motion length; the higher the value, the more blurred the image. A5: Overbrightness and Underbrightness Distortion: Image brightness is changed by adjusting the grayscale values ​​of the RGB channels. When the grayscale values ​​of all three channels are increased simultaneously, the image becomes brighter, and conversely, when the grayscale values ​​are decreased, the image appears darker.

3. The real-time filtering method for surveillance video data oriented towards anomaly detection according to claim 2, characterized in that, In step S1, an incremental method is used to add distortion. First, JPEG compression distortion or JPEG2000 compression distortion and Gaussian noise, which are common in monitoring scenes, are added to the monitoring image. Then, motion blur is added. Finally, over-brightness distortion and over-darkness distortion are added respectively. Specifically, there are 12 distortion combinations: (1) JPEG+Gau, (2) JP2K+Gau, (3) JPEG+Gau+Bri, (4) JP2K+Gau+Bri, (5) JPEG+Gau+Dark, (6) JP2K+Gau+Dark, (7) JPEG+Gau+Mot, (8) JP2K+Gau+Mot, (9) JPEG+Gau+Mot+Bri, (10) JP2K+Gau+Mot+Bri, (11) JPEG+Gau+Mot+Dark, and (12) JP2K+Gau+Mot+Dark.

4. The real-time filtering method for surveillance video data oriented towards anomaly detection according to claim 3, characterized in that, In step S1, reconstruction-based and prediction-based video anomaly detection algorithms are used. The algorithms are trained and tested on a mixed distortion dataset to determine the impact of data quality fluctuations on algorithm performance. It is determined that the accuracy of the algorithm drops significantly when the distortion level reaches 4 or above. Therefore, video frames with distortion levels of 4 and 5 are excluded as data to be excluded.

5. The real-time filtering method for surveillance video data oriented towards anomaly detection according to claim 1, characterized in that, The distortion classification network module in step S2 is a distortion classification network based on ResNet-18. The specific distortion classification process is as follows: First, the image is input into a convolutional layer and a max pooling layer, then through three residual blocks, and finally through a fully connected layer to obtain a 1*12 vector. The distortion type corresponding to the maximum value is taken as the result.

6. The real-time filtering method for surveillance video data oriented towards anomaly detection according to claim 1, characterized in that, In step S3, a video anomaly detection network based on the U-net architecture is used to generate the predicted next frame. Anomalies in the video are detected by comparing the difference between the predicted frame and the actual frame. The specific process is as follows: C1: Input consecutive video frames I1, I2, I3, ..., I t U-Net is used as the generator to predict the next frame. Constraints are imposed on the predicted frame in terms of appearance and motion information to make the predicted frame closer to the actual next frame. t+1 : Where: L int For strength loss, L gd For gradient loss, L op For optical flow loss, i,j represents the spatial position of the pixel in the frame, and f is the pre-trained FlowNet used to extract optical flow information; C2: Calculate the peak signal-to-noise ratio between the actual frame and the predicted frame to measure the degree of difference between the two frames: C3: Normalize the PSNR of all frames to the range [0,1], and calculate the anomaly score for each frame: Based on the anomaly score S(t), a threshold is set according to the application scenario to determine whether anomalies exist.

7. A real-time filtering method for surveillance video data oriented towards anomaly detection according to claim 6, characterized in that, In step S3, a self-attention mechanism is added to the video anomaly detection network based on the U-net architecture. The video anomaly detection network includes three downsampling processes and three upsampling processes, and the self-attention mechanism is mapped in each sampling process.

Images