Lightweight video anomaly detection method and system based on information gating compression

Abstract

The invention provides a lightweight video anomaly detection method and system based on information gating compression, and the method comprises the steps: carrying out the feature extraction of a video sequence, and obtaining a high-dimensional semantic feature; calculating an L2 norm, an energy feature vector and a mean feature vector by using the high-dimensional semantic features; enhanced features are obtained through the L2 norm, the energy features, the mean value features and the high-dimensional semantic features; utilizing the enhanced feature vector to obtain a dynamic gating weight; obtaining a value judgment matrix by using an L2 norm; based on the video sequence length, obtaining a dynamic mask through calculation; obtaining a final feature vector through the dynamic mask; obtaining a final abnormal score by using the final feature vector; and obtaining a detection result through the final abnormal score. According to the method, by introducing an attention mechanism and an amplitude-based gating strategy, the non-key feature flow is adaptively weighted or cut off, and the model is guided to focus effective information.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and video analysis, and in particular to a lightweight video anomaly detection method and system based on information gating compression. Background Art

[0002] Video anomaly detection aims to identify abnormal behaviors or events in videos and is a core technology for intelligent security and industrial automation. The current mainstream methods mainly include two categories. One is the dual-branch architecture, which captures details and context information through local and global branches respectively (such as HF2-VAD), but the feature interaction between branches is complex, and the amount of calculation increases quadratically with the length of the video. The other is the diffusion model, which reconstructs the normal mode based on the idea of ​​generative adversarial (such as AnoDiff), which requires stacking multiple layers of networks to fit the data distribution, with a large number of parameters and unstable training.

[0003] The existing methods have the following core problems: First, in terms of blind feature processing, the input features are calculated indiscriminately, resulting in more than 70% of the features being redundant or noisy; second, the structure is complex and inefficient, relying on deep networks (such as ResNet-50) or multi-head attention, and the model parameters exceed 10M. Third, the generalization ability is limited, and the interference of noise features causes the model's migration performance in cross-scene data (such as UCF-Crime→ShanghaiTech) to drop by 12%~15%. Summary of the invention

[0004] In view of the above situation, the main purpose of the present invention is to propose a lightweight video anomaly detection method and system based on information gating compression to solve the above technical problems.

[0005] The present invention proposes a lightweight video anomaly detection method based on information gating compression, the method comprising the following steps: Step 1, dividing the video in the video sequence into a static image sequence at a certain rate; Perform bilinear difference scaling on each frame of the static image sequence to obtain a scaled static image; The scaled static image is input into the CLIP-ViT model for semantic feature extraction to obtain a high-dimensional semantic feature vector. Step 2: Calculate the L2 norm, energy feature vector and mean feature vector using high-dimensional semantic features; The L2 norm, energy feature vector, mean feature vector and high-dimensional semantic feature vector are concatenated to obtain an enhanced feature vector; The dynamic gating weights are obtained using the enhanced feature vectors; Step 3, normalizing the L2 norm to obtain a normalized L2 norm; The feature relationship matrix is ​​constructed through the normalized L2 norm, and the value judgment matrix is ​​obtained using the feature relationship matrix and dynamic gating weights; Based on the length of the video sequence, a dynamic mask is obtained by calculation; Perform linear projection on the enhanced feature vector to obtain the query vector, key vector and value vector; Using the query vector, key vector and value vector, combined with the value judgment matrix and dynamic mask, the final feature vector is obtained through attention calculation; Step 4: Input the final feature vector into the classification network to obtain the final anomaly score; The detection result is obtained through the final anomaly score.

[0006] The present invention also proposes a lightweight video anomaly detection system based on information gating compression, the system comprising: Preprocessing module for: Perform bilinear difference scaling on each frame of the static image sequence to obtain a scaled static image; The scaled static image is input into the CLIP-ViT model for semantic feature extraction to obtain a high-dimensional semantic feature vector. Gated weight module, used to: Use high-dimensional semantic features to calculate L2 norm, energy eigenvector and mean eigenvector; The L2 norm, energy feature vector, mean feature vector and high-dimensional semantic feature vector are concatenated to obtain an enhanced feature vector; The dynamic gating weights are obtained using the enhanced feature vectors; The final feature output module is used to: Normalize the L2 norm to obtain a normalized L2 norm; The feature relationship matrix is ​​constructed through the normalized L2 norm, and the value judgment matrix is ​​obtained using the feature relationship matrix and dynamic gating weights; Based on the length of the video sequence, a dynamic mask is obtained by calculation; Perform linear projection on the enhanced feature vector to obtain the query vector, key vector and value vector; Using the query vector, key vector and value vector, combined with the value judgment matrix and dynamic mask, the final feature vector is obtained through attention calculation; Result output module, used for: Input the final feature vector into the classification network to obtain the final anomaly score; The detection result is obtained through the final anomaly score.

[0007] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention significantly improves detection accuracy and real-time performance by dynamically managing feature flow and computing resource allocation while reducing model complexity; 2. The present invention introduces the attention mechanism and the amplitude-based gating strategy to adaptively weight or truncate non-critical feature streams, thereby guiding the model to focus on effective information; 3. The present invention adopts a lightweight classification network to hierarchically compress feature dimensions and combines residual connections to improve the efficiency of gradient propagation.

[0008] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description or learned through embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 This is a flow chart of the lightweight video anomaly detection method based on information gating compression proposed by the present invention; Figure 2 It is a performance comparison diagram of the present invention and the prior art; Figure 3 This is a schematic diagram of the overall framework of the lightweight video anomaly detection system based on information gating compression proposed in the present invention. DETAILED DESCRIPTION

[0010] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0011] These and other aspects of the embodiments of the present invention will be apparent with reference to the following description and accompanying drawings. In these descriptions and accompanying drawings, some specific implementations of the embodiments of the present invention are specifically disclosed to represent some ways of implementing the principles of the embodiments of the present invention, but it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0012] See also Figure 1 The embodiment of the present invention proposes a lightweight video anomaly detection method based on information gating compression, which includes the following steps: Step 1, dividing the video in the video sequence into a static image sequence at a certain rate; Perform bilinear difference scaling on each frame of the static image sequence to obtain a scaled static image; The scaled static image is input into the CLIP-ViT model for semantic feature extraction to obtain a high-dimensional semantic feature vector. In step 1, the scaled static image is input into the CLIP-ViT model for semantic feature extraction to obtain a high-dimensional semantic feature vector. The corresponding process has the following relationship: ; in, Indicates a resizing operation. Indicates the total number of frames, The index representing the frame number. Indicates The high-dimensional semantic feature vector of the frame, Indicates A static image of a frame, Indicates that it has been processed by the CLIP-ViT model.

[0013] Furthermore, in order to unify the size of the images after video segmentation, the video is segmented into frames, adjusted to 224x224 pixels, and the pixel values ​​are normalized to [-1,1] to ensure the consistency and availability of the data and prepare for subsequent model processing; In addition, the CLIP-ViT model divides the image into multiple fixed-size image blocks, performs linear projection on each image block, maps the linearly projected image block to a low-dimensional space, and then adds position encoding to retain the spatial position information of the image block.

[0014] Step 2: Calculate the L2 norm, energy feature vector and mean feature vector using high-dimensional semantic features; The L2 norm, energy feature vector, mean feature vector and high-dimensional semantic feature vector are concatenated to obtain an enhanced feature vector; The dynamic gating weights are obtained using the enhanced feature vectors; In step 2, the L2 norm, energy feature vector, mean feature vector and high-dimensional semantic feature vector are concatenated to obtain an enhanced feature vector. The relationship between the corresponding process is: ; in, Indicates The enhanced feature vector of the frame, Indicates The high-dimensional semantic feature vector of the frame dimensions, Represents dimension index; The dynamic gating weight is obtained by using the enhanced feature vector. The specific steps are as follows: The enhanced feature vector is used to calculate the exponentially decayed Manhattan distance to generate the inter-frame similarity matrix. The relationship between the corresponding process is: ; in, represents the inter-frame similarity matrix, The index representing the frame number. Indicates Enhanced feature vector of the frame; Using the inter-frame similarity matrix, the screening threshold is adjusted through learnable parameters to generate dynamic gating weights. The corresponding process has the following relationship: ; in, represents the dynamic gating weight, Indicates that it has been processed by the Sigmoid function. represents the learnable parameters, Indicates that it has been processed by expected value operation.

[0015] Furthermore, three complementary statistics are introduced based on the 512-dimensional high-dimensional semantic feature vector: L2 norm, energy feature and mean feature. The L2 norm calculates the Euclidean distance between feature vectors to characterize their overall strength; the energy feature is the sum of squares of each dimension to reflect the activity of the feature; and the mean feature provides information about the distribution center of the feature.

[0016] Step 3, normalizing the L2 norm to obtain a normalized L2 norm; The feature relationship matrix is ​​constructed through the normalized L2 norm, and the value judgment matrix is ​​obtained using the feature relationship matrix and dynamic gating weights; Based on the length of the video sequence, a dynamic mask is obtained by calculation; Perform linear projection on the enhanced feature vector to obtain the query vector, key vector and value vector; Using the query vector, key vector and value vector, combined with the value judgment matrix and dynamic mask, the final feature vector is obtained through attention calculation; In step 3, the L2 norm is normalized to obtain the normalized L2 norm. The relationship between the corresponding process is: ; in, After normalization, The L2 norm of the frame, Indicates The L2 norm of the frame, represents the set of L2 norms of all eigenvectors, Represents a very small positive number; The feature relationship matrix is ​​constructed through the normalized L2 norm, and the value judgment matrix is ​​obtained by using the feature relationship matrix and dynamic gating weights. The specific steps are as follows: The normalized L2 norm is used to construct the feature relationship matrix, and the relationship between the corresponding process is: ; in, represents the feature relationship matrix, It means the normalized The L2 norm of the frame, represents an adjustable hyperparameter; The value judgment matrix is ​​obtained by using the feature relationship matrix and dynamic gating weights. The relationship between the corresponding process is: ; in, represents the value judgment matrix, represents the dynamic gating weight; Based on the length of the video sequence, the dynamic mask is obtained by calculation. The specific steps are as follows: The dynamic window size is calculated by the length of the video sequence, and the corresponding relationship is: ; in, Indicates the dynamic window size, Indicates the length of the video sequence; The start index and end index of the dynamic window are calculated based on the dynamic window size. The corresponding relationship is: ; in, Indicates the starting index of the dynamic window, The technical index representing the dynamic window, Indicates the maximum length of the current batch of video sequences, represents the time step; Generate a dynamic mask based on the start index and end index of the dynamic window. The corresponding process has the following relationship: ; in, represents a dynamic mask, represents the index of the time step in the video sequence; Using the query vector, key vector and value vector, combined with the value judgment matrix and dynamic mask, the final feature vector is obtained through attention calculation. The specific steps are as follows: Based on the query vector, key vector, dynamic mask and value judgment matrix, the attention weight of value perception and the attention weight of high value perception are obtained through attention calculation. The relationship between the corresponding process is: ; in, represents the attention score, represents the query vector, represents the transpose of the key vector, represents the dimension of the key vector, represents the value-aware attention weight, represents the attention weight of high value perception, represents element-wise multiplication, represents a dynamic mask, A supplementary matrix representing the original value information; Using the attention weight of value perception and the attention weight of high value perception, combined with the value vector, the final feature vector is obtained. The relationship between the corresponding process is: ; in, represents the feature output calculated by value-aware attention weights, represents the learnable parameters, represents the feature output calculated by high-value-aware attention weights, represents the fused features, represents a value vector, It means that after feature normalization operation, It means that after linear transformation operation, represents the final feature vector.

[0017] Furthermore, in order to eliminate the influence of the absolute value of the feature amplitude and normalize the feature value evaluation results to the [0,1] interval, the L2 norm is normalized; By calculating the difference in L2 norm between feature vectors and constructing a feature relationship matrix, the differences and uniqueness between features are captured.

[0018] Furthermore, in this step, the dynamic gating weight is a dynamic threshold used to adaptively screen high-value feature relationships. In the code implementation, the dynamic gating weight can be set to the mean plus the standard deviation of the feature relationship matrix.

[0019] Dynamic gating weights are used to adaptively define high-value standards based on the feature relationship matrix, and high-value feature relationships are preliminarily screened through comparison operations. Then, the learnable parameters can be used to control the Sigmoid gating to further gently adjust the strictness of value selection, so as to achieve dynamic and learnable reinforcement of high-value feature relationships, making the final output information more discriminative, laying the foundation for subsequent value-aware attention calculations.

[0020] Furthermore, in this step, the dynamic mask mechanism can dynamically adjust the size of the calculation window according to the length of the video sequence, thereby realizing dynamic calculation.

[0021] Furthermore, in order to ensure the most basic context information capturing capability, the present invention sets the minimum window of the dynamic window to 1.

[0022] Step 4: Input the final feature vector into the classification network to obtain the final anomaly score; The detection result is obtained through the final anomaly score; In step 4, the final feature vector is input into the classification network to obtain the final anomaly score. The relationship between the corresponding process is: ; in, represents the final anomaly score, , , and Both represent weight matrices, Indicates processing through the GELU activation function.

[0023] Furthermore, the classification network in this step adopts a four-layer fully connected structure, and each layer uses the GELU activation function and random dropout regularization to prevent overfitting; The network input is a 515-dimensional enhanced feature, which is compressed layer by layer in 512, 256, 128, and 64 dimensions to output the final anomaly score. Subsequently, the final anomaly score is normalized to the [0, 1] interval through the Sigmoid function.

[0024] To verify the effectiveness of the present invention (OURS), please refer to Figure 2 ,By comparing the performance data with the existing invention, it is shown that the ,presentation is superior to the existing methods in both detection accuracy ,and computational efficiency.

[0025] Among them, the dataset 1 (SH-AUC) used contains 437 videos of 13 types of scenes, covering general abnormal events, and the dataset 2 (UCF-AUC) used contains 1900 long videos, covering more complex abnormal types, to ensure the reliability of the model's robustness evaluation.

[0026] See also Figure 3 The embodiment of the present invention further provides a lightweight video anomaly detection system based on information gating compression, the system comprising: Preprocessing module for: Perform bilinear difference scaling on each frame of the static image sequence to obtain a scaled static image; The scaled static image is input into the CLIP-ViT model for semantic feature extraction to obtain a high-dimensional semantic feature vector. Gated weight module, used to: Use high-dimensional semantic features to calculate L2 norm, energy eigenvector and mean eigenvector; The L2 norm, energy feature vector, mean feature vector and high-dimensional semantic feature vector are concatenated to obtain an enhanced feature vector; The dynamic gating weights are obtained using the enhanced feature vectors; The final feature output module is used to: Normalize the L2 norm to obtain a normalized L2 norm; The feature relationship matrix is ​​constructed through the normalized L2 norm, and the value judgment matrix is ​​obtained using the feature relationship matrix and dynamic gating weights; Based on the length of the video sequence, a dynamic mask is obtained by calculation; Perform linear projection on the enhanced feature vector to obtain the query vector, key vector and value vector; Using the query vector, key vector and value vector, combined with the value judgment matrix and dynamic mask, the final feature vector is obtained through attention calculation; Result output module, used for: Input the final feature vector into the classification network to obtain the final anomaly score; The detection result is obtained through the final anomaly score.

[0027] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0028] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0029] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A lightweight video anomaly detection method based on information gating compression, characterized in that: The method comprises the following steps: Step 1, dividing the video in the video sequence into a static image sequence at a certain rate; Perform bilinear difference scaling on each frame of the static image sequence to obtain a scaled static image; The scaled static image is input into the CLIP-ViT model for semantic feature extraction to obtain a high-dimensional semantic feature vector. Step 2: Calculate the L2 norm, energy feature vector and mean feature vector using high-dimensional semantic features; The L2 norm, energy feature vector, mean feature vector and high-dimensional semantic feature vector are concatenated to obtain an enhanced feature vector; The dynamic gating weights are obtained using the enhanced feature vectors; Step 3, normalizing the L2 norm to obtain a normalized L2 norm; The feature relationship matrix is ​​constructed through the normalized L2 norm, and the value judgment matrix is ​​obtained using the feature relationship matrix and dynamic gating weights; Based on the length of the video sequence, a dynamic mask is obtained by calculation; Perform linear projection on the enhanced feature vector to obtain the query vector, key vector and value vector; Using the query vector, key vector and value vector, combined with the value judgment matrix and dynamic mask, the final feature vector is obtained through attention calculation; Step 4: Input the final feature vector into the classification network to obtain the final anomaly score; The detection result is obtained through the final anomaly score.

2. The lightweight video anomaly detection method based on information gating compression according to claim 1 is characterized in that: In step 1, the scaled static image is input into the CLIP-ViT model to perform semantic feature extraction to obtain a high-dimensional semantic feature vector. The corresponding process has the following relationship: ； in, Indicates a resizing operation. Indicates the total number of frames, The index representing the frame number. Indicates The high-dimensional semantic feature vector of the frame, Indicates A static image of a frame, Indicates that it has been processed by the CLIP-ViT model.

3. The lightweight video anomaly detection method based on information gating compression according to claim 2 is characterized in that: In step 2, the L2 norm, energy feature vector, mean feature vector and high-dimensional semantic feature vector are concatenated to obtain an enhanced feature vector. The relationship between the corresponding process is: ； in, Indicates The enhanced feature vector of the frame, Indicates The high-dimensional semantic feature vector of the frame dimensions, Represents a dimension index.

4. The lightweight video anomaly detection method based on information gating compression according to claim 3 is characterized in that: In step 2, the dynamic gating weight is obtained by using the enhanced feature vector, and the specific steps are as follows: The enhanced feature vector is used to calculate the exponentially decayed Manhattan distance to generate the inter-frame similarity matrix. The relationship between the corresponding process is: ； in, represents the inter-frame similarity matrix, The index representing the frame number. Indicates Enhanced feature vector of the frame; Using the inter-frame similarity matrix, the screening threshold is adjusted through learnable parameters to generate dynamic gating weights. The corresponding process has the following relationship: ； in, represents the dynamic gating weight, Indicates that it has been processed by the Sigmoid function. represents the learnable parameters, Indicates that it has been processed by expected value operation.

5. The lightweight video anomaly detection method based on information gating compression according to claim 4 is characterized in that: In step 3, the L2 norm is normalized to obtain the normalized L2 norm. The relationship between the corresponding process is: ； in, After normalization, The L2 norm of the frame, Indicates The L2 norm of the frame, represents the set of L2 norms of all eigenvectors, Represents a very small positive number.

6. The lightweight video anomaly detection method based on information gating compression according to claim 5 is characterized in that: In step 3, the feature relationship matrix is ​​constructed by the normalized L2 norm, and the value judgment matrix is ​​obtained by using the feature relationship matrix and the dynamic gating weight. The specific steps are as follows: The normalized L2 norm is used to construct the feature relationship matrix, and the relationship between the corresponding process is: ； in, represents the feature relationship matrix, It means the normalized The L2 norm of the frame, represents an adjustable hyperparameter; The value judgment matrix is ​​obtained by using the feature relationship matrix and dynamic gating weights. The relationship between the corresponding process is: ； in, represents the value judgment matrix, represents the dynamic gating weight.

7. The lightweight video anomaly detection method based on information gating compression according to claim 6 is characterized in that: In step 3, a dynamic mask is obtained by calculation based on the length of the video sequence. The specific steps are as follows: The dynamic window size is calculated by the length of the video sequence, and the corresponding relationship is: ； in, Indicates the dynamic window size, Indicates the length of the video sequence; The start index and end index of the dynamic window are calculated based on the dynamic window size. The corresponding relationship is: ； in, Indicates the starting index of the dynamic window, The technical index representing the dynamic window, Indicates the maximum length of the current batch of video sequences, represents the time step; Generate a dynamic mask based on the start index and end index of the dynamic window. The corresponding process has the following relationship: ； in, represents a dynamic mask, Represents the index of a time step in the video sequence.

8. The lightweight video anomaly detection method based on information gating compression according to claim 7 is characterized in that: In step 3, the query vector, key vector and value vector are used, combined with the value judgment matrix and the dynamic mask, to obtain the final feature vector through attention calculation. The specific steps are as follows: Based on the query vector, key vector, dynamic mask and value judgment matrix, the attention weight of value perception and the attention weight of high value perception are obtained through attention calculation. The relationship between the corresponding process is: ； in, represents the attention score, represents the query vector, represents the transpose of the key vector, represents the dimension of the key vector, represents the value-aware attention weight, represents the attention weight of high value perception, represents element-wise multiplication, represents a dynamic mask, A supplementary matrix representing the original value information; Using the attention weight of value perception and the attention weight of high value perception, combined with the value vector, the final feature vector is obtained. The relationship between the corresponding process is: ； in, represents the feature output calculated by value-aware attention weights, represents the learnable parameters, represents the feature output calculated by high-value-aware attention weights, represents the fused features, represents a value vector, It means that after feature normalization operation, It means that after linear transformation operation, represents the final feature vector.

9. The lightweight video anomaly detection method based on information gating compression according to claim 8 is characterized in that: In step 4, the final feature vector is input into the classification network to obtain the final anomaly score. The relationship between the corresponding process is: ； in, represents the final anomaly score, , , and Both represent weight matrices, Indicates processing through the GELU activation function.

10. A lightweight video anomaly detection system based on information gating compression, characterized in that: The system applies any one of claims 1 to 9 of the lightweight video anomaly detection method based on information gating compression, and the system comprises: Preprocessing module for: Perform bilinear difference scaling on each frame of the static image sequence to obtain a scaled static image; The scaled static image is input into the CLIP-ViT model for semantic feature extraction to obtain a high-dimensional semantic feature vector. Gated weight module, used to: Use high-dimensional semantic features to calculate L2 norm, energy eigenvector and mean eigenvector; The L2 norm, energy feature vector, mean feature vector and high-dimensional semantic feature vector are concatenated to obtain an enhanced feature vector; The dynamic gating weights are obtained using the enhanced feature vectors; The final feature output module is used to: Normalize the L2 norm to obtain a normalized L2 norm; The feature relationship matrix is ​​constructed through the normalized L2 norm, and the value judgment matrix is ​​obtained using the feature relationship matrix and dynamic gating weights; Based on the length of the video sequence, a dynamic mask is obtained by calculation; Perform linear projection on the enhanced feature vector to obtain the query vector, key vector and value vector; Using the query vector, key vector and value vector, combined with the value judgment matrix and dynamic mask, the final feature vector is obtained through attention calculation; Result output module, used for: Input the final feature vector into the classification network to obtain the final anomaly score; The detection result is obtained through the final anomaly score.

Images