Patch-Wise VQ Autoencoder Codebook for Video Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Intelligent CCTV systems face challenges in accurately detecting anomalies due to imbalanced data samples and low-resolution anomalies, leading to degraded performance and biased results.
Innovation Solution
A Video-based Patch-Wise Vector-Quantized Auto-Encoder (VBPVQAE) system that learns a codebook from normal video data, using vector quantization to identify anomalies by encoding and decoding video patches, incorporating convolutional neural networks and multi-head attention mechanisms to capture local and global patterns.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If deep learning is trained on imbalanced data with few anomaly samples, then the model can be trained with limited anomaly data, but the detection performance degrades and results become biased toward normal samples
Solution Approach 1:
The patent inverts the traditional anomaly detection approach by training the model exclusively on normal samples and using reconstruction error to identify anomalies. Instead of teaching the model what anomalies look like, it teaches the model what normal behavior looks like and detects deviations from this learned normal pattern, thereby solving the imbalanced data problem.
Solution Approach 2:
The patent introduces a reconstruction model as an intermediary between the input data and anomaly detection. The model learns to reconstruct normal patterns, and the reconstruction error serves as a mediator to indicate anomaly presence, allowing detection without direct anomaly examples.
2Area of stationary object
If the entire CCTV frame is processed for anomaly detection, then global context is captured, but small local anomalies are missed due to low resolution
Solution Approach 1:
The patent segments the CCTV frame into multiple patches and processes each patch independently through the trained model. This segmentation allows the model to focus computational resources on local regions, improving the detection of small anomalies while maintaining overall frame coverage through systematic patch processing.
Solution Approach 2:
The patent transitions from processing the entire 2D frame at once to processing multiple smaller 2D patches, effectively adding a spatial resolution dimension. This dimensional change allows the model to achieve higher effective resolution for detecting small anomalies while maintaining comprehensive coverage.
3Reliability
If a comprehensive model is trained to handle various anomaly types and resolutions, then detection accuracy improves, but model complexity and tuning requirements increase
Solution Approach 1:
The patent creates a universal model that handles various anomaly types and resolutions through a single reconstruction framework. The model learns general normal patterns from diverse training data and applies them universally across different anomaly scenarios, eliminating the need for multiple specialized models or extensive dataset-specific tuning.
Solution Approach 2:
The patent achieves adaptability to different anomaly types and resolutions by changing input parameters (patch size, processing resolution) rather than changing the model architecture. This allows a single model to handle diverse anomaly detection tasks through parameter adjustment rather than structural modification.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
According to some embodiments, a system includes: a memory, an encoder; a decoder, wherein the system is arranged to: receive, at the encoder, an input video; divide, by the encoder, the input video into a plurality of video patches; select, by the encoder, codes corresponding to the plurality of video patches of the input video, from a codebook comprising the codes; determine, by the encoder, an assigned code matrix comprising the codes corresponding to the plurality of video patches of the input video; receive, by the decoder, the assigned code matrix from the encoder; and generate, by the decoder, a reconstructed video based on the assigned code matrix.