Video Anomaly Detection Using Human-Scene Semantic Drift
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video anomaly detection methods struggle with false alarms due to focusing on global features, missing significant scene and human features, and failing to accurately detect subtle anomalies.
Innovation Solution
A method and system that separately extracts human-related and scene-related features, determines semantic drift between them, and combines these features into an augmented feature map using an encoder-decoder model and attention mechanisms to enhance anomaly detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If global snippet/frame-level features are used for anomaly detection, then the detection coverage is improved, but false alarms increase and significant features are missed
Solution Approach 1:
The video is segmented into human-related features and scene-related features, which are then processed separately through dedicated encoder-decoder models. This segmentation allows each feature type to be analyzed independently, preserving significant features that would otherwise be lost in global feature aggregation.
Solution Approach 2:
Semantic drift is introduced as an intermediary mechanism that bridges human-related features and scene-related features. By computing semantic drift between these separately extracted features, the system identifies anomalies caused by inconsistencies between human actions and scene context, thereby reducing false alarms while maintaining detection accuracy.
2Ease of manufacture
If single video-level binary labels are used for training, then the training process is simplified, but frame-level temporal boundaries for abnormal events cannot be accurately determined
Solution Approach 1:
The system performs preliminary extraction of human-related features and scene-related features separately before combining them. This preliminary action enables the subsequent computation of semantic drift at frame-level, which accurately determines temporal boundaries of abnormal events while maintaining the simplicity of video-level binary label training.
3Loss of information
If only global features are focused on, then the overall video understanding is improved, but subtle local anomalies are missed and irrelevant features dominate
Solution Approach 1:
The patent applies local quality by treating human-related features and scene-related features differently through separate encoder-decoder models. Each model is optimized for its specific feature type, enabling precise detection of subtle local anomalies in human actions while maintaining overall video understanding through the combination of both feature streams.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computer-implemented method for detecting an anomaly in a video, comprising: - obtaining human-related features (FH) and scene-related features (FS) extracted from the video; - determining a semantic drift (FD) between the human-related features (FH) and the scene-related features (FS); - selectively combining the human-related features (FH) and the scene-related features (FS), based on the semantic drift (FD), into an augmented feature map (F**); - detecting the anomaly based on the augmented feature map (F**).