Abnormal Object Detection via Spatio-Temporal Reconstruction Errors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing number of CCTV installations leads to a significant increase in manpower required for monitoring and analyzing video images, making it impractical to accurately analyze all images for abnormal object detection, as existing methods lack the necessary accuracy and efficiency.
Innovation Solution
An apparatus and method utilizing an inpainting technique with auto-encoder and LSTM models to generate and combine reconstructed frames, calculating spatial and time domain reconstruction errors to detect abnormal objects by applying inpainting patterns to video frames, thereby increasing detection accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual monitoring and analysis of CCTV images is performed, then detection accuracy can be maintained, but the required manpower increases significantly and becomes impractical
Solution Approach 1:
The patent replaces manual mechanical analysis of CCTV images with an automated computer-based system that uses deep learning models (auto-encoder and LSTM) to detect abnormal objects. The system automatically processes video frames, generates reconstructed frames, calculates reconstruction errors, and identifies abnormal objects without human intervention, thereby maintaining detection accuracy while eliminating the need for substantial manpower.
2Productivity
If automated abnormal object detection is implemented, then productivity increases, but detection accuracy is insufficient with existing methods
Solution Approach 1:
The patent segments the abnormal object detection task into distinct processing stages: generating reconstructed frames through auto-encoder, predicting temporal features using LSTM, calculating spatial and temporal reconstruction errors separately, and fusing these errors to identify abnormal objects. This segmentation allows each component to be optimized independently, achieving both high automation and high detection accuracy.
Solution Approach 2:
The patent introduces a temporal dimension by using LSTM to predict features from sequential video frames and calculating temporal reconstruction errors. This transforms the detection from a single-frame spatial analysis to a multi-frame spatio-temporal analysis, significantly improving detection accuracy while maintaining full automation.
3Speed
If simple detection methods are used, then processing speed is fast, but detection accuracy is insufficient for abnormal objects
Solution Approach 1:
The patent performs preliminary actions by pre-training the auto-encoder model on normal video frames before actual detection. The model learns to reconstruct normal patterns, and during detection, any significant reconstruction error indicates abnormal objects. This preliminary training enables fast real-time detection without sacrificing accuracy, as the model is already optimized for the specific application.
Data Source
AI summary
Disclosed are a method and an apparatus for detecting abnormal objects in a video. The method for detecting abnormal objects in a video reconstructs a restored batch by applying each input batch to which an inpainting pattern is applied to a trained auto-encoder model, and fuses a time domain reconstruction error using time domain restored frames output by extracting and restoring a time domain feature point by applying a spatial domain reconstruction error and a plurality of successive frames using a restored frame output by combining the reconstructed restoring batch to a trained LSTM auto-encoder model to estimate an area where an abnormal object is positioned.