Behavioral Recognition System Using Semantic Object Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video surveillance systems require pre-defined knowledge of behaviors to recognize abnormal activities, are labor-intensive, and costly, as they need separate software for distinct behaviors and cannot accurately characterize changes in normal scenes.
Innovation Solution
A method and system that analyze video frames to learn normal and abnormal behaviors in real-time by generating semantic representations of object movements, allowing the system to identify and predict abnormal behaviors without pre-defined codes, using a machine learning engine to classify and track objects within a scene.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If pre-defined behavior codes are used to recognize abnormal activities, then the system can identify specific behaviors, but the system becomes labor-intensive and costly requiring separate software for each behavior
Solution Approach 1:
The patent applies universality by creating a single surveillance system that can recognize multiple distinct behaviors through one unified software platform. The system uses a behavior database that stores various behavior patterns (lurking, non-swimming, loitering, etc.) and can adapt to different monitoring scenarios without requiring separate software products for each behavior type.
Solution Approach 2:
The system changes parameters by using configurable behavior definitions that can be adjusted through a user interface. Instead of hard-coded behavior recognition, the system allows modification of behavior parameters such as time thresholds, movement patterns, and zone definitions, enabling flexible adaptation to different monitoring requirements without reprogramming the entire system.
2Reliability
If pre-programmed change detection is used to monitor normal scenes, then the system can generate alarms for abnormalities, but the system cannot accurately characterize what has actually occurred
Solution Approach 1:
The system implements feedback by continuously comparing current scene data against learned normal patterns and providing detailed behavior characterization in addition to alarm generation. When an anomaly is detected, the system doesn't just alert operators but also provides specific information about the abnormal behavior type, allowing for more informed decision-making and reducing information loss.
Solution Approach 2:
The system performs preliminary action by pre-learning normal behavior patterns through machine learning algorithms before actual monitoring begins. This allows the system to establish a baseline of normal activities and quickly identify deviations, enabling both reliable alarm generation and accurate behavior characterization from the outset.
3Measurement precision
If distinct software products are developed for each behavior type, then each behavior can be monitored accurately, but the development becomes prohibitively costly and labor-intensive
Solution Approach 1:
The patent applies segmentation by dividing the behavior recognition system into modular components: a behavior database storing individual behavior patterns, a pattern matching engine, and a configurable interface. This modular architecture allows the system to maintain high monitoring accuracy for specific behaviors while simplifying development, as new behavior types can be added by simply adding new pattern definitions rather than developing entirely new software products.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Embodiments of the present invention provide a method and a system for analyzing and learning behavior based on an acquired stream of video frames. Objects depicted in the stream are determined based on an analysis of the video frames. Each object may have a corresponding search model used to track an object's motion frame-to-frame. Classes of the objects are determined and semantic representations of the objects are generated. The semantic representations are used to determine objects' behaviors and to learn about behaviors occurring in an environment depicted by the acquired video streams. This way, the system learns rapidly and in real-time normal and abnormal behaviors for any environment by analyzing movements or activities or absence of such in the environment and identifies and predicts abnormal and suspicious behavior based on what has been learned.