3D Surveillance Video Augmentation for AI Detection Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for training AI models for object detection are inefficient and costly, often requiring manual labeling of large image datasets or creating virtual environments that lack real-world context, leading to less accurate training.
Innovation Solution
A method and system for augmenting surveillance video footage with 3D objects by automatically positioning and obscuring them behind distractors, using real-world video and computer vision techniques to generate labeled training images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If real-world surveillance footage is used for training AI models, then the model learns from authentic data, but the quantity and variety of training data are insufficient
Solution Approach 1:
The patent creates synthetic 3D surveillance footage by rendering virtual target objects into real-world surveillance video frames. This copying approach generates additional training data with controlled variability while maintaining authentic background characteristics, thereby increasing both the quantity and diversity of training datasets without requiring extensive real-world data collection
Solution Approach 2:
The patent transitions from 2D video frames to 3D virtual object rendering and back to 2D augmented footage. By introducing the third dimension through virtual 3D models and projecting them into the 2D surveillance space with varying parameters (position, size, orientation, occlusion), the system expands the training data variety while maintaining compatibility with existing surveillance systems
2Adaptability or versatility
If synthetic 3D target objects are rendered into surveillance footage, then training data variety increases, but the complexity of the system increases
Solution Approach 1:
The patent employs a universal rendering pipeline that can handle multiple target object types, surveillance camera configurations, and environmental conditions through a single system architecture. The same 3D rendering engine processes diverse object categories and integrates with various surveillance video formats, reducing overall system complexity despite the enhanced versatility
Solution Approach 2:
The patent introduces a virtual camera as an intermediary between the 3D virtual environment and the 2D surveillance footage. This virtual camera mediates the projection of 3D objects into 2D space, handling complex transformations (perspective, distortion, resolution matching) in a unified manner that simplifies the overall integration process
3Reliability
If virtual target objects are integrated into real surveillance video, then realistic training scenarios are created, but the computational resources required increase
Solution Approach 1:
The patent applies partial rendering by only processing regions of the surveillance footage where target objects are present or likely to be present. Rather than rendering the entire video frame at full resolution, the system focuses computational resources on generating and integrating target objects, reducing overall computational load while maintaining training scenario realism
Solution Approach 2:
The patent implements batch processing of surveillance footage, where virtual target objects are integrated at specific intervals or for specific time periods during training. This periodic approach allows computational resources to be allocated in manageable cycles rather than continuously, reducing peak resource consumption while still achieving comprehensive training coverage
Data Source
Figure 1
Figure 2A~2B
Figure 3
AI summary
Disclosed is a method for augmenting a video footage of a surveillance space with a target three-dimensional (3D) object from one or more perspectives for training an artificial intelligence (AI) model, comprising: acquiring the video footage from a target camera in the surveillance space; determining a ground plane and screen coordinates of corners of the ground plane; normalizing screen coordinates from the ground plane and determining a relative position of each object in the ground plane; preparing a model of the target 3D object to be used for training the AI model; iteratively generating a random position and a random rotation for the target 3D object in the ground plane for positioning the target 3D object in front of or behind a distractor object from among the objects in the ground plane; rendering the model of the target 3D object on the ground plane and composing the rendered 3D object and the ground plane with the acquired video footage to generate a composited image; and calculating coordinates of a bounding box that frames the relative position of the target 3D object in the composited image.