Video Object Detection Model Training with Pseudo Labels
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge in training video object detection models is the high cost and difficulty in collecting diverse backgrounds for training datasets, which limits the robustness of video object detection due to issues like motion blur, video defocus, and rare poses, making it hard to achieve accurate object detection in videos.
Innovation Solution
The method involves using the video object detection model as a pseudo labeler to generate pseudo temporal information, allowing the model to train using both image and video data, and employing prior information such as masks and bounding box vectors to refine object detection and minimize loss, thereby improving detection accuracy without the need for extensive diverse background data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If many videos are collected to create a training dataset with various backgrounds, then the diversity of backgrounds is improved, but the cost and time required for data collection increase significantly
Solution Approach 1:
The patent uses video generation models to create synthetic video clips that copy and transform background patterns from existing videos. These generated backgrounds are then used to train the object detection model, replacing the need to collect numerous real videos for background diversity.
Solution Approach 2:
The system uses its own generated data to train itself. The video generation model creates training data from existing videos, and this self-generated data is used to improve the object detection model, creating a self-improving loop without external data collection.
2Adaptability or versatility
If many videos are collected to create a training dataset with various backgrounds, then the diversity of backgrounds is improved, but the cost of generating the training dataset increases
Solution Approach 1:
Instead of collecting and processing numerous real videos, the patent copies background patterns from a small set of existing videos and generates new background variations computationally, significantly reducing data collection and processing costs.
Solution Approach 2:
The system generates its own training data using video generation models, eliminating the need for expensive manual data collection and annotation processes typically required for diverse background datasets.
3Measurement precision
If temporal information from previous frames is used to detect objects in current frames, then detection accuracy is improved, but the complexity of the detection system increases
Solution Approach 1:
The patent pre-generates background information and temporal context data before object detection. By preparing background models and temporal patterns in advance, the system reduces the computational complexity during actual detection while maintaining high accuracy through the use of pre-processed temporal information.
Data Source
AI summary
A method of training a video object detection model by using a training dataset is provided, including steps of: a learning device (a) after acquiring a training image (i) inputting the training image and first prior information, set as having probabilities of objects existing in locations in the training image, into the video object detection model, to thereby detect the objects and thus output first object detection information, and (ii) generating second prior information, which includes location information of the objects on the training image; (b) inputting the training image and the second prior information into the video object detection model, to detect the objects on the training and thus output second object detection information; and (c) generating a loss by referring to the second object detection information and a ground truth corresponding to the training image, and train the video object detection model to minimize the loss.