Video Training Data Sampling via Detection-Tracking Inconsistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for sampling training data from videos for improving neural network performance often select well-processed images, which do not effectively enhance the network's performance when retrained, as they fail to identify and address suboptimal processing issues such as missed or false detections.
Innovation Solution
A method and apparatus that input a video to a target detection model to obtain detection results, match these with tracking results using a bipartite graph matching algorithm, and determine inconsistent frames as sample images for annotation, which are then refined using a temporal non-maximum suppression algorithm to improve the target detection model's performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If existing sampling methods select images from videos for annotation, then the training set can be expanded, but the selected images are often well-processed by the neural network and do not effectively improve performance
Solution Approach 1:
The patent applies local quality by selecting specific frames with suboptimal detection results for annotation rather than uniformly sampling all frames. The determination unit identifies frames where detection results are inconsistent with tracking results, focusing annotation efforts on locally problematic areas that will most improve model performance.
Solution Approach 2:
Instead of selecting well-processed frames as traditional methods do, the patent inverts the selection criterion to choose frames with poor detection results. The determination unit specifically identifies frames where detection and tracking results differ, selecting these suboptimal cases for annotation to maximize performance improvement.
2Quantity of substance
If all frames from videos are selected for annotation, then comprehensive training data is obtained, but the workload and time consumption increase significantly
Solution Approach 1:
The patent extracts only the necessary subset of frames for annotation by using the determination unit to identify frames with detection-tracking inconsistencies. This extraction principle selects only those frames that require annotation, discarding frames with consistent results that do not need further processing.
Solution Approach 2:
The patent applies partial action by annotating only a portion of frames that are identified as having suboptimal detection results. Rather than annotating all frames excessively, the method selectively processes only the necessary subset, reducing overall annotation workload while maintaining effectiveness.
3Productivity
If simple sampling methods are used to select training images, then the process is fast and simple, but the selected images do not target suboptimal processing cases
Solution Approach 1:
The patent implements feedback by using the determination unit to compare detection results with tracking results and identify inconsistencies. This feedback mechanism guides the selection of frames for annotation, ensuring that only frames with actual detection quality issues are selected, thereby improving both precision and efficiency.
Solution Approach 2:
The determination unit acts as an intermediary between the detection model and the annotation process. It mediates by evaluating detection results against tracking results and selectively passing only problematic frames to the annotation stage, improving sampling precision without significantly increasing overall process complexity.
Data Source
AI summary
The present disclosure provides a method and an apparatus for sampling training data and a computer server. The method includes: inputting a video to a target detection model to obtain a detection result for each frame of image; inputting the detection results for all frames of images in the video to a target tracking model, to obtain a tracking result for each frame of image; and for each frame of image in the video: matching the detection result and the tracking result for the frame of image, and when the detection result and the tracking result for the frame of image are inconsistent with each other, determining the frame of image as a sample image to be marked, for which processing by the target detection model is not optimal.