Video Frame Labeling via Feature Matching Guide Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video label information propagation methods suffer from poor accuracy and accumulation of propagation errors due to object occlusion and rapid movement, where adjacent video frames are used as guide frames, leading to incorrect label information for subsequent frames.
Innovation Solution
A method that performs feature extraction on a target video frame to determine a guide video frame with a high image feature matching degree from labelled video frames, rather than relying on adjacent frames, to generate accurate label information, thereby improving selection quality and preventing propagation errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If adjacent video frames are used as guide video frames for label information propagation, then the processing simplicity is improved, but the label information accuracy deteriorates due to object occlusion and rapid movement causing propagation errors to accumulate
Solution Approach 1:
The patent introduces an image feature matching mechanism as an intermediary between the target video frame and candidate guide video frames. Instead of directly using adjacent frames, the system calculates matching degrees based on image features (such as optical flow, feature points, or deep learning features) to select the most appropriate guide frame, thereby mediating the trade-off between processing simplicity and accuracy
Solution Approach 2:
The patent changes the selection criterion from temporal adjacency (fixed parameter) to image feature matching degree (variable parameter). By dynamically selecting guide video frames based on matching degrees calculated from image features, the system adapts to different scenarios (occlusion, rapid movement) while maintaining reasonable processing complexity
2Device complexity
If a fixed adjacent video frame is used as the guide video frame, then the device complexity is reduced, but the adaptability to different video content and motion patterns deteriorates
Solution Approach 1:
The patent transforms the static guide frame selection (fixed adjacent frame) into a dynamic selection process where the guide video frame is determined based on image feature matching degrees. This allows the system to adapt to different motion patterns, occlusion levels, and video content while maintaining manageable device complexity through efficient feature matching algorithms
3Productivity
If label information is propagated sequentially from adjacent frames, then the computational efficiency is improved, but the reliability of label information deteriorates due to continuous accumulation of propagation errors
Solution Approach 1:
The patent introduces image feature matching degree calculation as an intermediary verification step between sequential frames. This mediator mechanism allows the system to maintain sequential processing efficiency while periodically verifying and correcting label information accuracy by selecting guide frames with high matching degrees, thus preventing error accumulation
Solution Approach 2:
The system implements a feedback mechanism where the image feature matching degree serves as a quality metric. When the matching degree between target frame and guide frame is high, label propagation is reliable; when it's low, the system can switch to alternative strategies (such as using earlier frames with better matching or re-initialization), thereby maintaining reliability without sacrificing overall computational efficiency
Data Source
AI summary
A method and apparatus for labelling information of a video frame, includes: obtaining a video; performing feature extraction on a target video frame in the video, to obtain a target image feature of the target video frame; determining, according to image feature matching degrees between the target video frame and labelled video frames, a guide video frame of the target video frame from the labelled video frames, the guide video frame being used for guiding the target video frame for information labelling, and the image feature matching degrees being matching degrees between the target image feature and image features corresponding to the labelled video frames; and generating target label information corresponding to the target video frame according to label information corresponding to the guide video frame.