Frame Object Matching for Long-Range Motion Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current state-of-the-art DNN-based motion estimation methods struggle with challenging cases where objects undergo severe appearance variation and small objects move a long distance between frames, falling outside the spatial range of these methods.
Innovation Solution
A method involving instance segmentation, feature vector clustering, and refinement using machine learning algorithms to match objects between frames, followed by estimating translational and non-translational motion vectors, enhances motion estimation accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If DNN-based motion estimation methods are used, then computational efficiency and accuracy are improved for typical cases, but performance deteriorates for challenging cases where objects undergo severe appearance variation and small objects move long distances
Solution Approach 1:
The patent segments the motion estimation problem into two distinct parts: (1) DNN-based estimation for typical cases, and (2) instance segmentation-based matching for challenging cases. By dividing the problem space, the system can apply the most appropriate method for each case type, thereby maintaining high accuracy while improving reliability across diverse scenarios.
Solution Approach 2:
The patent performs instance segmentation and object matching as a preliminary action before final motion estimation. By pre-identifying and matching object instances between frames using robust feature extraction and clustering, the system establishes reliable correspondences that remain valid even when objects undergo severe appearance changes, thus improving reliability before the final motion calculation.
2Reliability
If instance segmentation and clustering methods are used, then reliability for challenging cases is improved, but device complexity increases
Solution Approach 1:
The patent applies local quality by using instance segmentation and clustering only in regions and cases where DNN-based methods fail (challenging cases with severe appearance variation). Rather than applying complex methods globally, the system selectively activates robust matching only where needed, thereby improving reliability without unnecessarily increasing overall system complexity.
Solution Approach 2:
The patent creates a universal motion estimation system that can handle both typical and challenging cases through a unified framework. The system integrates multiple approaches (DNN-based estimation and instance segmentation-based matching) into a single multi-functional platform that automatically selects the appropriate method based on input characteristics, thereby managing complexity while maintaining broad applicability.
3Adaptability or versatility
If small objects move long distances between frames, then the spatial range of DNN-based methods is exceeded, but motion estimation is still required
Solution Approach 1:
The patent introduces instance segmentation and object matching as an intermediary process between frame capture and final motion estimation. This intermediary step extracts robust object features and establishes correspondences even for small objects moving long distances, providing reliable input to the motion estimation algorithm and thereby maintaining measurement precision beyond the normal spatial range of DNN-based methods.
Solution Approach 2:
The patent transitions from direct pixel-based spatial comparison to feature-based representation in a different dimensional space. By extracting object features and performing matching in feature space rather than direct spatial coordinates, the system can accurately track small objects undergoing large displacements, effectively extending the measurable spatial range while maintaining precision.
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
A method of analysing objects in a first frame and a second frame is disclosed. The method includes segmenting the frames, and matching at least one object in the first frame with a corresponding object in the second frame. The method optionally includes estimating the motion of the at least one matched object between the frames. Also disclosed is a method of generating a training dataset suitable for training machine learning algorithms to estimate the motion of objects. Also provided are processing systems configured to carry out these methods.