Object Motion Estimation Using Feature Vectors and Instance Masks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current state-of-the-art DNN-based motion estimation methods struggle with challenging cases where objects undergo severe appearance variation and small objects move a long distance between frames, often failing to accurately estimate motion.
Innovation Solution
A method involving instance segmentation, feature vector extraction, and matching of objects between frames, followed by a neural network-based refinement of motion estimates, using a modified Mask R-CNN algorithm to generate object-level representations and improve motion estimation accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If classical motion estimation methods are used, then computational efficiency is maintained, but accuracy deteriorates for objects with severe appearance variation and small objects moving long distances
Solution Approach 1:
The patent segments the image into multiple regions of interest (ROIs) and processes each ROI separately through instance segmentation to identify individual objects. This segmentation allows the system to focus computational resources on specific objects, improving motion estimation accuracy for small objects while managing overall complexity through localized processing.
Solution Approach 2:
The patent extracts feature vectors that capture appearance information in an additional dimensional space beyond simple pixel coordinates. By representing objects in this enhanced feature space and matching them across frames, the system achieves higher accuracy for objects with severe appearance variation without proportionally increasing computational complexity.
2Measurement precision
If instance segmentation and feature vector matching are implemented, then motion estimation accuracy for small objects improves, but computational cost increases
Solution Approach 1:
The patent applies instance segmentation and feature extraction only to regions of interest containing detected objects, rather than processing the entire image uniformly. This local quality approach concentrates computational energy where it is most needed (on small objects requiring accurate motion estimation) while reducing overall computational burden by excluding background regions.
Solution Approach 2:
The patent performs instance segmentation and feature vector extraction as preliminary steps before motion estimation. By pre-processing and identifying objects of interest in advance, the system avoids redundant computations during the actual motion estimation phase, thereby reducing total computational energy while maintaining high accuracy for small objects.
3Reliability
If object-level representations are used, then handling of complex deformations improves, but processing time increases
Solution Approach 1:
The patent segments objects into instance-level representations that capture their unique characteristics and deformations independently. By processing each object instance separately through masking and feature extraction, the system can accurately handle complex deformations without being overwhelmed by the complexity of the entire scene, thus managing processing time more effectively.
Data Source
AI summary
A method of analyzing one or more objects in a set of frames. A first frame is segmented to produce a plurality of first masks each identifying pixels belonging to a potential object-instance detected in the first frame. A first feature vector is extracted from the first frame for each potential object-instance detected therein, characterizing the potential object-instance. A second frame is segmented to produce a plurality of second masks each identifying pixels belonging to a potential object-instance detected in the second frame. A second feature vector is extracted for each potential object-instance detected in the second frame, characterizing the potential object-instance. A potential object-instance in the first frame is matched with one of the potential object-instances in the second frame.