Neural Image Sequence Trajectories for Occlusion-Aware Visual Odometry
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computing accurate image sequence trajectories for long-term tracking, especially in environments with moving objects and occlusions, is not straightforward in existing methods, leading to inefficiencies and inaccuracies.
Innovation Solution
A neural network-based approach that uses a feature extraction neural network to adaptively aggregate spatial information, incorporates deformable dilated convolutions, and employs a probabilistic formulation to predict trajectories with confidence data, handling uncertainties and ambiguities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional methods are used for long-term trajectory tracking, then computational complexity is reduced, but tracking accuracy deteriorates in environments with moving objects and occlusions
Solution Approach 1:
The trajectory recognition task is segmented into multiple independent trajectory hypotheses, each representing a possible path. The neural network processes multiple hypotheses in parallel, allowing the system to handle complex tracking scenarios by breaking down the overall problem into manageable segments that can be evaluated independently and then combined.
Solution Approach 2:
The system generates excessive trajectory hypotheses beyond what is strictly necessary, then uses confidence data to filter and select the most likely trajectories. This approach of generating more candidates than needed allows the system to maintain high accuracy by having multiple options to choose from, even though it increases computational workload.
2Reliability
If confidence data is provided for each trajectory, then downstream process performance is improved, but neural network computational load increases
Solution Approach 1:
The neural network computes confidence data for all trajectory hypotheses in advance, before the downstream process needs to make decisions. This preliminary computation of confidence scores allows downstream applications to efficiently select the best trajectories without requiring additional computational analysis, thereby improving overall system reliability while managing computational load through advance preparation.
3Measurement precision
If a neural network is used for trajectory recognition, then tracking accuracy is improved, but processing time increases
Solution Approach 1:
The neural network processes multiple trajectory hypotheses in parallel segments rather than sequentially evaluating all possibilities. This segmentation of the computation into independent parallel paths significantly reduces processing time while maintaining the high accuracy benefits of neural network-based trajectory recognition.
Data Source
AI summary
Images captured by a camera moving in an environment are received, and for each of a plurality of points in one of the images, outputs are computed using a neural network. The outputs comprise: a trajectory depicting the point in each of the plurality of images, as well as, for each trajectory, a prediction of visibility of the trajectory in each of the images and a prediction of whether the trajectory depicts a static or moving surface in the environment. The neural network receives the images and points as input and computes the outputs, wherein the outputs comprise for each of the trajectories, confidence data. The outputs are sent to a downstream process selected from any of: visual odometry, structure from motion, human body tracking, video editing, vehicle tracking.