Agent Trajectory Estimation Using Monocular-Stereo Image Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for deriving agent trajectories from monocular camera images face challenges due to the lack of depth information and occlusion, leading to inaccurate and unreliable trajectory estimates, particularly for agents farther away or partially occluded.
Innovation Solution
A method involving a monocular camera to identify a tracking point at the lowest-observed pixel within the image, combined with a motion model to account for depth and correct for occlusions, and a stereo camera system to fuse position information from both types of images for enhanced accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If a monocular camera is used to capture images for trajectory estimation, then the system cost is reduced and availability is improved, but depth information is lost and measurement precision deteriorates
Solution Approach 1:
The patent transforms 2D image coordinates into 3D world coordinates by introducing depth estimation through motion models. The system uses temporal sequences of 2D images and applies motion constraints to infer the third dimension (depth), effectively converting a 2D measurement problem into a 3D solution without requiring 3D sensors.
Solution Approach 2:
The system changes the parameter representation from direct depth measurement to motion-based depth inference. By utilizing temporal changes in 2D image coordinates and applying motion model constraints, the system derives depth information indirectly through parameter transformations rather than direct measurement.
2Device complexity
If standard image processing is used without occlusion handling, then the processing complexity is reduced, but trajectory reliability deteriorates due to occlusions
Solution Approach 1:
The system implements feedback mechanisms where the motion model continuously refines trajectory estimates by comparing predicted positions with actual detections. When occlusions cause detection failures, the feedback loop uses previous reliable positions and motion constraints to infer positions during occlusion periods, then validates these inferences when the agent reappears.
Solution Approach 2:
The system prepares for potential occlusions by maintaining motion models and trajectory predictions in advance. When occlusions occur, the pre-established motion constraints and predicted trajectories serve as cushioning that prevents complete trajectory failure, allowing the system to bridge gaps caused by occlusions smoothly.
3Measurement precision
If motion models are applied to correct for occlusions and estimate depth, then trajectory accuracy is improved, but computational complexity increases
Solution Approach 1:
The system applies motion models selectively rather than to all trajectory points. Full motion model computation is performed only when necessary (e.g., during occlusions or for depth estimation), while other points use simpler interpolation or direct detection results, reducing overall computational burden while maintaining accuracy where critical.
Data Source
AI summary
Examples disclosed herein may involve a computing system that is operable to (i) derive, from a first set of one or more images captured by a monocular camera associated with a vehicle, a first set of position information for a given agent; (ii) derive, from a second set of one or more image pairs captured by a stereo camera associated with the vehicle, a second set of position information for the given agent; (iii) input the first and second sets of position information for the given agent into a motion model that encodes knowledge regarding physical constraints on the given agent's real-world behavior; and (iv) determine a trajectory for the given agent based on an output of the motion model.