Stereo Imagery Distance Estimation Using Interleaved Frames
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for determining motion and distance in robotic navigation using video imagery are inefficient and require significant computational resources, making them energy-intensive and costly, especially when dealing with complex motion and multiple objects in dynamic environments.
Innovation Solution
The use of a computer-readable storage medium with instructions for producing interleaved composite frames from images captured by spatially separated sensors, followed by motion estimation and disparity analysis to determine depth and motion information, leveraging hardware video encoders for efficient processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional motion estimation methods are used to determine depth and motion in robotic navigation, then measurement precision is improved, but use of energy and computational complexity increase significantly
Solution Approach 1:
The patent divides the video stream into separate left-eye and right-eye frame sequences, then selectively interleaves only specific frames from each sequence to create composite frames for disparity analysis. This segmentation approach allows the system to process only the necessary frame pairs for depth estimation rather than analyzing all frames, reducing computational load and energy consumption while maintaining measurement precision.
Solution Approach 2:
The system performs partial motion estimation by applying it only to selected composite frames rather than continuously to all video frames. The frame selector chooses specific frames based on motion characteristics and disparity requirements, enabling the system to achieve sufficient depth and motion measurement precision with reduced processing effort and lower energy consumption compared to full-frame continuous analysis.
2Productivity
If continuous motion estimation is applied to all video frames, then productivity in terms of motion detection coverage is improved, but use of energy and processing load increase
Solution Approach 1:
The patent implements periodic motion estimation by applying it only to selected composite frames rather than every frame. The frame selector determines which frames require motion estimation based on periodic sampling criteria and motion detection requirements, maintaining adequate motion detection coverage across the video stream while significantly reducing the frequency of computationally intensive motion estimation operations and associated energy consumption.
Solution Approach 2:
The system applies motion estimation partially to only those composite frames that meet specific selection criteria rather than to all frames. This partial action approach ensures adequate motion detection coverage for navigation purposes while avoiding redundant processing of frames that do not contribute significantly to motion understanding, thereby reducing processing energy consumption.
3Measurement precision
If full stereo processing is applied to all frames, then measurement precision for depth is improved, but device complexity increases
Solution Approach 1:
The patent segments the stereo video processing into distinct stages: frame selection from left and right sequences, composite frame generation, and disparity analysis. By separating these functions and applying them only when and where needed, the system achieves accurate depth measurement while managing device complexity through modular, selective processing rather than continuous full-stereo analysis of all frames.
Solution Approach 2:
The system applies full stereo processing partially to selected frames rather than to all frames. The frame selector identifies specific frames where disparity analysis will be most beneficial, and only those frames undergo the complex stereo processing pipeline. This approach maintains depth measurement accuracy for critical moments while reducing overall system complexity by avoiding unnecessary processing of other frames.
4Speed
If high frame rate processing is implemented for real-time motion detection, then speed of response is improved, but use of energy and computational resources increase
Solution Approach 1:
The patent implements periodic processing at strategically selected frames rather than at every frame interval. The frame selector and composite frame generator operate periodically on subsets of frames that are most informative for motion and depth detection, maintaining adequate real-time response speed for robotic navigation while significantly reducing the cumulative energy consumption compared to processing every frame at the same rate.
Solution Approach 2:
The system processes only a partial subset of frames at high rate, focusing computational resources on frames that provide the most critical motion and depth information. This partial high-rate processing maintains sufficient response speed for real-time navigation decisions while avoiding the excessive energy consumption that would result from processing all frames at the same high rate.
Data Source
AI summary
Frame sequences from multiple image sensors may be combined in order to form, for example, an interleaved frame sequence. Individual frames of the combined sequence may be configured a by combination (e.g., concatenation) of frames from one or more source sequences. The interleaved/concatenated frame sequence may be encoded using a motion estimation encoder. Output of the video encoder may be processed (e.g., parsed) in order to extract motion information present in the encoded video. The motion information may be utilized in order to determine a depth of visual scene, such as by using binocular disparity between two or more images by an adaptive controller in order to detect one or more objects salient to a given task. In one variant, depth information is utilized during control and operation of mobile robotic devices.