Learned Stereo Architecture for Flexible Synthetic-Data Disparity Estimation

The patent uses synthetic image data that copies the appearance and structure of real-world stereo images but is generated computationally rather than captured physically. This synthetic data is then used to train the neural network, eliminating the need for time-consuming real-world data collection and labeling while maintaining the ability to handle diverse camera systems through virtual representation.

The patent employs fully differentiable 3D convolutions that allow continuous parameter adjustments in the neural network architecture. This enables the model to adapt to different baseline distances, camera configurations, and depth ranges by modifying network parameters rather than requiring retraining, thus achieving flexibility across various stereo camera systems.

The patent divides the stereo depth estimation task into multiple processing stages: feature extraction using 2D convolutions, cost volume computation using 3D convolutions, and disparity estimation. This segmentation allows each component to be optimized independently and enables progressive training strategies where simpler components are trained first, reducing overall computational burden while maintaining training efficiency.

The patent introduces a cost volume dimension that combines spatial information from both images with depth information, creating a 4D tensor structure. This additional dimension allows the 3D convolutions to process depth relationships more efficiently, improving training productivity while the structured approach to complexity management keeps the architecture manageable.

The patent applies partial action by processing only the most relevant portions of the image data through the computationally intensive 3D convolution stages. The feature extraction stage uses lighter 2D convolutions to identify salient features, and then only regions with high feature density or semantic importance are subjected to the more resource-intensive 3D processing, thereby reducing overall computational resource consumption while maintaining high accuracy.

The patent implements continuous refinement where each processing stage builds upon the output of the previous stage in a sequential manner. The feature maps from the first stage continuously feed into the cost volume computation, which then feeds into the disparity estimation, creating a continuous flow of useful computational action that maximizes accuracy while minimizing redundant calculations.