Stereo Matching Using 2D Convolution for Mobile Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing stereo matching networks face challenges in balancing accuracy and computational resource efficiency, particularly in mobile environments, due to the resource-intensive processing of 4D cost volumes using 3D convolution.
Innovation Solution
The proposed method employs a stereo matching system that generates cost volumes for partial disparities using 2D convolution, with a sequential feature fusion module that processes feature maps through a series of spatial pyramid pooling and CNN blocks, refining the disparity maps to improve accuracy and reduce computational waste.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 3D convolution is used to process 4D cost volumes, then stereo matching accuracy is improved, but computational resource consumption increases
Solution Approach 1:
The patent divides the 4D cost volume processing into multiple 2D cost volume slices at different disparity levels. Each slice is processed independently using 2D convolution, avoiding the need for computationally intensive 3D convolution while maintaining matching accuracy through sequential processing of disparity-specific feature maps.
Solution Approach 2:
The patent transforms the 4D cost volume processing problem into a series of 2D processing tasks by slicing along the disparity dimension. This dimensional reduction allows the use of efficient 2D convolution operations instead of 3D convolution, significantly reducing computational resource requirements while preserving the ability to handle multiple disparity levels.
2Measurement precision
If 3D convolution is used to process 4D cost volumes, then stereo matching accuracy is improved, but device complexity increases
Solution Approach 1:
The patent segments the complex 4D cost volume processing into multiple independent 2D processing stages, each handling a specific disparity level. This segmentation reduces the computational complexity of each processing unit, allowing implementation on devices with limited hardware capabilities while maintaining overall matching accuracy through the aggregation of results across all disparity slices.
Solution Approach 2:
By reducing the processing dimension from 3D to 2D through slicing the cost volume along the disparity dimension, the patent decreases the computational complexity and hardware requirements. The 2D convolution operations are less demanding on computing hardware compared to 3D convolution, enabling deployment on mobile and embedded devices.
3Measurement precision
If sequential feature fusion with multiple SFF modules is used, then processing accuracy is improved, but processing time increases
Solution Approach 1:
The patent divides the disparity processing range into multiple partial disparities, each handled by a separate SFF module. This segmentation allows parallel processing of different disparity ranges, and the sequential arrangement of modules enables progressive refinement without requiring iterative adjustments, thus improving accuracy while controlling processing time through structured sequential operations.
Solution Approach 2:
Each SFF module performs preliminary feature fusion for its assigned partial disparity range before passing results to the next module. This preliminary action approach allows each module to focus on a specific disparity range with optimized processing, avoiding redundant computations and enabling more efficient sequential processing compared to handling the entire disparity range in a single stage.
Data Source
AI summary
Disclosed is a stereo matching method of images performed by a system implemented by a computer, including the steps of: receiving a pair of images obtained at different time points; generating a feature map by extracting features of each pixel of the pair of images; generating sequentially cost volumes for partial time points based on the feature map and generating a feature map for the entire disparity by fusing the cost volumes using 2D convolution; and generating a final disparity map by refining the generated feature map.