SIPP Accelerator Pipeline for Mobile Stereo Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computational imaging, which requires substantial computational resources for tasks like stereo matching to create 3D models from 2D images, faces challenges in being implemented on mobile devices due to limited processing power and memory capacity, necessitating efficient optimization techniques to operate within low power budgets.
Innovation Solution
A pipelined and flexible hardware architecture is introduced, incorporating a Streaming Image Processing Pipeline (SIPP) accelerator with components like Census Transform engines, descriptor buffer engines, cost matching engines, and SGBM aggregation engines, allowing for efficient processing of stereo vision algorithms with reduced power consumption by bypassing unnecessary stages and using circular buffers to minimize data movement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If computational imaging tasks like stereo matching are implemented on mobile devices, then image processing capabilities are improved, but power consumption increases and processing efficiency deteriorates due to limited processing power and memory capacity
Solution Approach 1:
The stereo matching process is divided into multiple independent pipeline stages: cost computation stage, cost aggregation stage, and disparity computation stage. Each stage processes data independently and can be optimized separately, allowing mobile devices to execute only necessary stages based on power availability while maintaining overall image processing capability.
Solution Approach 2:
Cost computation is performed in advance during a first pass through the image data before aggregation and disparity computation. This preliminary action allows subsequent stages to work with pre-computed cost values, reducing the computational burden during real-time processing and lowering overall power consumption on mobile devices.
2Measurement precision
If traditional stereo matching algorithms are used, then processing accuracy is maintained, but processing speed decreases and real-time performance is lost due to substantial computational resource requirements
Solution Approach 1:
The matching pipeline is segmented into parallel independent stages that can process different regions of image data simultaneously. The cost computation stage processes multiple pixel locations in parallel, while the aggregation stage operates independently on computed cost values, maintaining accuracy while achieving real-time processing speeds on mobile hardware.
Solution Approach 2:
The pipeline uses periodic processing where cost computation is performed for all pixel locations, followed by periodic aggregation passes that refine results. This periodic action pattern allows the system to maintain high accuracy through multiple refinement passes while controlling processing speed to meet real-time requirements.
Data Source
AI summary
Methods, apparatus, systems, and articles of manufacture to optimize pipeline execution are disclosed. An example apparatus includes at least one memory, machine readable instructions, and processor circuitry to execute the machine readable instructions to determine a value associated with a first location of a first pixel of a first image and a second location of a second pixel of a second image by calculating a matching cost between the first location and the second location, generate a disparity map including the value, and determine a minimum value based on the disparity map corresponding to a difference in horizontal coordinates between the first location and the second location.