Imaging Circuit Layout for Low-Power Depth Map Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing depth estimation systems, such as TOF and stereo cameras, are bulky due to their components, and digital neural network operations require large circuit areas and high power consumption, limiting processing speed.
Innovation Solution
A system incorporating an imaging device and arithmetic circuit with memory portions for performing convolutional processing in neural networks, reducing the need for digital multiplication and addition circuits, and utilizing a novel operation method for product-sum operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If digital multiplication and addition circuits with multi-bit operation specifications are used, then calculation accuracy is improved, but circuit area increases and power consumption increases
Solution Approach 1:
The patent divides the neural network calculation into two separate stages: a first neural network performs coarse depth estimation to generate preliminary depth information, and a second neural network performs fine depth estimation to refine the results. This segmentation allows each network to use simplified circuits with lower bit specifications, reducing overall circuit area while maintaining final calculation accuracy through the two-stage processing approach.
Solution Approach 2:
The first neural network performs a preliminary calculation that provides sufficient accuracy for most cases, and the second neural network only performs additional calculations where needed to refine the depth estimation. This partial action approach avoids the need for both networks to use full multi-bit operation circuits, reducing total circuit area while achieving the required final precision.
2Measurement precision
If digital multiplication and addition circuits with multi-bit operation specifications are used, then calculation accuracy is improved, but power consumption increases
Solution Approach 1:
By segmenting the depth estimation into coarse and fine stages using two separate neural networks, the patent enables each network to operate with lower bit-width circuits that consume less power. The first network handles the bulk of the estimation with simplified circuits, while the second network refines results only when needed, reducing overall power consumption compared to using full multi-bit circuits throughout.
Solution Approach 2:
The system performs only the necessary amount of high-precision calculation in the second neural network based on the output from the first network. This partial action approach ensures that power-intensive multi-bit operations are executed only when required to improve accuracy, rather than continuously, thereby reducing average power consumption while maintaining calculation accuracy when needed.
3Measurement precision
If larger circuit area is used for multi-bit operation, then calculation accuracy is improved, but processing speed decreases
Solution Approach 1:
The patent segments the processing into two passes: the first neural network quickly generates coarse depth estimates using simplified circuits, and the second neural network refines specific regions needing higher precision. This segmentation allows the system to achieve high processing speed in the first pass and targeted accuracy improvement in the second pass, rather than processing everything at high precision from the start.
Solution Approach 2:
The system applies high-precision multi-bit operations only partially in the second neural network where depth estimation refinement is needed, rather than applying them universally. This selective approach maintains high processing speed for the majority of calculations while achieving accurate results in critical areas, resolving the trade-off between circuit size and processing speed.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system achieves efficient product-sum operations with low power consumption and high processing speed, generating depth maps and three-dimensional images.
Implementation Method 1
The imaging portion has a function of converting light reflected by an external subject into image data
Data Source
AI summary
A system with high processing speed and low power consumption is provided. The system includes an imaging device and an arithmetic circuit. The imaging device includes an imaging portion, a first memory portion, and an arithmetic portion, and the arithmetic circuit includes a second memory portion. The imaging portion has a function of converting light reflected by an external subject into image data, and the first memory portion has a function of storing the image data and a first filter for performing first convolutional processing in a first layer of a neural network. The arithmetic portion has a function of performing the first convolutional processing using the image data and the first filter to generate first data. The second memory portion has a function of storing the first data and a plurality of filters for performing convolutional processing in and after a second layer of the neural network. The arithmetic circuit has a function of performing processing in and after the second layer of the neural network using the first data to generate a depth map of the image data.