Neural Network Imaging Device With Split Convolution Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing depth estimation systems, such as TOF and stereo cameras, are bulky due to required components and digital neural networks require large circuit areas and high power consumption for multiplication and addition operations, limiting processing speed.
Innovation Solution
An imaging device with an arithmetic circuit that performs convolutional processing using a neural network, incorporating memory for filter storage and operation, reducing the need for digital multiplication and addition circuits.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If digital multiplication and addition circuits with multi-bit operation specifications are used for neural network arithmetic operations, then calculation accuracy is improved, but circuit area increases and power consumption increases
Solution Approach 1:
The patent divides the neural network calculation into two stages: first-layer convolutional processing performed by the imaging device itself, and subsequent processing (second layer onwards) performed by an external arithmetic circuit. This segmentation allows the imaging device to use simpler circuits while maintaining overall calculation accuracy through the coordinated external arithmetic system.
Solution Approach 2:
The patent introduces an external arithmetic circuit as an intermediary to perform the computationally intensive multiplication and addition operations. The imaging device outputs intermediate results to this external circuit, which then processes the data through multiple neural network layers before returning the final depth map, thereby distributing the computational burden and reducing the imaging device's circuit complexity.
2Measurement precision
If digital multiplication and addition circuits with multi-bit operation specifications are used for neural network arithmetic operations, then calculation accuracy is improved, but power consumption increases
Solution Approach 1:
The patent segments the computational workload between the imaging device and an external arithmetic circuit. The imaging device performs only first-layer convolutional processing with lower power consumption, while the energy-intensive subsequent processing is offloaded to the external circuit, reducing overall power consumption while maintaining calculation accuracy.
Solution Approach 2:
An external arithmetic circuit acts as an intermediary to handle the power-consuming multiplication and addition operations. This intermediary processes the neural network calculations externally, allowing the imaging device to maintain high accuracy while the system overall achieves lower power consumption through distributed computation.
3Measurement precision
If larger circuit area is used for digital multiplication and addition circuits, then calculation accuracy is improved, but processing speed decreases
Solution Approach 1:
The patent segments the neural network processing into first-layer convolution (performed by imaging device) and subsequent layers (performed by external arithmetic circuit). This segmentation enables parallel processing capabilities where the external circuit handles computationally intensive operations without bottlenecking the imaging device's processing speed.
Solution Approach 2:
The external arithmetic circuit serves as an intermediary that processes neural network operations externally, preventing the imaging device's processing speed from being limited by large-scale multiplication and addition circuits. This intermediary approach maintains high processing speed while achieving accurate depth estimation through multi-layer neural network processing.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables efficient product-sum operations with low power consumption and high processing speed for depth estimation, providing a compact and efficient system.
Implementation Method 1
The imaging portion has a function of converting light reflected by an external subject into image data
Data Source
AI summary
A system with high processing speed and low power consumption is provided. The system includes an imaging device and an arithmetic circuit. The imaging device includes an imaging portion, a first memory portion, and an arithmetic portion, and the arithmetic circuit includes a second memory portion. The imaging portion has a function of converting light reflected by an external subject into image data, and the first memory portion has a function of storing the image data and a first filter for performing first convolutional processing in a first layer of a neural network. The arithmetic portion has a function of performing the first convolutional processing using the image data and the first filter to generate first data. The second memory portion has a function of storing the first data and a plurality of filters. The arithmetic circuit has a function of generating a depth map of the image data.