3D Stacked Imaging Sensor With On-Chip Neural Feature Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing imaging devices face challenges in achieving high optical sensitivity for photoelectric conversion elements and low noise characteristics for transistors, while also requiring efficient arithmetic operations for neural networks.
Innovation Solution
A three-dimensionally integrated imaging device is developed, featuring a neuron in a neural network with a plurality of first pixels, first, second, and third circuits. Each pixel includes a photoelectric conversion element connected to the first circuit, which amplifies analog signals. The second circuit adds these signals, and the third circuit converts them into feature data using an activation function, thereby enhancing arithmetic operation speed and reducing signal deterioration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If photoelectric conversion elements and transistors are integrated in traditional two-dimensional structures, then manufacturing process complexity is reduced, but optical sensitivity and noise characteristics deteriorate
Solution Approach 1:
The patent transitions from two-dimensional planar integration to three-dimensional stacked integration, where photoelectric conversion elements and transistors are arranged in vertical layers. This dimensional change allows both components to maintain their optimal two-dimensional footprints while achieving three-dimensional spatial separation, thereby improving optical sensitivity and noise characteristics without significantly increasing manufacturing complexity.
Solution Approach 2:
The imaging device is segmented into distinct functional layers: photoelectric conversion elements in one layer and transistors in separate underlying or adjacent layers. This segmentation allows each component type to be optimized independently for its specific function while maintaining compact integration through vertical stacking.
2Measurement precision
If pixel data is processed and transmitted in high resolution, then image quality is improved, but arithmetic operation load and processing time increase
Solution Approach 1:
The patent extracts and processes only the essential feature information from pixel data using neural network operations performed directly in the imaging device. By extracting key features at the sensor level rather than transmitting all raw pixel data, the system maintains high image quality while dramatically reducing the arithmetic operation load and processing time required for subsequent image analysis.
Solution Approach 2:
The imaging device performs preliminary neural network-based feature extraction and compression operations directly at the sensor before data transmission. This preliminary processing reduces the volume of data that requires extensive arithmetic operations later, thereby improving overall processing efficiency while preserving essential image quality information.
3Productivity
If signal processing circuits are added to improve neural network operations, then arithmetic operation capability is improved, but device complexity increases
Solution Approach 1:
The patent implements multi-functional circuits that serve both traditional image processing roles and neural network computation functions. The same transistor structures and interconnect architectures are used for both conventional pixel operations and neural network arithmetic operations, thereby improving arithmetic capability without proportionally increasing device complexity.
Solution Approach 2:
The patent merges neural network processing circuits with traditional pixel circuit structures, combining multiple functions into unified circuit blocks. This merging allows the device to perform both conventional imaging operations and neural network computations using shared hardware resources, thereby enhancing arithmetic operation capability while controlling overall device complexity.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The proposed imaging device achieves efficient compression and conversion of pixel data, increases arithmetic operation speed, and provides a three-dimensionally integrated structure with reduced signal deterioration, thereby enhancing its functional capabilities.
Implementation Method 1
Each of the plurality of first pixels includes a photoelectric conversion element. The photoelectric conversion element generates an analog signal in response to incident light
Data Source
AI summary
An imaging device connected to a neural network is provided. An imaging device having a neuron in a neural network includes a plurality of first pixels, a first circuit, a second circuit, and a third circuit. Each of the plurality of first pixels includes a photoelectric conversion element. The plurality of first pixels is electrically connected to the first circuit. The first circuit is electrically connected to the second circuit. The second circuit is electrically connected to the third circuit. Each of the plurality of first pixels generates an input signal of the neuron. The first circuit, the second circuit, and the third circuit function as the neuron. The third circuit includes an interface connected to the neural network.