Analog Inference Integrated Circuit for Image Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image processing systems face inefficiencies in transmitting and processing large amounts of image data from image sensors, particularly for computations involving intensive multiplications and accumulations, which are not effectively accelerated by general-purpose microprocessors.
Innovation Solution
An integrated circuit device is configured with an image sensing pixel array, a memory cell array, and inference computation circuits, utilizing hybrid bonding for direct connections between the image sensor chip and memory chip, enabling efficient multiplication and accumulation operations through a 3D memory array, allowing for on-chip processing of image data using artificial neural networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If image data is transmitted from image sensors to general-purpose microprocessors for processing, then processing flexibility is maintained, but processing speed and efficiency deteriorate due to intensive data transmission and computation requirements
Solution Approach 1:
The patent merges the image sensor chip and memory chip into a single integrated circuit package, allowing direct on-chip communication between the image sensor array and memory array without external data transmission. This integration eliminates the data transmission bottleneck and enables fast multiplication and accumulation operations for image processing directly within the chip.
Solution Approach 2:
The patent replaces the traditional mechanical/data transmission-based processing approach with an analog computational approach using hybrid-bonded memory cells that perform multiplication and accumulation operations through electrical signal interactions, achieving faster processing speeds compared to general-purpose microprocessors.
2Productivity
If specialized circuits like multiplier-accumulator units are used to accelerate computations, then computation performance improves, but device complexity increases
Solution Approach 1:
The patent designs the memory cell array to serve multiple functions: storing weight matrices for neural network inference, performing multiplication and accumulation operations through analog signal processing, and enabling both image processing and machine learning computations. This multi-functionality reduces the need for separate dedicated hardware components.
Solution Approach 2:
The patent utilizes changes in electrical parameters (voltages, currents) within the memory cells to represent and process computational data. By manipulating electrical signals through the memory cell array, the system achieves complex multiplication and accumulation operations without requiring complex digital logic circuits, thus improving computation performance while managing device complexity.
3Productivity
If on-chip processing is implemented using integrated circuits, then processing efficiency improves, but manufacturing complexity increases due to hybrid bonding requirements
Solution Approach 1:
The patent divides the integrated circuit into distinct functional segments: an image sensor chip containing the image sensor array, a memory chip containing the memory cell array, and inference computation circuits. These segmented components are then hybrid-bonded together, allowing each module to be manufactured and optimized independently while achieving efficient on-chip processing when assembled.
Data Source
AI summary
A method to enhance images, including: receiving, in an image processing logic circuit in an integrated circuit device, first data representative of an input image; generating, by the image processing logic circuit, input data for an inference logic circuit in the integrated circuit device; generating, by the inference logic circuit, a column of bits from the input data; performing, by the inference logic circuit using memory cells in the integrated circuit device having threshold voltages programmed to represent at least one weight matrix, operations of multiplication and accumulation, via reading concurrently rows of the memory cells selected according to the column of bits; generating, by the inference logic circuit, output data based on results of the operations multiplication and accumulation; and generating, by the image processing logic circuit using the output data, second data representative of an output image enhanced from the input image.