Image Sensor Analog Inference Circuit for On-Chip MAC Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Image sensors generate large amounts of data that are inefficient to process using general-purpose microprocessors for tasks like image segmentation and object recognition, requiring specialized circuits for efficient multiplication and accumulation operations.
Innovation Solution
An integrated circuit device with an image sensing pixel array, memory cell array, and inference computation circuits, utilizing hybrid bonding to connect an image sensor chip and memory chip to a logic wafer, enabling direct multiplication and accumulation operations using memory cells to perform inference computations like artificial neural networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If general-purpose microprocessors are used for image processing, then device complexity is reduced, but computation performance and processing efficiency deteriorate
Solution Approach 1:
The system segments the image processing function by separating the pixel array from the processing logic. The pixel array generates image data while dedicated inference computation circuits perform specialized processing tasks, dividing the overall system into functionally distinct modules that can operate independently yet cooperatively.
Solution Approach 2:
The patent introduces an intermediary inference computation circuit that sits between the image sensor and general-purpose microprocessor. This intermediary handles intensive multiplication and accumulation operations, transferring only essential results to the microprocessor rather than raw image data, thus reducing the microprocessor's burden while maintaining high computation performance.
2Adaptability or versatility
If image data is transmitted to external processors, then processing flexibility is maintained, but data transmission overhead and processing efficiency deteriorate
Solution Approach 1:
The patent extracts the intensive computation operations (multiplication and accumulation) from the external processor and places them in dedicated inference computation circuits within the image sensor module. This extraction eliminates the need to transmit large volumes of raw image data externally, removing data transmission overhead while preserving the ability to perform flexible image processing through programmable inference circuits.
3Productivity
If specialized inference circuits are integrated into the image sensor, then computation performance is improved, but device complexity and manufacturing difficulty increase
Solution Approach 1:
The inference computation circuits are designed with multi-functionality, capable of performing various image processing tasks including multiplication, accumulation, and different inference algorithms. This universal design allows a single integrated circuit structure to handle multiple processing requirements, improving computation performance without proportionally increasing device complexity.
Solution Approach 2:
The patent uses memory cell arrays to store weight values and input data, effectively copying data structures needed for computation. This approach simplifies the integration of inference circuits by leveraging existing memory architectures rather than requiring entirely new computational hardware, thus improving performance while managing device complexity.
4Device complexity
If multiplication and accumulation operations are performed externally, then circuit design is simplified, but processing speed and energy efficiency deteriorate
Solution Approach 1:
The patent merges the multiplication and accumulation operations into a single integrated inference computation circuit within the image sensor. This merging eliminates the need for separate processing stages and data transfers, significantly improving processing speed and energy efficiency while maintaining manageable circuit design through modular architecture.
Data Source
AI summary
An integrated circuit device including: a first integrated circuit die having an image sensing pixel array; a second integrated circuit die having an image processing logic circuit and an inference logic circuit; and a third integrated circuit die having a memory cell array. The second integrated circuit die and the third integrated circuit die are connected via a direct bond interconnect. The inference logic circuit is configured to process an image from the image sensing pixel array via multiplication and accumulation operations based on memory cells in the memory cell array having threshold voltages programmed to store data in multiplications and output currents from the memory cells connected to lines in summations.