Hybrid-Bonded Analog Inference for On-Device Image Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Image sensors generate large amounts of data that are inefficient to transmit to general-purpose microprocessors for processing tasks like image segmentation and object recognition, and existing specialized circuits for multiplication and accumulation operations face limitations in high-density connections.
Innovation Solution
An integrated circuit device with an image sensing pixel array, memory cell array, and inference computation circuits is implemented, utilizing hybrid bonding to connect the image sensor chip and memory chip directly to a logic wafer, enabling efficient multiplication and accumulation operations through memory cell arrays configured for analog inference computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If image data is transmitted to general-purpose microprocessors for processing, then processing capability is provided, but transmission efficiency and processing speed deteriorate due to the inefficiency of general-purpose processors for intensive computations
Solution Approach 1:
The patent segments the processing system into specialized functional units: image sensors for data generation, specialized circuits (MAC units, neural network accelerators) for specific computation tasks, and memory systems for data storage. This segmentation allows each component to be optimized for its specific function, dramatically improving image processing speed while reducing the complexity burden on general-purpose processors.
Solution Approach 2:
The patent introduces intermediary specialized processing circuits between the image sensors and general-purpose microprocessors. These intermediaries (such as MAC units and neural network accelerators) handle intensive computation tasks, freeing the general-purpose processors from bottleneck operations and improving overall system efficiency.
2Productivity
If specialized circuits are developed for multiplication and accumulation operations, then computation performance is improved, but connection density and integration limitations worsen
Solution Approach 1:
The patent merges multiple specialized circuits into integrated hybrid architectures. For example, MAC units are integrated with memory systems, and neural network accelerators are combined with image processing pipelines. This merging maintains high computation performance while reducing the complexity of interconnections by consolidating functions into unified structures.
Solution Approach 2:
The patent designs specialized circuits with multi-functional capabilities. For instance, memory systems are designed to simultaneously serve as storage and computation units (computational memory), and processing circuits are configured to handle multiple operation types (multiplication, accumulation, logical operations). This universality reduces the number of separate components needed, thereby reducing connection complexity.
3Loss of time
If image data is processed externally, then processing flexibility is maintained, but data transmission requirements and system latency increase
Solution Approach 1:
The patent implements preliminary processing actions at the source by embedding specialized processing circuits directly at or near the image sensors. This allows initial image processing (such as compression, feature extraction, or neural network inference) to occur before data leaves the sensor array, significantly reducing both processing latency and the volume of data that needs to be transmitted externally.
Solution Approach 2:
The patent transitions from external centralized processing to on-device distributed processing by integrating computation capabilities directly into the imaging system. This dimensional shift from external to internal processing eliminates transmission bottlenecks and reduces latency by performing computations in the same physical location where data is generated.
Data Source
AI summary
A method in an integrated circuit device to compress images, including: generating, by an image processing logic circuit and based on first data representative of an input image, input data; generating, by an inference logic circuit and based on the input data, a column of inputs; converting, by the inference logic circuit using voltage drivers connected to wordlines and memory cells storing a weight matrix, and into output currents of the memory cells summed in bitlines, results of bitwise multiplications of bits in the column of inputs and bits stored in the memory cells in a form of threshold voltages of the memory cells; digitizing currents summed in the bitlines to obtain column outputs; generating, by the inference logic circuit, output data based on the column outputs; and generating, using the output data, second data representative of an output image compressed from the input image.