Compute-In-Memory Engine With Input Reshaping for Low-Latency VMM
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing learning networks face challenges with time-consuming computations, disparate hardware component requirements, and high power consumption, particularly in edge devices, which hinder efficient data processing and model training.
Innovation Solution
The integration of a compute-in-memory (CIM) module and input buffer within a compute engine, which includes storage cells and compute logic, allows for efficient data reshaping and parallel vector-matrix multiplications, reducing latency and power consumption by enabling local updates and accommodating different data transfer speeds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If learning networks use millions of parameters multiplied by activations to solve challenging problems, then the problem-solving capability is improved, but the computation time becomes excessively long
Solution Approach 1:
The patent replaces traditional digital computing mechanisms with analog computing mechanisms. Weight layers are implemented as analog circuits where resistors represent weights and current represents activation values. The multiplication operation is performed through analog current summation at nodes, eliminating the need for digital multiplication cycles. This analog substitution dramatically reduces computation time while maintaining the neural network's problem-solving capability.
2Adaptability or versatility
If hardware accelerators use different formats for data transfer, storage, and operations in different components, then component flexibility is improved, but system integration complexity and throughput sacrifice increase
Solution Approach 1:
The patent implements a unified data format (e.g., 8-bit or 16-bit digital-to-analog converter output format) that is used consistently across all components including weight storage, activation input, and computation nodes. This universal format eliminates the need for format conversion between components, reducing integration complexity while maintaining the flexibility to handle different neural network operations. The analog computation nodes universally accept current inputs regardless of the original digital format.
Solution Approach 2:
The patent introduces digital-to-analog converters (DACs) as intermediary components that bridge the digital control domain and analog computation domain. These DACs serve as standardized interfaces that convert digital weight and activation values into analog currents, enabling seamless integration between digital control logic and analog computation elements without requiring each component to support multiple formats.
3Adaptability or versatility
If edge devices use learning networks for inference, then intelligence capability is improved, but power consumption increases
Solution Approach 1:
The patent replaces energy-intensive digital computation with lower-power analog computation. In analog circuits, multiplication and accumulation operations occur simultaneously through passive current summation at circuit nodes, eliminating the need for sequential digital operations, memory access, and control logic that consume significant power in traditional implementations. This analog approach dramatically reduces power consumption while maintaining inference capability.
Data Source
AI summary
A compute engine (CE) is described. The CE includes a compute-in-memory (CIM) module and an input buffer coupled with the CIM module. The CIM module includes storage cells and compute logic coupled with the storage cells. The storage cells are arranged in rows and columns. The input buffer is configured to receive data, reshape the data, and provide reshaped data to the CIM module.