Compute Tile Architecture for Parallel Convolution Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing learning networks face inefficiencies and high latencies, particularly when dealing with large models and data sets, due to limitations in memory and parallel processing capabilities.
Innovation Solution
The implementation of a compute tile architecture within an AI accelerator, which includes compute engines with compute-in-memory (CIM) hardware modules and a general-purpose processor, allows for efficient profiling and rescheduling of operations like convolutions to improve parallelism and reduce memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If parallel processing is used to speed up convolutions, then processing speed is improved, but memory usage increases and may exhaust available memory
Solution Approach 1:
The patent divides the convolution operation into multiple stages, separating weight loading from input data processing. Weights are loaded once into the CIM module's local memory, then multiple input batches can be processed using these weights without reloading, thereby segmenting the memory usage pattern to avoid exhaustion while maintaining parallel processing capability.
Solution Approach 2:
The patent introduces a temporal dimension to the processing architecture by implementing a pipeline where weights are pre-loaded and stored in the CIM module's local memory buffer. This allows subsequent input data to be processed in parallel against the stored weights over time, effectively adding a time-based dimension that decouples memory usage from parallel processing intensity.
2Productivity
If more memory is allocated for parallel convolution operations, then processing capacity is improved, but device complexity and memory cost increase
Solution Approach 1:
The patent merges the weight storage function with the computation function by implementing compute-in-memory (CIM) architecture. The CIM module combines SRAM cells for weight storage with multiplication and accumulation logic, allowing weights to be stored locally and used directly in parallel computations without requiring separate large-scale external memory, thereby reducing overall memory architecture complexity while maintaining high parallel processing capacity.
3Quantity of substance
If sequential processing is used to reduce memory usage, then memory efficiency is improved, but processing time increases
Solution Approach 1:
The patent implements preliminary action by pre-loading weights into the CIM module's local memory before processing input data batches. This preliminary weight loading enables subsequent parallel processing of multiple input batches without requiring additional weight memory allocations, achieving both memory efficiency and reduced processing time through advance preparation.
Data Source
AI summary
A method is described. The method includes profiling a learning network. The learning network includes compute tile(s) and a model. A compute tile includes compute engines and a general-purpose (GP) processor. Each compute engine includes a compute-in-memory (CIM) hardware module. The model includes convolutions and activation functions. The convolutions correspond to the compute engines. The method also includes determining, based on the profiling, a reschedule operation for a convolution of the convolutions. The reschedule operation provides multiple tensors based on an input tensor for the convolution. The tensors are configured to undergo at least a portion of the convolution in accordance with a temporal distribution. The method also includes performing, on the learning network and using the reschedule operation, a forward pass for input data