Multi-Row Storage Cells for Adaptive Neural Dataflow
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI accelerators are limited by a fixed dataflow architecture that does not optimize for the diverse layer types in neural networks, leading to suboptimal performance and inefficiencies in energy consumption and throughput.
Innovation Solution
A memory circuit with adaptive processing elements that can selectively store input or weight data elements based on the layer type, utilizing a data router and column-wise write circuit to enhance efficiency and reduce energy consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a fixed dataflow architecture is used in AI accelerators, then the device structure is simple, but the adaptability to diverse layer types is poor
Solution Approach 1:
The patent implements dynamic reconfiguration of processing elements between different dataflow modes (row-wise and column-wise) based on the layer type being processed. The processing elements can selectively activate different storage cells and data paths, allowing the architecture to adapt its behavior dynamically rather than being fixed, thus resolving the contradiction between adaptability and complexity.
Solution Approach 2:
The patent designs processing elements that can perform multiple functions by selectively storing and processing either input data elements or weight data elements based on control signals. This multi-functionality allows a single unified architecture to handle diverse layer types (convolutional, fully connected, attention mechanisms) without requiring separate specialized hardware for each, thereby improving adaptability while maintaining reasonable complexity.
2Adaptability or versatility
If additional buffer and transpose operations are implemented, then the adaptability to different layer types is improved, but the device complexity and energy consumption increase
Solution Approach 1:
The patent merges the storage of input data and weight data within the same processing elements, eliminating the need for separate buffer memory structures. By combining these functions in-place, the architecture reduces the energy overhead associated with additional memory access and data movement operations while maintaining support for diverse layer types.
Solution Approach 2:
The processing elements perform transpose operations on-demand within their local storage cells rather than relying on separate buffer and transpose circuits. This self-service approach allows the computation units to handle their own data reorganization needs, reducing the energy consumption associated with dedicated transpose hardware and buffer management.
3Productivity
If processing elements store multiple data elements, then the throughput is improved, but the manufacturing precision requirements increase
Solution Approach 1:
The patent segments the data storage within processing elements into distinct storage cells that can be independently controlled. Each processing element contains multiple storage cells that can be selectively activated based on the layer type, allowing parallel storage and processing of multiple data elements without requiring uniform high-precision manufacturing across all cells simultaneously. This segmentation enables gradual scaling of precision requirements.
Data Source
AI summary
A memory circuit includes a first buffer configured to store a plurality of first data elements; a second buffer configured to store a plurality of second data elements; a controller configured to generate a control signal based on a layer type; an array comprising a plurality of processing elements (PEs), each of the PEs including a plurality of storage cells; and a data router configured to receive the control signal and determine whether to store, in the storage cells of each of the PEs, a corresponding one of the plurality of first data elements or corresponding ones of the plurality of second data elements based on the control signal.