Reconfigurable AI Accelerator Array for Long-Sequence Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Self-attention-based transformers face challenges in processing long sequences due to increased computational requirements, making it difficult to model long contexts efficiently.
Innovation Solution
A computing unit with a first operator circuit and a reconfigurable array is designed to perform operations through recursive matrix generation and reconfigurable connections, supporting complex number operations, FFT, and convolution, while minimizing memory usage and optimizing throughput.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If self-attention-based transformers are used to process long sequences, then sequence modeling capability is improved, but computational requirements increase quadratically with sequence length
Solution Approach 1:
The input sequence is divided into multiple chunks that are processed independently and then combined. The transformer model processes each chunk separately rather than the entire sequence at once, reducing the quadratic computational burden while maintaining the ability to capture long-range dependencies through the chunking strategy and residual connections.
2Measurement precision
If memory is allocated to store intermediate results for long sequence processing, then processing accuracy is improved, but memory usage increases
Solution Approach 1:
The method extracts and processes only the necessary intermediate results from each chunk rather than storing all possible intermediate states. By selectively computing and retaining only the essential information needed for final sequence modeling, memory consumption is reduced while maintaining processing accuracy.
Solution Approach 2:
Different memory allocation strategies are applied to different parts of the sequence processing pipeline. Critical intermediate results that impact final accuracy are stored with higher precision, while less critical data uses reduced precision or is computed on-demand, optimizing the trade-off between accuracy and memory usage.
3Device complexity
If a fixed architecture is used for processing, then device complexity is reduced, but adaptability to different operations (FFT, convolution, complex number operations) is limited
Solution Approach 1:
The computing unit is designed with a universal architecture that can perform multiple operations including FFT, convolution, and complex number operations through a single integrated structure. The same hardware components are configured to execute different mathematical operations, eliminating the need for separate dedicated circuits for each function and reducing overall device complexity.
Data Source
AI summary
A computing unit, a hardware accelerator including a computing unit, and a method of operating a computing unit are disclosed. The computing unit includes a first operator circuit configured to generate a matrix used for a first operation with an input chunk through a recursive operation, and a reconfigurable array configured to reconfigure a connection to an input port or an output port to perform the first operation between the matrix and the input chunk and to perform second operations, wherein the second operations are different from the first operation.