Warp Accumulation Hardware for Low-Overhead Vector Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing technologies face significant instruction overhead due to the need for numerous operations within a warp for accumulation, particularly in vector processing units like RNNs, leading to a 'long tail' effect with high overhead costs.
Innovation Solution
Decouple accumulation operations in a warp to dedicated hardware, using an accumulation computing unit and storage unit to process accumulation instructions independently, allowing parallel processing and reducing the need for one-time, full-lane accumulations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If accumulation is performed within the vector processing unit using shared storage units or warp lane shuffling, then accumulation can be completed, but a large number of instructions are consumed causing huge overhead
Solution Approach 1:
The patent extracts the accumulation function from the vector processing unit and implements it as a separate, dedicated accumulation computing unit. This dedicated unit handles accumulation operations independently, eliminating the need for complex instruction sequences within the vector processing unit and significantly reducing instruction overhead while improving accumulation performance
2Measurement precision
If 1024 elements are accumulated into the warp through 16 MMV instructions followed by 15 accumulation instructions, then the accumulation result is obtained, but the overhead reaches up to 94%
Solution Approach 1:
The patent segments the accumulation process into two independent parts: vector processing operations (MMV instructions) executed by the vector processing unit, and accumulation operations executed by a dedicated accumulation computing unit. This segmentation allows both operations to proceed in parallel without interfering with each other, dramatically reducing the total instruction execution time while maintaining accumulation accuracy
3Adaptability or versatility
If constant shuffling of values in warp lanes is performed to complete accumulation, then accumulation result is generated, but significant instruction overhead is incurred
Solution Approach 1:
The patent introduces a dedicated accumulation computing unit as an intermediary between the vector processing unit and the final result storage. This intermediary unit receives accumulation data from the vector processing unit, performs the accumulation operations using optimized algorithms, and returns results, thereby simplifying the operation while maintaining flexibility through configurable accumulation strategies
Data Source
AI summary
The embodiments of the disclosure relate to a computing device, a computing apparatus, and a method of warp accumulation and relate to the field of computers. The computing device includes a storage unit and an accumulation computing unit coupled to the storage unit. The accumulation computing unit is configured to receive, from a vector processing unit coupled to the computing device, a first warp accumulation instruction, a plurality of first values corresponding to a warp lane number, and a first storage address; generate a current accumulation result based on the plurality of first values in response to the first warp accumulation instruction; and store the current accumulation result in the first storage address in the storage unit to be read by the vector processing unit. In this way, accumulation in a warp may be decoupled to dedicated hardware for processing, and overall accumulation performance may thus be significantly improved.