GPU Multiply-Accumulate Pipeline for Neural Network Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing GPU processing devices face limitations in efficiently performing high volumes of mathematical operations, particularly in parallel processing scenarios.
Innovation Solution
The implementation of a GPU pipeline architecture that includes a computational pipeline with a multiplier and summer, along with a sequencer that manages the transfer of operand segments to perform multiply/accumulate operations efficiently, even for double-width input arguments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If GPU processing devices are configured with many processing pipelines to perform operations in parallel, then processing speed and throughput are improved, but device complexity and difficulty of managing parallel operations increase
Solution Approach 1:
The patent divides the multiply-accumulate operation into segmented steps: loading operand segments from memory, performing partial products, and accumulating results in stages. This segmentation allows complex parallel operations to be broken down into manageable units that can be executed across multiple pipelines while maintaining control over the overall process
Solution Approach 2:
The patent implements preliminary actions by pre-loading operand segments into processing pipelines before the main computation begins. The sequencer prepares and stages data in advance, allowing pipelines to be ready for operation and reducing the complexity of coordinating data flow during parallel execution
2Productivity
If existing multiply-accumulate operations are used for neural network applications, then computational functionality is provided, but processing efficiency for high volumes of operations is limited
Solution Approach 1:
The patent ensures continuity of useful action by implementing continuous loading of operand segments from memory into the processing pipelines while computations are ongoing. The sequencer coordinates this continuous data flow to ensure pipelines remain busy and productive, maximizing the volume of operations that can be executed efficiently
Solution Approach 2:
The patent creates a universal multiply-accumulate architecture that can handle various neural network operations through a single standardized pipeline design. The same basic unit can perform different mathematical operations by changing operands and control parameters, improving ease of operation across different applications
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A device for performing multiply/accumulate operations processes values in first and second buffers and having a first width using a computational pipeline with a second width, such as half the first width. A sequencer processes combinations of portions (high-high, low-low, high-low, low-high) of the values in the first and second buffers using a multiply/accumulate circuit and adds the accumulated result of each combination of portions to a group accumulator. Adding to the group accumulator may be preceded by left shifting the accumulated result (the first width for the high-high combination and the second width for the low-high and high-low combination).