Matrix Multiplication Hardware Architecture DSP48 Chain Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing matrix multiplication hardware architectures, such as flexible sparse block (FSB) on FPGAs, suffer from low computational efficiency and excessive resource consumption due to inefficient adder tree structures and high LUT resource usage, particularly when accelerating large language models.
Innovation Solution
A matrix multiplication hardware architecture featuring a reduction network with a tree topology and a DSP48 chain, where digital signal processing units are cascaded with data selectors and pre/post-adders, optimizing the adder tree structure to improve resource utilization and timing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If an adder tree structure is used in FSB hardware architecture, then reduction operations can be performed, but computational efficiency is low and resource consumption is high
Solution Approach 1:
The patent extracts and removes redundant adder components from the traditional adder tree structure. By eliminating unnecessary adders and optimizing the reduction network topology, the design reduces hardware resource consumption while maintaining computational functionality, directly addressing the contradiction between computational efficiency and hardware resource usage
Solution Approach 2:
The patent merges multiple adder functions into optimized reduction network nodes and combines computation and reduction operations more efficiently. By merging redundant operations and sharing hardware resources across multiple computation units, the design achieves higher computational efficiency with reduced hardware footprint
2Productivity
If multiple computation units are configured to accelerate large language model inference, then computational efficiency improves, but LUT resource consumption increases significantly
Solution Approach 1:
The patent designs computation units with universal, multi-functional capabilities that can handle various matrix multiplication operations and sparsity patterns. By making each computation unit adaptable to different workloads and configurations, the system can accelerate LLM inference effectively while reducing the total number of units needed, thereby conserving LUT resources
Solution Approach 2:
The patent introduces dynamic configuration capabilities to computation units, allowing them to adapt their operational characteristics based on workload requirements. This dynamic adaptability enables fewer, more versatile computation units to replace multiple fixed-function units, reducing overall LUT consumption while maintaining inference acceleration performance
Data Source
Figure 1(a)~1(f)
Figure 2
AI summary
The present disclosure provides a matrix multiplication hardware architecture, including: a reduction network, including a tree topology with multiple levels formed by a plurality of reduction network nodes, where the reduction network node includes a data selector and two computation paths; and a digital signal processing unit DSP48 chain, formed by cascading a plurality of digital signal processing units DSP48, where output ends of adjacent digital signal processing units DSP48 are respectively connected to two computation paths of a same reduction network node in a first level of the tree topology, and outputs of two computation paths pass through a data selector and then are connected to a reduction network nodes in an upper level of the tree topology. According to the present application, an addition tree of FSB is changed to an addition chain adapted to the DSP48 structure, thereby reusing a post-adder of the DSP48 and improving hardware utilization. Meanwhile, the hardware architecture of the present application can change a higher-bit sign bit extension to zero padding, thereby greatly saving resources and optimizing timing.