Matrix Multiplication Data Splitting for Memory Access Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The general matrix multiplication operator in deep learning models is memory-intensive and time-consuming due to inefficient memory access, leading to sub-optimal performance of artificial intelligence chips.
Innovation Solution
A method for determining optimal matrix combinations and splitting information based on storage space capacities and bandwidth to minimize memory access overhead, reducing repeated access and improving efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the general matrix multiplication operator is implemented with conventional memory access methods, then the operation can be performed, but the memory access efficiency is low and the execution time is long
Solution Approach 1:
The patent segments the matrix multiplication operation into multiple matrix combinations, where each combination includes a first matrix and a second matrix. The segmentation is performed based on storage space capacities and bandwidth characteristics, allowing parallel processing of different matrix combinations to improve execution efficiency and reduce overall execution time.
Solution Approach 2:
The patent introduces a new dimension of optimization by considering bandwidth information of computing units alongside storage space capacity. This multi-dimensional approach enables selecting matrix combinations that optimize both space utilization and data transfer efficiency, thereby improving memory access performance and reducing execution time.
2Productivity
If more matrix combinations are processed, then the optimization opportunities increase, but the memory access resource overhead increases
Solution Approach 1:
The patent dynamically adjusts processing parameters by determining target matrix combinations based on bandwidth information and target splitting information. This parameter optimization ensures that the number of matrix combinations processed is balanced against memory access resource overhead, achieving productivity improvement without excessive energy loss.
Solution Approach 2:
The patent implements a feedback mechanism where memory access resource overhead is calculated for each matrix combination and used to select the optimal target matrix combination. This feedback loop ensures that optimization opportunities are pursued only when they do not excessively increase memory access resource overhead, maintaining energy efficiency.
3Productivity
If the matrix multiplication operation is optimized for memory access, then the execution efficiency improves, but the complexity of determining optimal matrix combinations increases
Solution Approach 1:
The patent performs preliminary actions by pre-determining target sub-shape information based on storage space capacities before executing the matrix multiplication operation. This preliminary processing of matrix combination parameters simplifies the main computation and reduces algorithmic complexity while maintaining execution efficiency improvements.
Data Source
AI summary
A method for processing matrix multiplication data, an electronic device, and a storage medium are provided, which relate to artificial intelligence technology, and in particular to chip technology. The method includes: determining a first target sub-shape information of each combination of I matrix combinations according to a shape information of the second matrix of each combination and a capacity of the second storage space of each combination; determining a second target sub-shape information of each combination according to a capacity of the first storage space of each combination and the first target sub-shape information of each combination; determining a target splitting information of each combination according to the second target sub-shape information of each combination; and determining a target matrix combination from the I matrix combinations according to a bandwidth information of a computing unit that performs the matrix multiplication operation and the target splitting information of each combination.