Grouped Convolution Weight Matrix Remapping for Faster AI Chips
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The computation speed of grouped convolution is decreased when executed by AI chips, which are optimized for normal convolution, leading to inefficiencies in parallel processing and increased overhead.
Innovation Solution
A method to redefine the grouped convolution processing by changing the number of groups and generating new weight matrices, placing them on a diagonal line with zero values in unused locations, optimizing the computation for AI chips.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If grouped convolution is executed by AI chips optimized for ordinary convolution, then computational accuracy is improved, but computation speed decreases
Solution Approach 1:
The patent changes the parameter of weight matrix structure by transforming grouped convolution weights into a block-diagonal format where each block corresponds to a group. This parameter transformation allows AI chips to execute grouped convolution more efficiently by reducing the impact of overhead operations and maximizing parallel processing capability, thereby increasing computation speed while maintaining accuracy
Solution Approach 2:
The patent segments the weight matrix into multiple smaller block-diagonal matrices, each corresponding to a specific group of channels. This segmentation enables the AI chip to process each group independently with optimized operations, reducing the overhead associated with processing all channels simultaneously and thus improving computation speed
2Quantity of substance
If grouped convolution divides input data into multiple groups, then arithmetic operations are reduced, but overhead impact increases
Solution Approach 1:
The patent transforms the weight matrix structure into a block-diagonal form where each diagonal block corresponds to a group of channels. This parameter change allows the system to maintain the reduced arithmetic operation count of grouped convolution while minimizing overhead impact by enabling more efficient memory access patterns and parallel execution on AI chips
3Productivity
If weight matrices are placed on diagonal line with zero elsewhere, then parallel processing is maximized, but weight matrix complexity increases
Solution Approach 1:
The patent segments the weight matrix into block-diagonal structures where each block corresponds to a group of channels. This segmentation creates a pattern where weight matrices are placed on diagonal lines with zero elsewhere, which maximizes parallel processing capability on AI chips while the structured pattern actually simplifies memory access and computation compared to dense weight matrices
Data Source
AI summary
A generation unit generates a new j-th weight matrix to be used in the convolution computation of data consisting of {(j−1)×N/g+1} to (j×N/g) channels (j=1 to g) divided after the divided number is changed over j=1 to j=g respectively. The generation unit generates the new j-th weight matrix by placing {(j−1)×G/g+1} to (j×G/g) weight matrices from the upper left to the lower right of the new j-th weight matrix in the order of {(j−1)>G/g+1} to (j×G/g) weight matrices on the diagonal line, and setting values of all components except for components at locations where the weight matrices are placed to 0.