Grouped Convolution Optimization via Zero Padding for AI Chip Memory Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
AI chips designed for CNNs face challenges in executing grouped convolution efficiently due to memory addressing constraints, leading to reduced computation speed and potential incorrect operations.
Innovation Solution
A grouped convolution processing optimization device and method that modify weight matrices by inserting zeros to align memory addresses, allowing grouped convolution to be executed efficiently on AI chips with memory addressing constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If grouped convolution is executed on AI chips with memory addressing constraints, then computation speed is reduced and operations may become incorrect, but modifying weight matrices to align memory addresses increases device complexity
Solution Approach 1:
The patent applies preliminary action by modifying weight matrices before grouped convolution execution to align memory addresses with hardware constraints. The optimization device pre-processes weight matrices by inserting zero columns or rows to ensure that memory addresses of weight data satisfy the addressing constraints of AI chips, thereby preventing computation errors before they occur.
Solution Approach 2:
The patent uses an intermediary approach by introducing an optimization device that acts as a mediator between the grouped convolution algorithm and the AI chip hardware. This optimization device transforms weight matrices into a format compatible with hardware constraints without requiring changes to the underlying hardware architecture or the fundamental grouped convolution algorithm.
2Speed
If grouped convolution is executed on AI chips with memory addressing constraints, then computation speed is reduced, but modifying weight matrices to align memory addresses increases processing overhead
Solution Approach 1:
The optimization device performs preliminary modification of weight matrices to align memory addresses before grouped convolution execution. By pre-aligning the memory addresses of weight data with hardware constraints, the system avoids runtime penalties and ensures optimal computation speed without incurring excessive processing overhead during actual convolution operations.
3Productivity
If grouped convolution is executed without memory address alignment, then processing is faster, but computation accuracy deteriorates
Solution Approach 1:
The patent applies preliminary action by modifying weight matrices before grouped convolution execution to align memory addresses with hardware constraints. The optimization device pre-processes weight matrices by inserting zero columns or rows to ensure that memory addresses of weight data satisfy the addressing constraints of AI chips, thereby preventing computation errors before they occur.
Data Source
AI summary
The device includes a first unit that executes at least one of a column process that inserts (M−N/G) columns of zeros at the right end of an i-th weight for each i from 1 to G or a row process that inserts (M−N/G) rows of zeros at the bottom of the i-th weight for each i from 1 to G, when a constraint that requires convolution computation to be executed on data composed of M channels for a grouped convolution is imposed and a second unit that executes at least one of an input weight process that inserts (M−N/G) columns of zeros to the right of the i×N/G-th column of the input weight for each i from 1 to G or an output weight process that inserts (M−N/G) rows of zeros below the i×N/G-th row of the output weight for each i from 1 to G.