Convolution Decomposition for Edge Device Memory Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning networks face challenges in efficiently implementing convolution operations on edge devices due to limited resources, leading to high data transfer, memory usage, and computational requirements, which are not effectively managed by existing methods.
Innovation Solution
The approach decomposes convolutions into matrix multiplications, reducing data manipulations and redundancy by storing and processing input samples and kernels in a specific format that allows for parallel computations and efficient data reuse, avoiding the need for reordering and duplication of data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If convolutions are implemented using traditional methods on edge devices, then computational accuracy is maintained, but data transfer volume and memory usage increase significantly
Solution Approach 1:
The patent segments the convolution operation into two independent matrix multiplications: one for spatial convolution and one for channel convolution. This segmentation allows each matrix multiplication to process data independently, reducing the need to transfer and store the entire multi-channel input data repeatedly, thereby decreasing data transfer volume and memory usage while maintaining computational accuracy.
Solution Approach 2:
The patent transforms the traditional 2D spatial convolution into a combination of operations that leverage matrix multiplication in a different dimensional arrangement. By reorganizing the convolution computation into matrix multiplication form with specific dimension mappings, the system reduces data redundancy and improves memory access patterns, leading to lower data transfer requirements and enhanced efficiency.
2Productivity
If convolutions are implemented using traditional methods, then computational completeness is maintained, but computational overhead and processing time increase
Solution Approach 1:
The patent replaces the traditional mechanical convolution computation process with matrix multiplication operations. This substitution leverages the optimized performance of matrix multiplication hardware and algorithms, significantly reducing computational overhead and processing time while maintaining the same computational completeness and accuracy.
Solution Approach 2:
The patent performs preliminary reorganization of the input data into a format suitable for efficient matrix multiplication processing. By preparing the data in advance in the correct dimensional arrangement, the system eliminates redundant computations and optimizes memory access patterns during the actual convolution process, reducing overall processing time.
3Quantity of substance
If convolutions are implemented on edge devices, then resource constraints are acknowledged, but existing methods do not effectively manage memory usage
Solution Approach 1:
The patent segments the convolution operation into independent matrix multiplications that can be processed with smaller, more manageable data blocks. This segmentation enables the system to process convolutions within the memory constraints of edge devices without requiring excessive memory capacity, while the modular structure simplifies resource management through parallel processing capabilities.
Data Source
AI summary
Convolutions of an input sample with multiple kernels is decomposed into matrix multiplications of a V×C matrix of input values times a C×K matrix of kernel values, producing a V×K product. For the second matrix, C is a channel dimension (i.e., each row of the second matrix is a different channel of the input sample and kernel) and K is the kernel dimension (i.e., each column of the second matrix is a different kernel), but all the values correspond to the same pixel position in the kernel. In the matrix product, V is the output dimension and K is the kernel dimension. Thus, each value in the output matrix is a partial product for a certain output pixel and kernel, and the matrix multiplication parallelizes the convolutions by calculating partial products for multiple output pixels and multiple kernels.