Convolution Streaming Engine Matrix Decomposition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning neural networks, particularly convolutional neural networks, face significant computational inefficiencies due to the high computational load of convolutional layers, which account for over 90% of total computations, regardless of configuration parameters like feature map size, kernel size, and stride.
Innovation Solution
The implementation of a convolution streaming engine that converts convolution operations into matrix multiplication operations by decomposing inputs into panel matrices, which are then processed by a matrix multiplication engine, decoupling compute efficiency from specific neural network configurations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If convolution operations are performed using traditional methods in neural networks, then the computational accuracy is maintained, but the computational efficiency deteriorates due to the high computational load of convolutional layers
Solution Approach 1:
The patent segments the input feature map into multiple non-overlapping regions or panels, allowing the convolution operation to be divided into multiple independent matrix multiplication operations. This segmentation enables parallel processing and reduces the overall computational complexity, directly addressing the efficiency-time tradeoff by breaking down the large-scale convolution into smaller, manageable units that can be processed faster
Solution Approach 2:
The patent substitutes the traditional mechanical convolution operation (sliding window approach) with matrix multiplication operations. By reformulating the convolution as matrix multiplications between kernel matrices and panel matrices, the system achieves higher computational efficiency while maintaining the same mathematical equivalence, thereby reducing processing time without sacrificing accuracy
2Adaptability or versatility
If the neural network uses arbitrary configurations of feature map size, kernel size, and stride, then the model's adaptability is improved, but the computational complexity increases making efficiency evaluation difficult
Solution Approach 1:
The patent creates a universal matrix multiplication framework that can handle arbitrary neural network configurations (different feature map sizes, kernel sizes, and strides) through a single unified approach. The method universally applies panel decomposition and matrix multiplication regardless of specific configuration parameters, allowing the system to maintain adaptability while simplifying the computational complexity through a standardized processing pipeline
Solution Approach 2:
The patent changes the parameter representation by transforming the convolution operation parameters (feature map size, kernel size, stride) into matrix dimensions for the panel decomposition process. This parameter transformation allows arbitrary configurations to be handled systematically through matrix operations, reducing the perceived complexity while maintaining configuration flexibility
Data Source
AI summary
A method, an electronic device, and computer readable medium are provided. The method includes receiving an input into a neural network that includes a kernel. The method also includes generating, during a convolution operation of the neural network, multiple panel matrices based on different portions of the input. The method additionally includes successively combining each of the multiple panel matrices with the kernel to generate an output. Generating the multiple panel matrices can include mapping elements within a moving window of the input onto columns of an indexing matrix, where a size of the window corresponds to the size of the kernel.