Convolution Circuitry Factorization for Neural Network Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolution operations in neural networks are computationally intensive and require significant memory access, which can be inefficient due to the large number of weights and computations involved in traditional filter operations.
Innovation Solution
The method decomposes original larger convolution filters into smaller refactorization components, specifically K NxN first filters and C 1x1 second filters, to reduce computation and memory requirements by performing two smaller convolution operations that approximate the original operation, thereby reducing the number of multiply and accumulate operations and memory accesses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional large filter convolution operations are used, then the accuracy of output feature maps is maintained, but the computation and memory access requirements increase significantly
Solution Approach 1:
The patent divides a large CxNxN filter into multiple smaller filters by factorizing it into a product of smaller filter matrices. Specifically, the large filter is segmented into first filters of size KxNxN and second filters of size Cx1x1, where C>K. This segmentation reduces the computational complexity and memory access requirements while maintaining the same output accuracy, as the combined effect of smaller filters approximates the original large filter convolution operation
2Measurement precision
If the number of filters C is increased to improve feature extraction, then the accuracy improves, but the number of memory accesses and computations increases
Solution Approach 1:
The patent segments the filter set into two groups: first filters (KxNxN) and second filters (Cx1x1). By performing convolution with the first filters followed by the second filters, the system achieves efficient feature extraction with reduced computational overhead compared to directly applying C large filters, thus improving productivity while maintaining accuracy
Solution Approach 2:
The patent transforms the filter representation by introducing an intermediate dimension K. Instead of directly convolving with CxNxN filters, the system convolves with KxNxN filters first, then applies Cx1x1 filters to the intermediate results. This dimensional transformation reduces the total number of computations from O(C*N²) to O(K*N² + C*K), improving convolution efficiency
3Measurement precision
If filter size NxN is increased to capture more spatial information, then the measurement precision improves, but the device complexity and computation requirements worsen
Solution Approach 1:
The patent segments the spatial convolution operation into two stages: first convolving with KxNxN filters to capture spatial information, then convolving with Cx1x1 filters to process the intermediate features. This segmentation maintains the spatial feature detection capability of NxN filters while reducing the overall computation intensity by avoiding direct application of large filters to all C output channels simultaneously
4Ease of manufacture
If traditional convolution operations are performed without refactorization, then the implementation is simpler, but the memory storage needs and computation steps are higher
Solution Approach 1:
The patent implements a segmented filter architecture where large filters are factorized into smaller filter components. This segmentation reduces the memory storage requirements from storing CxNxN filters to storing KxNxN and Cx1x1 filters, and reduces computation resources by breaking down the convolution into two smaller operations instead of one large operation, while maintaining implementation feasibility
Data Source
AI summary
Disclosed approaches for convolving input feature maps in a neural network include a circuit arrangement circuit that includes memory circuitry and convolution circuitry. The memory circuitry is configured to store K NxN first filters, and C 1x1 second filters, wherein N ≥ 1, and 1 < K < C. The convolution circuitry is coupled to the memory circuitry and configured to convolve a three-dimensional input feature map with the K NxN first filters into an intermediate volume having a depth of K, and convolve the intermediate volume with the C 1x1 second filters into an output feature map having a depth of C.