Dilated Convolution Acceleration via R×S Decomposition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for accelerating dilated convolution computation face challenges of high complexity and low generality, particularly in supporting various convolution kernel sizes, which increases memory usage and data access complexity.
Innovation Solution
The method decomposes a R×S dilated convolution operation into S R×1 sub-dilated convolution operations, caches weight values in parallel across computation units, performs sliding window and multiplication operations, and accumulates results to achieve the final convolution output, eliminating the need for the Im2col function and simplifying data access logic.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the internal memory is divided into many small blocks to support different convolution kernel sizes, then the generality for various convolution kernels is improved, but the area of internal memory increases and the complexity of data access logic increases
Solution Approach 1:
The patent segments the R×S dilated convolution operation into S separate R×1 sub-dilated convolution operations. Each computation unit handles one sub-convolution, which simplifies the data access pattern to sequential reading along rows. This segmentation resolves the contradiction by breaking down the complex multi-dimensional access into simpler one-dimensional access patterns, reducing both memory area requirements and data access logic complexity while maintaining support for various kernel sizes.
Solution Approach 2:
The patent transforms the two-dimensional R×S convolution kernel into S separate one-dimensional R×1 kernels by decomposing the operation dimensionally. This dimensionality change allows the system to use a simpler memory structure (one-dimensional access) instead of requiring complex two-dimensional memory blocking, thereby reducing memory area and access logic complexity while preserving the ability to handle different convolution configurations.
2Adaptability or versatility
If the internal memory is divided into many small blocks to support different convolution kernel sizes, then the generality for various convolution kernels is improved, but the area of internal memory increases
Solution Approach 1:
By segmenting the R×S convolution into S R×1 sub-convolutions, the patent reduces the memory requirements. Instead of needing a large two-dimensional memory block to store all possible kernel configurations, each computation unit only needs to store one-dimensional weight values, significantly reducing the total internal memory area while maintaining support for various kernel sizes through software configuration.
Solution Approach 2:
The patent changes the memory storage dimension from two-dimensional (R×S kernels) to one-dimensional (R×1 sub-kernels). This dimensional reduction allows the system to achieve generality for various convolution kernels without requiring proportionally large memory area, as the weight values can be stored and accessed in a more compact one-dimensional structure.
3Productivity
If conventional Im2col function is used to accelerate convolution computation, then the convolution operation is converted to matrix multiplication, but special design for arrangement of internal memory is required which increases complexity
Solution Approach 1:
The patent segments the convolution operation into independent R×1 sub-convolutions that can be processed in parallel by different computation units. Each sub-convolution operates on separate data streams, eliminating the need for complex Im2col memory rearrangement. This segmentation achieves parallelization and computational acceleration while maintaining simple sequential data access patterns, thus resolving the contradiction between productivity and complexity.
Solution Approach 2:
Instead of rearranging memory data into column-major format (Im2col approach), the patent inverts the approach by processing row-wise sub-convolutions in parallel. This inversion of the traditional optimization strategy achieves matrix multiplication equivalence without requiring special memory arrangement, thereby improving productivity while reducing data access logic complexity.
Data Source
AI summary
A dilated convolution acceleration calculation method and apparatus. The method comprises: decomposing a dilated convolution computation of R×S into S sub-dilated convolution computations of R×1 (301); for each sub-dilated convolution computation, caching a plurality of weight values in parallel to a plurality of calculation units in a calculation unit array (302); determining, from input image data, a plurality of input data streams respectively corresponding to the plurality of weight values, and inputting the plurality of input data streams in parallel into the plurality of calculation units (303); within the plurality of calculation units, executing a sliding window operation and a multiplication operation on the basis of the cached weight values and the input data streams, and executing an accumulation operation between the plurality of calculation units, so as to output an intermediate result of the sub-dilated convolution computation (304); and superimposing intermediate results of the S sub-dilated convolution computations of R×1, so as to obtain a convolution result of the dilated convolution computation (305). By using the method, a dilated convolution operation is accelerated with a relatively low complexity, and the function of Im2col does not need to be separately realized, thereby reducing the complexity.