2D Convolution Mapping on Dot Product Architecture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing processing circuits are inefficient in performing two-dimensional (2D) convolutional layer operations on a dot product architecture, leading to slow computation and high energy consumption, particularly in resource-constrained devices.
Innovation Solution
A processing circuit that accelerates 2D convolutional layer mapping by storing convolution kernels and input feature maps in dedicated storage devices, performing simultaneous multiplications and accumulations using a multiply-accumulate (MAC) tile architecture, and optimizing memory access through caching and broadcasting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If general purpose central processing unit or graphics processing unit is used to perform tensor operations, then the operations can be performed with flexible hardware, but the computation speed is slow and energy consumption is high
Solution Approach 1:
The processing circuit is divided into multiple processing elements (PEs) organized in a grid architecture, where each PE handles specific portions of the tensor operations. This segmentation allows parallel processing of different elements of the input feature map and convolution kernel, significantly improving computation speed while maintaining hardware efficiency.
Solution Approach 2:
The patent transforms the traditional 2D convolution operation into a series of dot product operations by unfolding the convolution kernel and input feature map into 1D vectors. This dimensional transformation allows the use of efficient dot product hardware units to perform 2D convolution, achieving high-speed computation with reduced energy consumption.
2Adaptability or versatility
If general purpose central processing unit or graphics processing unit is used to perform tensor operations, then the operations can be performed with general purpose hardware, but the energy consumption per operation is high
Solution Approach 1:
The processing circuit incorporates on-chip storage devices that are directly integrated with the processing elements. These storage devices automatically cache the convolution kernel and input feature map data, reducing the need for repeated memory accesses and thereby significantly reducing energy consumption. The system serves itself by maintaining necessary data locally without requiring external memory operations.
Solution Approach 2:
The patent implements preliminary loading and caching of the convolution kernel and input feature map into on-chip storage devices before the actual computation begins. This preliminary action ensures that all necessary data is readily available in fast, energy-efficient on-chip memory, avoiding energy-intensive external memory accesses during the computation phase.
3Productivity
If 2D convolutional layer mapping is performed on dot product architecture, then computational efficiency can be improved, but the mapping complexity increases
Solution Approach 1:
The patent employs dynamic data flow control and adaptive mapping strategies that adjust the computation pipeline based on the specific dimensions and characteristics of the input tensors. The processing elements dynamically allocate their computational resources and the data flow is adaptively routed through the network, simplifying the mapping process while maintaining high computational efficiency.
Solution Approach 2:
The processing elements are designed with universal functionality to handle various tensor operation types and dimensions. Each PE can perform dot product operations, accumulate results, and handle different data formats, allowing the same hardware structure to efficiently map 2D convolution operations without requiring complex specialized circuits for each operation type.
Data Source
AI summary
A method for performing a convolution operation includes storing, a convolution kernel in a first storage device, the convolution kernel having dimensions x by y; storing, in a second storage device, a first subset of element values of an input feature map having dimensions n by m; performing a first simultaneous multiplication, of each value of the first subset of element values of the input feature map with a first element value from among the x*y elements of the convolution kernel; for each remaining value of the x*y elements of the convolution kernel, performing, a simultaneous multiplication of the remaining value with a corresponding subset of element values of the input feature map; for each simultaneous multiplication, storing, result of the simultaneous multiplication in an accumulator; and outputting, the values of the accumulator as a first row of an output feature map.