Adaptive 3D Multiplier Layer for Convolutional Neural Network Loop Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks face inefficiencies in computation due to the need for eight layers of nested loops to process multi-dimensional activation and filter data, leading to high computational complexity and resource utilization issues.
Innovation Solution
An adaptive multiply accumulate layer is constructed by determining and setting dimensions based on activation data map width, channel depth, and filter set number, allowing for the creation of a 3D array of multipliers that can be adjusted to optimize processing efficiency, reducing the number of loops and improving resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If eight layers of nested loops are used to process multi-dimensional activation and filter data, then the convolution operation can be performed, but the computational complexity and resource utilization increase significantly
Solution Approach 1:
The patent segments the eight-layer nested loop structure into multiple independent processing stages. By dividing the computation into separate dimension processing steps (spatial dimensions, channel dimensions, batch dimensions), the complex nested structure is broken down into manageable segments that can be processed more efficiently with reduced inter-dependency between loops.
Solution Approach 2:
The patent transforms the computational approach by introducing additional processing dimensions. Instead of purely iterative loop nesting, the invention reorganizes data layout and processing order across multiple dimensions (e.g., transforming between NCHW and NHWC formats), enabling parallel processing paths that reduce the effective depth of nested loops required.
2Reliability
If eight layers of nested loops are used to process multi-dimensional activation and filter data, then the convolution operation can be performed, but the processing time increases
Solution Approach 1:
The patent performs preliminary data layout transformations and pre-computation steps before entering the main nested loop structure. By pre-organizing activation and filter data into optimized formats and pre-computing certain intermediate values, the actual convolution computation requires fewer iterative steps and less time within the nested loops.
Solution Approach 2:
The patent ensures continuous utilization of processing resources by overlapping data transfer, computation, and transformation operations. The reorganized loop structure enables pipelining where different stages of convolution computation for different batches or channels can proceed concurrently, reducing idle time and improving overall processing throughput.
3Ease of manufacture
If traditional nested loop structure is used, then convolution processing can be implemented, but resource utilization is inefficient
Solution Approach 1:
The patent introduces dynamic data layout transformations that adapt the memory and computation layout to match the processing stage. By dynamically switching between different data format representations (NCHW, NHWC, and intermediate formats) at different stages of computation, the system optimizes resource utilization for each specific operation while maintaining implementation feasibility through systematic transformation rules.
Data Source
AI summary
A method of constructing an adaptive multiply accumulate layer in a convolutional neural network, including determining an activation data map width, an activation data map height, a channel depth, a batch, a kernel width, a kernel height and a filter set number, setting a first dimension of an adaptive multiplier layer based on the activation data map width, setting a second dimension of the adaptive multiplier layer based on the channel depth, setting a third dimension of the adaptive multiplier layer based on the filter set number and constructing the adaptive multiplier layer based on the first dimension, the second dimension and the third dimension.