CNN Layer Partitioning With Bias Accumulation for Faster Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In convolutional neural networks, dividing the network model into multiple pieces for processing increases the need for additional hardware resources and processing time due to the necessity of integrating convolution results and applying activation functions, especially when using hardware with multi-stage memory configurations.
Innovation Solution
The method involves dividing a convolutional neural network layer into sublayers in the channel direction, performing convolution processing on each sublayer, cumulatively adding convolution results with a bias using an adder, and inputting the final addition result to an activation function, thereby reducing hardware resource usage and processing time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of stationary object
If the network model is divided into multiple pieces for processing, then the memory size required can be reduced, but additional adder circuits and processing time are needed for integration
Solution Approach 1:
The patent merges the integration function into the existing adder circuit by having it accumulate convolution results across multiple processing steps. Instead of adding separate integration circuits, the adder circuit performs dual functions: adding bias values and accumulating convolution results from divided network pieces, thereby reducing hardware complexity while maintaining memory efficiency
Solution Approach 2:
The adder circuit is designed to perform multiple functions: it adds bias values to convolution results and also accumulates convolution results from multiple divided pieces of the network model. This multi-functionality eliminates the need for separate integration circuits, resolving the contradiction between memory reduction and hardware complexity
2Volume of stationary object
If the network model is divided into multiple pieces for processing, then the memory size required can be reduced, but processing time increases due to sequential reading and addition
Solution Approach 1:
The adder circuit continuously accumulates convolution results as they are generated from processing divided network pieces, without requiring sequential reading and processing. This continuous accumulation eliminates idle time and maintains steady processing flow, reducing overall processing time while keeping memory requirements low
3Productivity
If all convolution results are read from external memory for integration, then processing can be completed, but processing time increases
Solution Approach 1:
The system maintains running accumulation of convolution results in the adder circuit during the processing of divided network pieces. This preliminary accumulation eliminates the need for subsequent reading and addition operations, as the final accumulated value is already available when all pieces are processed, significantly reducing processing time
Data Source
AI summary
An inference processing device includes: a division unit that divides a layer of a convolutional neural network into a plurality of sublayers in a channel direction; a convolution unit that executes convolution processing for each of the sublayers to output a convolution result; an addition unit that adds an intermediate value obtained by cumulatively adding convolution results up to a previous sublayer to the convolution result with an adder for adding a bias to the convolution result every time the convolution processing is executed, and outputs an addition result; and an activation unit that inputs, to an activation function, the addition result obtained by adding the convolution result of a last sublayer on which the convolution processing has been executed last.