CNN Convolution Core Using 3×3 Segmentation for Small FPGAs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Implementing large Convolution Neural Networks (CNNs) on small Programmable Logic Devices (PLDs) such as FPGAs and ASICs is challenging due to the high power consumption and cost associated with the large number of memory blocks, multipliers, and registers required for larger matrix sizes, making it difficult to implement CNNs larger than 3×3.
Innovation Solution
Divide the weights matrix of an N×N convolution into a plurality of 3×3 weights matrices, perform 3×3 convolutions on image values, and sum the results to generate an N×N convolution value, using a 3×3 CNN core in an IC device or PLD, reducing the need for large memory blocks and multipliers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large N×N convolution matrixes are implemented directly, then convolution accuracy is maintained, but power consumption and device cost increase significantly
Solution Approach 1:
The patent divides a large N×N convolution operation into multiple 3×3 convolution operations. The weights matrix is segmented into multiple 3×3 sub-matrices, and the input image is divided into corresponding regions. Each 3×3 convolution is performed independently using dedicated hardware circuits, and the results are summed to produce the final N×N convolution output. This segmentation reduces the power consumption of individual convolution units while maintaining overall accuracy through proper summation of results.
2Measurement precision
If large N×N convolution matrixes are implemented directly, then convolution accuracy is maintained, but device complexity and cost increase
Solution Approach 1:
The patent segments the large N×N convolution into multiple 3×3 convolutions. Each 3×3 convolution uses a dedicated hardware circuit with minimal memory blocks and multipliers. The weights matrix is divided into multiple 3×3 sub-matrices stored in separate memory blocks, and each sub-matrix is processed by a corresponding 3×3 convolution unit. This segmentation dramatically reduces the complexity of individual hardware units compared to implementing a single large N×N convolution unit.
Solution Approach 2:
The patent employs multiple identical 3×3 convolution circuits that can be reused across different spatial locations of the input image. Each 3×3 convolution unit serves multiple purposes by processing different 3×3 regions of the input image, effectively reducing the total number of unique hardware components needed compared to implementing separate large convolution units for each spatial location.
3Use of energy by moving object
If the number of memory blocks and registers is reduced, then power consumption decreases, but implementation of large CNNs becomes difficult
Solution Approach 1:
The patent segments the large N×N convolution into multiple 3×3 convolutions, each requiring minimal memory blocks and registers. The weights matrix is divided into multiple 3×3 sub-matrices that can be stored in small, manageable memory blocks. Each 3×3 convolution unit requires only a small number of registers and multipliers, making the overall system implementable on small FPGAs and ASICs with limited resources while maintaining low power consumption.
Data Source
AI summary
A method and apparatus for performing a convolution of a N×N matrix. A weights matrix for a N×N Convolutional Neural Network (CNN) is received and is divided into 3×3 weights matrixes. Lines of image values are read and are stored in a buffer as sets of image values. A 3×3 convolution is performed to generate a 3×3 convolution value. All 3×3 convolution values associated with a particular N×N convolution and a particular set of image values are summed. The 3×3 convolutions and the summing are repeated until all columns in the set of image values have been processed; and the reading, the storing, the performing 3×3 convolutions, the summing and the repeating performing 3×3 convolutions are repeated until all lines of image values have been processed. The sums associated with a particular N×N convolution are added together to generate an N×N convolution value for each of the N×N convolutions.