Convolutional Neural Network Acceleration via Weight Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network chips face challenges in achieving versatility and low complexity in accelerating convolutional neural networks due to the need for specialized memory layouts to support various convolutional kernel sizes, leading to increased memory area and data access complexity.
Innovation Solution
The method involves splitting the weight matrix of a convolutional layer into segments and caching them in computation units, allowing for parallel processing of input data streams formed by concatenating multiple rows from the input feature map, with sliding window operations and multiply-accumulating computations performed on each segment to generate output feature maps, eliminating the need for the Im2col function and reducing hardware complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the Im2col function is used to optimize convolution computation, then computation efficiency is improved, but memory area and data access complexity increase due to specialized memory layout requirements
Solution Approach 1:
The patent segments the weight matrix into multiple weight segments and caches them in different computation units. This segmentation allows the system to process different segments in parallel while using a unified memory layout, avoiding the need for specialized segmented memory structures required by traditional Im2col approaches.
Solution Approach 2:
The patent changes the processing dimension by reading multiple rows of input data simultaneously and concatenating them to form input data streams. This dimensional transformation allows the system to achieve efficient parallel processing without requiring complex column-wise memory access patterns.
2Speed
If internal memory is segmented to improve parallelism of accessing, then data access parallelism is improved, but memory area increases
Solution Approach 1:
The patent designs a unified memory layout that can serve multiple purposes: storing both input feature maps and weight matrices, and supporting various convolution kernel sizes without requiring separate memory segments. This universal memory structure eliminates the need for additional segmented memory while maintaining parallel access efficiency.
3Adaptability or versatility
If internal memory is segmented into many small segments to support various convolutional kernels, then versatility is improved, but data access logic complexity increases
Solution Approach 1:
The patent implements a dynamic processing approach where the system can handle various convolution kernel sizes using the same unified memory layout by dynamically adjusting the processing parameters. The computation units can adaptively process different kernel sizes without requiring static memory segmentation, simplifying the data access logic while maintaining versatility.
Data Source
AI summary
A method and apparatus for accelerating a convolutional neural network. The method comprises: splitting, according to rows, a weight matrix of a convolutional layer into a plurality of weight segments, and respectively caching the plurality of weight segments to a plurality of calculation units in a calculation unit array (step 301); reading a plurality of input data streams respectively corresponding to the plurality of weight segments, and inputting the plurality of input data streams in parallel into the plurality of calculation units (step 302), wherein the input data streams are formed by means of splicing a plurality of rows of data in an input feature map of the convolutional layer; and within each calculation unit, performing a sliding window operation and a multiply-accumulate computation on the input data streams on the basis of the cached weight segments, so as to obtain an output feature map of the convolutional layer (step 303). By means of the method, data that enters each row of calculation units is continuously read according to the whole row of data, and does not need to be read in a cross-row or cross-column manner, such that a special design is not needed for the arrangement of a memory, convolutions of different sizes are supported, and the function of Im2col does not need to be realized, thereby reducing the complexity.