CNN Accelerator Weight Classification and Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hardware accelerators for neural networks that apply both quantization and pruning techniques do not maximize optimization benefits, leading to increased hardware saturation but negligible scheduling effects for sparse matrices.
Innovation Solution
A method optimizing convolutional layer calculations by classifying weights as outlier and non-outlier elements, combining non-outlier elements with the same activation value, and scheduling operations to move elements within a fetch window to unassigned slots, along with a CNN accelerator design incorporating processing elements, activation, and weight buffers to perform efficient multiplications and accumulations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If quantization and pruning techniques are applied to reduce the number of weights and calculations, then data size and computational complexity are reduced, but hardware saturation increases and scheduling effects become negligible
Solution Approach 1:
The patent segments weights into two categories: non-outlier (NOL) weights that are combined and processed together, and outlier (OL) weights that are processed separately. This segmentation allows the system to handle sparse matrices more effectively by processing NOL weights in batches, reducing hardware saturation while maintaining computational efficiency.
Solution Approach 2:
The patent merges multiple NOL elements that have the same activation value and are located at the same slot across different channels into a single combined element. This combining operation reduces the number of individual weight processing operations, thereby reducing hardware saturation and making scheduling effects more significant for optimization.
2Speed
If scheduling is used to increase hardware saturation and accelerate calculation, then calculation speed improves, but the benefit is negligible for sparse matrices with both quantization and pruning
Solution Approach 1:
The patent introduces a dynamic scheduling mechanism that adapts to the sparse matrix structure. The scheduling operation moves scheduling candidate elements in a fetch window to unassigned slots based on the combined NOL elements, creating a dynamic allocation strategy that maximizes hardware utilization while accounting for the sparsity introduced by quantization and pruning.
3Quantity of substance
If data quantization is applied to lower the resolution of data expressions, then data size is reduced, but data accuracy degrades
Solution Approach 1:
The patent applies different processing qualities to different weight elements. NOL elements are processed with standard quantization for size reduction, while OL elements are handled separately with potentially higher precision or special processing. This local quality differentiation allows the system to reduce overall data size while preserving accuracy where it matters most.
Data Source
AI summary
Provided is a method of optimizing calculation of a convolutional layer including a plurality of channels and a convolutional neural network (CNN) accelerator. The method includes a classification operation of classifying weights of the plurality of channels as outlier (OL) elements and non-outlier (NOL) elements, a combination operation of combining two or more NOL elements which are put into calculation with the same activation value and present at the same slot in the plurality of channels, and a scheduling operation of moving a scheduling candidate element in a fetch window after the combination operation to a slot which is assigned no value.