Neural Network Weight Clustering for Low-Bit Quantization Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional neural network quantization methods fail to exploit the similarity of inter-layer-based weights and intra-layer-based weights, leading to reduced accuracy when representing weights with lower bits.
Innovation Solution
A data quantization method involving grouping neural network weights, performing clustering to determine center weights for each class, and encoding these weights to create a weight codebook, with retraining focused only on the codebook using algorithms like K-means or Clarans.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If traditional quantization method is used with layer as unit, then quantization process is simple, but accuracy of neural network is reduced when weight is represented by lower bits
Solution Approach 1:
The patent segments the quantization process into two distinct stages: clustering (performed once during model preparation) and quantization (performed during inference). This segmentation allows the clustering operation to capture weight distribution characteristics across the entire network, while the quantization stage can efficiently operate on pre-clustered weights, resolving the contradiction between simplicity and accuracy.
Solution Approach 2:
The patent performs clustering as a preliminary action before quantization. By pre-processing the weights to identify clusters and their centers, the system prepares the weight data in advance to optimize subsequent quantization operations. This preliminary clustering enables lower-bit representation while preserving accuracy, as the quantization process can directly map weights to nearest cluster centers.
2Quantity of substance
If low bit quantization is performed without exploiting weight distribution characteristics, then storage overhead is reduced, but accuracy is reduced
Solution Approach 1:
The patent changes the parameter representation by introducing cluster centers as intermediate parameters. Instead of directly quantizing individual weights to low-bit representations, the system represents weights through their cluster center assignments. This parameter transformation enables more efficient low-bit storage while maintaining accuracy through the clustering structure.
Solution Approach 2:
The patent creates a composite representation system combining cluster assignments and cluster center values. This composite approach allows the system to exploit weight distribution characteristics by storing both the discrete cluster indices (low-bit) and the continuous cluster center values, achieving a balance between storage efficiency and representation accuracy.
3Measurement precision
If clustering operation is performed on all weights, then weight distribution characteristics are fully exploited, but computational complexity increases
Solution Approach 1:
The patent segments the weight matrix into multiple groups or blocks before performing clustering operations. This segmentation reduces the computational complexity of clustering by applying it to smaller subsets of weights rather than the entire weight matrix at once, while still capturing local weight distribution characteristics that are important for accuracy.
Solution Approach 2:
The patent applies clustering operations selectively to only those weight groups that benefit most from clustering, rather than uniformly applying clustering to all weights. This partial action approach reduces overall computational complexity while maintaining quantization accuracy for the most critical weight regions.
Data Source
AI summary
The present disclosure provides a processing device including: a coarse-grained pruning unit configured to perform coarse-grained pruning on a weight of a neural network to obtain a pruned weight, an operation unit configured to train the neural network according to the pruned weight. The coarse-grained pruning unit is specifically configured to select M weights from the weights of the neural network through a sliding window, and when the M weights meet a preset condition, all or part of the M weights may be set to 0. The processing device can reduce the memory access while reducing the amount of computation, thereby obtaining an acceleration ratio and reducing energy consumption.