Decomposing Neural Network Layers for Weight Quantization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks require a large number of weights, which can lead to inefficiencies in terms of memory and computational resources, and existing techniques for reducing weights, such as sparsity, are not fully effective, especially in larger layers.
Innovation Solution
The approach involves decomposing a network layer into two successive layers, where the second layer has fewer filters but the same size as the original, and the third layer has smaller filters, allowing for reduced weight requirements while maintaining output feature map structure, combined with weight and output value quantization to further reduce resource needs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the number of weights in neural network layers is reduced using sparsity techniques, then memory and computational resources are saved, but the effectiveness and accuracy of the network deteriorates
Solution Approach 1:
The patent applies segmentation by decomposing each original layer into multiple sub-layers (typically two or three sub-layers). Each sub-layer contains a subset of the original filters, and the filters are distributed across sub-layers such that their combined computational capacity equals or exceeds the original layer. This segmentation allows the network to maintain accuracy while reducing the number of weights in each individual layer, thereby saving memory and computational resources.
2Quantity of substance
If layer decomposition is applied to reduce weights, then memory usage decreases, but the device complexity increases
Solution Approach 1:
The patent segments each layer into multiple sub-layers with distributed filters, which reduces memory usage by lowering the peak memory requirement for weight storage. The segmentation is designed so that the sum of weights across all sub-layers equals the original layer's weights, achieving memory efficiency.
Solution Approach 2:
The decomposed sub-layers use the same fundamental convolutional operations as the original layers, maintaining universality in the computational approach. The same filter decomposition strategy can be applied to any convolutional layer in the network, making the technique universally applicable across different network architectures and layers.
3Quantity of substance
If quantization is applied to weights and output values, then resource requirements are reduced, but training complexity increases
Solution Approach 1:
The patent applies parameter changes by quantizing the weight values and output values to discrete levels. Weights are quantized to low precision (e.g., 4-bit or 8-bit integers), and output values are similarly quantized. This parameter change reduces memory requirements and computational resource usage while enabling efficient implementation on specialized hardware.
Solution Approach 2:
The quantization process is applied separately to each decomposed sub-layer, allowing independent optimization and training. The segmentation into sub-layers enables the quantization to be managed in smaller, more tractable units, reducing the overall training complexity compared to quantizing the entire layer at once.
Data Source
AI summary
Some embodiments provide a method for training parameters of a network. the method receives a machine-trained (MT) network with multiple layers of computation nodes. Each computation node of a set of the layers computes an output value based on a set of input values and a set of trained weight values. A first layer of the MT network includes a first number of filters. The method replaces the first layer with (i) a second layer having a second number of filters that is less than the first number of filters and (ii) a third layer having the first number of filters. Output values of computation nodes of the second layer are quantized and the third layer using the quantized output values of the second layer as input values.