Fixed-Point Quantization for Complex Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Complex neural networks, such as GoogLeNet and SqueezeNet, with merged branches and parallel sub-networks, pose challenges in fixed-point data quantization due to inconsistencies in fixed-point ranges across concatenated layers, leading to inaccuracies and resource inefficiencies, which existing methods struggle to address effectively.
Innovation Solution
A method that involves fixed-point data quantization, sub-network identification, and dynamic fixed-point range modification for each layer, particularly for convolutional and fully connected layers, ensures consistent fixed-point representation across concatenated outputs, optimizing resource usage and accuracy by dynamically adjusting the bit width for each layer based on the concatenated layer's range.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If fixed-point data quantization is applied to complex neural networks with merged branches and parallel sub-networks, then resource usage is reduced and efficiency is improved, but inconsistencies in fixed-point ranges across concatenated layers cause inaccuracies and require complex coordination
Solution Approach 1:
The patent changes the fixed-point parameters (bit width and range) of layers based on their position in the network and their concatenation relationships. By dynamically adjusting these parameters to match the concatenated layer's range, the system maintains representation accuracy while optimizing for reduced resource usage in complex neural networks
Solution Approach 2:
The patent introduces dynamic fixed-point range modification where the bit width and range of each layer are not static but are adjusted according to the concatenated layer's requirements. This dynamic adaptation allows the system to handle the varying precision requirements across different parts of the complex network while maintaining overall accuracy
2Measurement precision
If fixed-point range is modified for each layer to maintain consistency, then representation accuracy is maintained, but computational complexity and processing overhead increase
Solution Approach 1:
The patent performs fixed-point range modification during the quantization process itself, before the neural network is deployed. By pre-adjusting the bit width and range of each layer to match their concatenated layers' requirements, the system eliminates the need for complex runtime adjustments while maintaining representation accuracy
Solution Approach 2:
The patent applies different fixed-point parameters (bit width and range) to different layers based on their specific requirements and concatenation relationships. Instead of using a uniform quantization scheme, each layer is optimized locally with appropriate precision levels, reducing overall complexity while maintaining necessary accuracy
Data Source
AI summary
Systems, apparatus and methods are provided for accelerating a complex neural network by fixed-point data quantization. An Artificial Neural Network (ANN) has branches and comprises convolutional layers CONV 1, CONV 2, . . . CONV n, fully connected layers FC 1, FC 2, . . . , FC m, and concatenation layers CONCAT1, CONCAT2, . . . , CONCAT L. n, m and L are positive integers. The ANN may be optimized by a method comprising: converting output of each of the CONV, FC and CONCAT layers into fixed-point numbers, identifying at least one sub-network from the ANN and for each sub-network, modifying the fixed-point range of each output of the previous-level layers of the CONCAT layer on the basis of the fixed-point range of the CONCAT layer. The sub-network has a CONCAT layer as its output. The CONCAT layer receives at least two outputs of previous-level layers as inputs and concatenates the inputs into one output.