Autoencoder Path Quantization for Skip Connection Compatibility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks face inefficiencies due to the time-consuming nature of operations in floating point formats, leading to the need for quantization to convert data into integer-based formats for faster processing, but this often requires additional quantization operations when combining paths, increasing computational complexity.
Innovation Solution
A system determines a shared set of quantization parameters for multiple paths in a neural network, allowing outputs to be combined without additional quantization operations, thereby reducing computational complexity and improving inference performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If quantization is applied to convert floating point data to integer-based format, then processing speed is improved, but additional quantization operations are required when combining paths, increasing computational complexity
Solution Approach 1:
The patent changes the quantization parameters (scale and zero-point values) of different paths to be compatible with each other. By adjusting these parameters, the patent enables direct concatenation of quantized tensors from multiple paths without requiring additional quantization operations, thus reducing computational complexity while maintaining processing speed improvements
Solution Approach 2:
The patent creates a universal quantization parameter set that can be applied across multiple paths in the neural network. This universal parameter configuration allows all paths to output tensors in a compatible format, enabling them to be combined directly without path-specific quantization operations, thereby reducing overall computational complexity
2Reliability
If quantization operations are performed at each path combination point, then data format compatibility is maintained, but the number of operations increases, reducing inference performance
Solution Approach 1:
The patent performs preliminary configuration of quantization parameters during the network design or training phase, establishing compatible scale and zero-point values for all paths before inference occurs. This preliminary setup ensures that during inference, quantized tensors from different paths can be directly concatenated without requiring additional quantization operations, thus maintaining data format compatibility while improving inference performance
3Measurement precision
If floating point format is used during training operations, then data precision is maintained, but processing time increases
Solution Approach 1:
The patent changes the data format parameter from floating point to integer-based quantized format during inference operations. By carefully configuring the quantization parameters to maintain compatibility and precision, the patent achieves both reduced processing time and maintained data precision, resolving the trade-off between these two parameters
Data Source
AI summary
The performance of a neural network is improved by applying quantization to data at various points in the network. In an embodiment, a neural network includes two paths. A quantization is applied to each path, such that when an output from each path is combined, further quantization is not required. In an embodiment, the neural network is an autoencoder that includes at least one skip connection. In an embodiment, the system determines a set of quantization parameters based on the characteristics of the data in the primary path and in the skip connection, such that both network paths produce output data in the same fixed point format. As a result, the data from both network paths can be combined without requiring an additional quantization.