Neural Network Inter-Layer Format Conversion for Predictable Quantization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network hardware accelerators face inefficiencies due to varying accuracy and unpredictable performance across layers, requiring manual human intervention for inter-layer data format conversion, which is time-consuming and sub-optimal.
Innovation Solution
Automated inter-layer data format conversion using data statistics to determine conversion parameters, reducing the need for human intervention and improving efficiency in neural network quantization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual human intervention is used for inter-layer data format conversion, then conversion accuracy can be optimized, but time consumption increases significantly
Solution Approach 1:
The system automatically determines data format conversion parameters by having the neural network layers self-report their input and output format requirements. The hardware accelerator autonomously selects appropriate conversion methods based on these parameters without human intervention, resolving the contradiction between accuracy and time consumption by enabling automated self-service format conversion.
Solution Approach 2:
The system changes the parameters of data format conversion by dynamically selecting different conversion methods (linear scaling, quantization, dequantization) based on the specific input-output format parameters reported by neural network layers. This allows accurate conversion adapted to each layer's requirements while maintaining automated operation.
2Ease of manufacture
If the same number of bits is used for all layers, then implementation is simple, but accuracy varies and becomes unpredictable across different layers
Solution Approach 1:
The system applies different bit precision and data format parameters to different layers based on their specific requirements. Each layer reports its optimal input and output formats, and the hardware accelerator configures layer-specific conversion parameters, ensuring each layer operates with appropriate precision while maintaining overall system manageability through automated configuration.
3Productivity
If multiple number formats are supported across layers, then computation efficiency improves, but the complexity of inter-layer data format conversion increases
Solution Approach 1:
The hardware accelerator is designed with universal data format conversion capabilities that can handle multiple number formats (FP32, FP16, INT8, etc.) through a unified conversion framework. The system uses standardized conversion parameters and methods that work across different format combinations, reducing conversion complexity while maintaining support for multiple formats to improve computation efficiency.
Data Source
AI summary
The present disclosure relates to a method of inter-layer format conversion for a neural network, the neural network comprising at least two computation layers including a first layer to process first data in a first data format and a second layer to process second data in a second data format, the method comprising: extracting data statistics from data output by the first layer, said data statistics being representative of the data output by the first layer; determining one or more conversion parameters based on the extracted data statistics and the second data format; and generating the second data for the second layer by modifying said data output by the first layer using the one or more conversion parameters.