Fixed-Point BatchNorm Quantization for Accurate GEMM Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional fixed-point batchnorm layer implementations in neural networks face performance and accuracy issues, particularly in converting floating-point to fixed-point data, with existing methods failing to maintain accuracy in nonlinear operations like Batchnorm.
Innovation Solution
Convert standalone floating-point Batchnorm layers to fixed-point layers by generating unique quantization parameters per channel, enabling matrix multiplication on GEMM engines for improved computation efficiency and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional fixed-point batchnorm implementation is used, then computation can be performed in fixed-point format, but accuracy is degraded due to quantization errors in nonlinear operations
Solution Approach 1:
The patent introduces an intermediary conversion process that transforms the fixed-point batchnorm computation into an equivalent floating-point computation using quantization parameters. The fixed-point input x_q is converted to floating-point x via x = x_q * scale + zero_point, allowing accurate batchnorm computation in floating-point while accepting fixed-point input, thus eliminating quantization errors in nonlinear operations
Solution Approach 2:
The patent changes the computational parameters by introducing floating-point quantization parameters (scale and zero_point) that bridge fixed-point and floating-point domains. By computing batchnorm in floating-point using these parameters and then converting back to fixed-point, the system maintains accuracy while preserving fixed-point input/output compatibility
2Productivity
If standalone floating-point batchnorm layer is used, then accuracy can be maintained, but performance is degraded due to inability to utilize efficient GEMM engines
Solution Approach 1:
The patent merges the standalone batchnorm layer with the preceding convolution layer by integrating batchnorm parameters (scale, zero_point) into the convolution layer's weight and bias parameters. This combination allows the merged layer to be computed using efficient GEMM (General Matrix Multiplication) engines, significantly improving computation performance while maintaining batchnorm functionality
Solution Approach 2:
The patent creates a multi-functional layer that simultaneously performs convolution and batchnorm operations. The merged layer can be computed using standard GEMM operations, making it compatible with efficient matrix multiplication hardware while preserving both feature extraction (convolution) and normalization (batchnorm) functionalities
Data Source
AI summary
A method to implement a fixed-point batchnorm layer in a neural network for data processing is provided in the present disclosure. The method includes: receiving fixed-point input data over a channel of a standalone floating-point batchnorm layer, and converting the floating-point input data into fixed-point input data of the standalone floating-point batchnorm layer; obtaining fixed-point quantization parameters in each channel based on the input data and floating-point parameters μi, σi, εi in each channel; converting the standalone floating-point batchnorm layer based on the fixed-point quantization parameters into a fixed-point batchnorm layer for processing the fixed-point input data to generate fixed-point output data; and mapping the fixed-point batchnorm layer to a fixed-point convolution layer and the computation of convolution is done by matrix multiplication that can be executed on a GEMM engine.