Neural Network Forward Propagation With Few-Bit Fixed-Point Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional 32-bit floating point data representation in artificial neural networks leads to redundant data storage, increasing hardware area overhead and power consumption, as it often exceeds the necessary data range for forward propagation processes.
Innovation Solution
A device and method for performing forward propagation using a floating point data statistics module to determine the decimal point location, converting long-bit floating-point data to short-bit fixed-point data, and employing rounding units to optimize data representation, thereby reducing storage and power requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 32-bit floating point data type is used for data storage, then recognition accuracy is maintained, but hardware area overhead increases due to redundant data representation
Solution Approach 1:
The patent changes the data representation parameters from 32-bit floating point to variable-bit fixed point formats (such as 8-bit, 16-bit, or 32-bit fixed point). By analyzing the actual data distribution and determining optimal decimal point positions, the system maintains sufficient precision for neural network operations while dramatically reducing the bit width required for data storage and transmission, thus reducing hardware area overhead.
Solution Approach 2:
The patent applies partial precision reduction by determining the minimum necessary precision for each layer and type of data in the neural network. Instead of uniformly reducing precision across all data, the system analyzes actual data ranges and distributions to determine the optimal fixed-point format for each specific case, using only the necessary precision required for accurate computation while discarding redundant high-precision representation.
2Adaptability or versatility
If 32-bit floating point data type is used, then data range coverage is sufficient, but power consumption increases due to redundant data processing
Solution Approach 1:
The patent changes the data representation parameters from 32-bit floating point to variable-bit fixed point formats. By analyzing actual data distributions and determining optimal decimal point positions for different layers and data types, the system maintains adequate data range coverage while reducing the number of bits that need to be processed, transmitted, and stored, thereby reducing power consumption in neural network accelerators.
Solution Approach 2:
The patent extracts and removes the redundant high-precision representation from the data format. By identifying and eliminating the unnecessary most significant bits and least significant bits based on actual data distribution analysis, the system retains only the essential precision required for accurate neural network computation, reducing power consumption while maintaining data range coverage.
3Area of stationary object
If long-bit floating-point data is converted to short-bit fixed-point data, then hardware area overhead is reduced, but data precision may be lost
Solution Approach 1:
The patent systematically changes the data representation parameters by analyzing actual data distributions to determine optimal fixed-point formats. The system determines the appropriate number of integer bits and fractional bits based on the minimum and maximum values, data ranges, and distribution characteristics of weights, biases, and activation values in each neural network layer, ensuring sufficient precision is retained while minimizing bit width.
Solution Approach 2:
The patent performs preliminary analysis of data distributions and determines optimal fixed-point conversion parameters before actual neural network inference or training. By pre-calculating the appropriate decimal point positions and bit allocations for each layer and data type, the system ensures that precision requirements are met while maximizing compression, avoiding precision loss during the conversion process.
4Quantity of substance
If short-bit fixed-point data is used for forward propagation, then transmission bandwidth requirements are reduced, but rounding errors may increase
Solution Approach 1:
The patent changes the data representation to optimized fixed-point formats with carefully determined decimal point positions. By analyzing data distributions and selecting appropriate integer and fractional bit allocations, the system minimizes the impact of rounding errors while achieving significant bandwidth reduction. The fractional bit allocation is specifically optimized to preserve the precision needed for accurate gradient computation and weight updates.
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
A device and method for performing a forward propagation process of artificial neural network, wherein the device comprises a floating point data statistics module used to perform a statistical analysis on various types of data to obtain a decimal point location, a data conversion unit used to convert a long-bit floating-point data type to a short-bit fixed-point data type according to the decimal point location of fixed point data, and a fixed point data operation module used to perform an artificial neural network forward propagation process on the short-bit fixed point data. The device of the invention realizes a forward propagation process of the short-bit fixed point of the artificial neural network by using a short-bit fixed point representation of the data in the forward propagation process of the multi-layer artificial neural network and using a corresponding fixed point operation module, thereby greatly improving the performance and power ratio for the hardware.