Multi-Precision Operand Splitting for Unified Neural Computation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network computations with mixed precision often require significant hardware resources due to the need for single-precision computation units, leading to reduced data bandwidth and precision when using low-bit quantization.
Innovation Solution
A multi-precision operand computation method that splits input operands into sub-operands with data precision supported by the computation unit, allowing individual computation of each sub-operand to maintain precision without reducing data bandwidth through low-bit quantization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data bandwidth of high-precision data is reduced through low-bit quantization to match data bandwidth of low-precision data, then hardware resource requirements are reduced, but precision of the data computations is lowered
Solution Approach 1:
The patent divides high-precision input operands into multiple sub-operands with lower precision (e.g., splitting a 32-bit operand into multiple 8-bit sub-operands). This segmentation allows the computation unit to process each sub-operand individually using low-bit quantization while maintaining the overall precision through proper combination of results, thus reducing hardware resource requirements without sacrificing computation precision.
2Measurement precision
If computation units support single precision to handle mixed precision computations, then computation precision is maintained, but hardware resources increase significantly
Solution Approach 1:
The patent enables a computation unit with unified data precision (single precision) to handle multiple precision computation tasks by dividing high-precision operands into sub-operands. This multi-functional approach allows the same computation unit to process both low-precision and high-precision data without requiring separate dedicated units, thereby maintaining computation precision while avoiding significant increases in hardware resources.
3Device complexity
If low-bit quantization is used to reduce hardware resources, then hardware resource requirements are reduced, but data bandwidth is reduced
Solution Approach 1:
By segmenting high-precision data into multiple low-precision sub-operands, the patent reduces the data bandwidth requirement for each individual data transmission while maintaining the overall data precision. The computation unit processes these segmented sub-operands separately and combines the results, achieving hardware resource reduction without proportionally reducing the effective data bandwidth for computation.
Data Source
AI summary
Embodiments of this disclosure disclose a multi-precision operand computation method and apparatus, a device, and a medium. The method includes: determining, based on first data precision of an input operand, at least one input sub-operand with second data precision that corresponds to the input operand, where the second data precision is data precision supported by a computation unit, and a data bit width of the first data precision is greater than or equal to that of the second data precision; controlling the computation unit to individually compute each input sub-operand to obtain a corresponding computation sub-result for each input sub-operand; and determining a computation result corresponding to the input operand based on the computation sub-result individually corresponding to each input sub-operand.