Data Processing Method Using Sub-Tensor Segmentation for High-Accuracy Calculation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current processors lack support for single-accuracy floating point formats, such as FP32, which are crucial for high-performance calculations in AI and data analysis, limiting their applicability and efficiency.
Innovation Solution
The method involves representing high-accuracy tensors using combinations of low-accuracy floating point sub-tensors, such as BF16 and BF24, to simulate higher accuracy formats like FP32, allowing for matrix multiplication and convolution operations, thereby expanding the processors' capabilities and improving efficiency without increasing calculation time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If processors use low-accuracy floating point formats (BF16, BF24), then device complexity and resource consumption are reduced, but measurement precision and calculation accuracy deteriorate
Solution Approach 1:
The patent segments a high-accuracy floating point number (FP32) into multiple low-accuracy floating point sub-tensors (BF16 or BF24). Each sub-tensor represents a portion of the original high-precision data, and multiple such sub-tensors are combined through addition to reconstruct the original high-precision value. This segmentation allows the processor to use simpler low-accuracy formats while maintaining the capability to perform high-accuracy calculations through composition of multiple components.
2Device complexity
If processors support only low-accuracy floating point formats, then device complexity is reduced, but adaptability to high-precision calculation requirements deteriorates
Solution Approach 1:
The patent enables a processor designed for low-accuracy floating point operations to also handle high-accuracy floating point calculations by representing FP32 numbers as combinations of BF16/BF24 sub-tensors. This multi-functionality allows the same hardware architecture to operate in both low-precision and high-precision modes, adapting to different computational requirements without requiring separate hardware paths for each precision level.
3Adaptability or versatility
If high-accuracy floating point calculations are performed using combined sub-tensors, then adaptability to high-precision formats is improved, but calculation time may increase
Solution Approach 1:
The patent performs preliminary decomposition of high-accuracy floating point numbers into low-accuracy sub-tensors before performing calculations. By pre-splitting the FP32 data into BF16/BF24 sub-tensors that can be processed in parallel, the system prepares the data in advance for efficient low-precision computation, reducing the overall time required compared to using a single high-precision format throughout.
Data Source
AI summary
A data processing method and data processing apparatus are provided. The data processing method includes: acquiring multiple input tensors as input parameters for calculation process; for each input tensor, using M input sub-tensors that are combined to represent the input tensor; for each of the input tensors, replacing the input tensors with the M input sub-tensors that are combined to represent the input tensor, and performing the calculation process to obtain a calculation result. The data processing method increases the applicable scenarios of calculation process, effectively utilizes the powerful calculation ability of the originally provided low-accuracy floating points, and greatly improves the overall calculation efficiency.