A single-precision hardware implementation double-precision simulation method based on floating-point decomposition reconstruction

Abstract

The application discloses a single-precision hardware implementation double-precision simulation method based on floating-point decomposition and reconstruction, including a 32-bit single-precision server; firstly, a prediction model is acquired to train floating-point 64-bit double-precision scalar or tensor original data, and the implementation double-precision simulation method comprises the following steps: step S1, data preprocessing; step S2, computation graph reconstruction; two floating-point 32-bit single-precision data are rewritten into a double-channel computation form by using a traditional computation flow; step S3, dynamic precision recovery; step S4, adaptive quantization control; an error feedback mechanism is introduced, when the error exceeds a threshold value, a dynamic decomposition result of computation is triggered, and the decomposition result is iterated to two floating-point 32-bit single-precision data, the application meets the high-precision computation demand of the single-precision server; the application is completely based on a single-precision instruction set and does not need special hardware support, and is suitable for an NPU cluster; compared with original double-precision storage, the memory occupation is reduced by 25%.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of high-performance computing and numerical analysis, and particularly relates to a single-precision hardware implementation double-precision simulation method based on floating-point decomposition reconstruction, which is a method for simulating double-precision calculation on a 32-bit computer chip. BACKGROUND

[0002] In the field of scientific computing and engineering simulation, double-precision floating-point operation (float 64 bits) has been regarded as the gold standard for numerical simulation due to its 64-bit storage format, i.e., 1-bit sign, 11-bit exponent, and 52-bit mantissa, which can provide about 16-bit effective decimal precision and a dynamic range of 10 -308 to 10 308 . This feature enables accurate capture of small physical quantity changes and suppression of error accumulation in iterative calculations.

[0003] Taking computational fluid dynamics as an example, when simulating transonic flow fields, the magnitude difference between pressure gradients and turbulent eddy viscosity coefficients is extremely large. Single-precision floating-point 32 bits (float 32 bits) only have 7-bit effective digits, and after thousands of iterations, the truncation error of key physical quantities will be amplified to more than 5%, leading to flow field instability. In the field of aircraft aerodynamic shape optimization, such errors may cause design rework and result in huge costs.

[0004] In the training process of aircraft flow field surrogate models, the accuracy of numerical precision directly determines the prediction ability of the model for complex flow field phenomena, especially under extreme conditions such as high speed and turbulence. Small calculation deviations can cause significant deviations between simulated results and actual physical laws. However, the current mainstream domestic servers (such as Huawei 910) used for model training are limited by hardware architecture and do not support float64 neural network training, only supporting float32 single-precision neural network training mode. In actual training, it is found that this precision limitation accumulates nonlinear errors, leading to prediction deviations of key parameters such as flow field pressure distribution and velocity gradient exceeding the acceptable range of engineering applications, seriously affecting the reliability and practicality of surrogate models.

[0005] With the application of deep learning in scientific computing, the demand for double-precision is more prominent. Physical information neural network gradient calculation is extremely sensitive to rounding errors, and single-precision training can significantly increase the divergence error of the velocity field, violating the law of conservation of mass. The covariance matrix inversion and log probability density calculation of probabilistic machine learning models also rely on high-precision floating-point operations.

[0006] But hardware development and high-precision computing demand is contrary. Modern accelerator architecture, such as NPU, TPU and edge computing chip, generally weakens or removes double precision support for pursuing energy efficiency ratio and computing density. Although commercial GPU retains double precision unit, but the computing power is far lower than single precision, and the price is expensive.

[0007] In the edge computing scenario, the device precision support is limited, resulting in the decline of model prediction reliability. Although the industry tries to use mixed precision training and other methods to alleviate the problem, there are still problems of rounding error or excessive computing overhead. Therefore, how to realize double precision equivalent calculation on single precision hardware (32-bit chip) has become a key problem to be solved to promote scientific computing popularization. SUMMARY

[0008] The application provides a single-precision hardware implementation double-precision simulation method based on floating point decomposition and reconstruction, which is a method for simulating double-precision calculation on a 32-bit computer chip.

[0009] In order to achieve the above purpose, the scheme of the application is:

[0010] A single-precision hardware implementation double-precision simulation method based on floating point decomposition and reconstruction, using two 32-bit single-precision data to represent a 64-bit double-precision data, establishing a full link of floating point number decomposition-computation-reconstruction, for constructing an aircraft flow field prediction model graph, including a 32-bit single-precision server; first, obtaining the prediction model for training floating point 64-bit double-precision scalar or tensor original data, the double-precision simulation method step includes:

[0011] Step S1, data preprocessing: decompose and convert the floating point 64-bit double-precision scalar or tensor original data into two floating point 32-bit single-precision data, which are 32-bit single-precision data of high bits and 32-bit single-precision data of low bits of 64-bit double-precision data respectively, set the iteration number comparison counter, and set the maximum number of iterations according to the error as the double-precision limit value;

[0012] Step S2, computation graph reconstruction: rewrite the two floating point 32-bit single-precision data into double-channel computation form with traditional computation flow, and merge and reconstruct the calculation results of the two channels;

[0013] Step S3, dynamic precision recovery: double-precision synthesis is performed on the calculation results to obtain the calculation results, and the error limit of the calculation results is analyzed;

[0014] Step S4, adaptive quantization control: introduce an error feedback mechanism, when the error exceeds the threshold, trigger the dynamic decomposition of the calculation results, iterate the decomposition results into two floating point 32-bit single-precision data, the iteration number comparison counter is increased by 1, and return to step S2;

[0015] When the error does not exceed the threshold, it is determined whether the iteration number reaches the maximum number, if not, the decomposition result is iterated two floating point 32-bit single precision data, the iteration number is added 1 to the comparison counter, and the step S2 is returned, otherwise, the calculation result is the final result, and the calculation graph reconstruction is completed.

[0016] The scheme further comprises that in step 1, the scalar or tensor original data decomposition formula is:

[0017]

[0018] Wherein,

[0019] Represents the 32-bit single-precision high-bit component of the original data;

[0020] Represents the 32-bit single-precision low-bit component of the original data;

[0021] The decomposition process is completed in two steps:

[0022] The first step is a truncated approximation, which converts the original data into a single-precision high-bit component and reverts to a double-precision representation. This process loses the low-bit significant digits;

[0023] The second step is residual capture, which calculates the difference between the original data and the high-bit component, and quantizes the difference into single-precision again. The decomposition process ensures that The error is a double-precision limit.

[0024] The scheme further comprises that in step S3, the error limit of the dynamic precision recovery is:

[0025]

[0026] Wherein,

[0027] The calculation result obtained by double-precision synthesis;

[0028]

[0029] The traditional calculation process is rewritten into a double-channel calculation form:

[0030] .

[0031] The scheme further comprises that in step S4, the activation condition of the error feedback mechanism is:

[0032]

[0033] Wherein,

[0034] Threshold value .

[0035] The scheme is further that in step S4, the dynamic re-decomposition is realized by a dynamic re-decomposition formula, and the dynamic re-decomposition formula is:

[0036] .

[0037] The beneficial effects of the present application are: the high-precision computing needs of single-precision servers are met; it is completely based on a single-precision instruction set and does not require special hardware support, and is suitable for NPU clusters; compared with native double-precision storage, the memory occupancy is reduced by 25%.

[0038] To break through the hardware precision bottleneck and improve the model training quality, the present method specifically develops a technical method for simulating a floating-point 64-bit (float64) double-precision neural network training in a floating-point 32-bit (float32) server environment, aiming to compensate for hardware limitations through algorithmic innovation and provide a feasible path for the research and development of high-precision flow field proxy models. The main problem to be solved is the low performance caused by insufficient hardware precision in the direction of aircraft flow field prediction models.

[0039] The present method can use 32-bit single-precision floating-point operations to realize 64-bit double-precision floating-point operations and achieve double-precision equivalent calculation on a single-precision 32-bit computer chip hardware. The double-precision is decomposed into a high-order component and a low-order component by a residual method, and then processed by single-precision hardware in two channels. Error values are recorded during the processing, and the re-decomposition mechanism is triggered when the error is too large. Finally, the calculation results of the two channels are summed to obtain the final double-precision simulation result.

[0040] The invention will be further explained in detail with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 The present method is a step flow chart. DETAILED DESCRIPTION

[0042] A single-precision hardware double-precision simulation method based on floating-point decomposition and reconstruction uses two 32-bit single-precision data to represent a 64-bit double-precision data, establishes a full link of floating-point decomposition-computation-reconstruction, and realizes double-precision simulation calculation of single-precision hardware through floating-point decomposition and reconstruction technology, which is used to construct an aircraft flow field prediction model graph, including a 32-bit single-precision server. First, the original data of the scalar or tensor used for training floating-point 64-bit double-precision of the prediction model (tensor is a data format specially used for deep learning, which can also be called matrix, and scalar is also a matrix, the difference between the two is that tensor can be automatically differentiated and scalar does not have this function) is obtained, and the double-precision simulation method is realized as shown in Figure 1 .

[0043] Step S1: Data preprocessing, including floating-point decomposition and encoding, including floating-point approximate truncation and residual capture, to obtain the single-precision high-order and low-order components. This involves decomposing and encoding the 64-bit double-precision scalar or tensor raw data into two 32-bit single-precision floating-point data representations: the high-order component of the 32-bit single-precision data, representing the high-order bits of the 64-bit double-precision data. The low-order component of the 32-bit single-precision data and the low-order bit of the 64-bit double-precision data. That is: for any double-precision scalar or tensor raw data Decompose it into single-precision high-bit components and single-precision low-bit components Satisfying the mathematical relationship:

[0044]

[0045] An iteration count comparison counter is set, and the maximum number of iterations is set based on the error as the double-precision limit. The decomposition process is completed in two steps: truncation approximation and residual capture. In the truncation approximation process, the original data is first converted into single-precision data. Then back to double precision. This process loses significant digits in the least significant place. The residual capture stage is used to calculate the difference between the original data and the high-precision components, and then quantizes the residuals again to single precision. ;

[0046] The residual error of this decomposition method Close to the order of magnitude This is approximately the limit of double precision. Compared to traditional high-low bit splitting (directly dividing binary bits), this method ensures precision through floating-point quantization. and The magnitude difference is controllable, avoiding underflow of calculations due to the small size of the lower-order values.

[0047] Step S2, Computational Graph Reconstruction: The traditional computational process is rewritten into a dual-channel computational architecture, and the computational results of the two channels are merged and reconstructed. To achieve lossless transfer of computational results, a high-level channel and a low-level channel are constructed according to the task requirements. The high-level channel is used for... Perform standard single-precision arithmetic Used to capture the main result; low-level channel pairs Perform the same operation Used to capture residual correction terms; the two channels are completely independent in the calculation, and the calculation results of the two channels are finally merged and reconstructed using the following formula:

[0048]

[0049] Step S3, dynamic precision recovery: double-precision synthesis is performed on the calculation result to obtain the calculation result, and the error limit of the calculation result is analyzed;

[0050] That is, a dynamic error control mechanism is set, a deep learning network is constructed, a neural network layer suitable for the method is constructed in the network architecture, the calculation pipeline is fused into the network model, and for a long-term iterative calculation task (such as a PDE solver and neural network training), an adaptive re-decomposition strategy is designed, which is divided into error monitoring and threshold triggering re-decomposition.

[0051] In the error monitoring step, the relative amplitude of the low-bit component of each iteration is calculated :

[0052]

[0053] When the relative amplitude is greater than 1 (usually ), the re-decomposition operation is triggered:

[0054]

[0055]

[0056] The re-decomposition process first adds the original high-bit component and the original low-bit component to obtain a new high-bit component and convert it to double-precision, then uses the sum of the two original components and the double-precision new high-bit component to obtain a new low-bit component, and finally converts the new high and low-bit components to single-precision and continues the iteration. This process avoids the residual term from exceeding the single-precision representation range by periodically merging high and low components.

[0057] Step S4, adaptive quantization control: an error feedback mechanism is introduced, when the error exceeds the threshold, triggering dynamic re-decomposition of the calculation result, iterating the re-decomposition result twice to 32-bit single-precision data, incrementing the iteration count by 1, and returning to step S2;

[0058] When the error does not exceed the threshold, it is determined whether the iteration count reaches the maximum number, if not, the re-decomposition result is iterated twice to 32-bit single-precision data, the iteration count is incremented by 1, and the step S2 is returned, otherwise, the calculation result is the final result, the calculation graph reconstruction is completed, and the maximum iteration count is used to improve the calculation precision of step S2.

[0059] In step 1, the scalar or tensor original data is , and the decomposition formula is:

[0060]

[0061] wherein,

[0062] 32-bit single-precision high component of the original data;

[0063] 32-bit single-precision low component of the original data;

[0064] The process of decomposition is completed in two steps:

[0065] The first step is a truncated approximation, which converts the original data into a single-precision high component and falls back to a double-precision representation. This process loses low significant digits;

[0066] The second step is a residual capture, which calculates the difference between the original data and the high component, and quantizes the difference again into single-precision. The decomposition process ensures that , the error is the double-precision limit ( ).

[0067] In step S3, the error bound of the dynamic precision recovery is:

[0068]

[0069] where:

[0070] is the calculation result obtained by double-precision synthesis;

[0071]

[0072] is rewritten into a double-channel calculation form:

[0073] .

[0074] In step S4, the activation condition of the error feedback mechanism is:

[0075]

[0076] where:

[0077] threshold .

[0078] In step S4, the dynamic re-decomposition is realized by a dynamic re-decomposition formula, which is:

[0079] .

[0080] The following is a specific implementation description of using the method to use the neural network Transformers architecture as an example:

[0081] The first step, corresponding to step S1, the input part of the original training data (8-dimensional features) remains single-precision floating point (float32) format, and only the output target (46x1231-dimensional regression target) is performed floating point decomposition. Specifically, for each output sample , the high-order main term is first defined as the single-precision truncated value of the original data, i.e. . Then, its double-precision residual with the original value is calculated, and the residual is quantized to single-precision form twice. This process satisfies the mathematical relationship:

[0082]

[0083] where the residual error is introduced by two quantization operations, and its magnitude is the double-precision limit, so it is in the acceptable range. The decomposed data and are stored as independent single-precision files for model training calls.

[0084] Taking the 16-bit number X = 3.141592653589793 as an example, the floating point decomposition process is expressed: the 64-bit floating point original data X = 3.141592653589793 is decomposed and truncated to 32-bit floating point high component X high = 3.1415927 and X-X high (residual capture) forms a 32-bit floating point low component X low = -0.000000046410207.

[0085] The second step, corresponding to step S2, the deep learning model architecture is adapted, and under the premise of maintaining single-precision calculation, the branch prediction of high-precision main term and residual term is realized through special linear layer design and feature routing mechanism. The neural network Transformers architecture model is composed of feature encoding module, spatial dependence modeling module and double-channel output module, and each component works together to ensure the mathematical completeness of the output decomposition.

[0086] In the feature encoding module, the precision feature (8-dimensional vector) is first extracted into high-order abstract features by the feature encoding module, which is composed of two fully connected layers. The first layer expands the input dimension to 512 dimensions, and after layer normalization (Layer Normalization) and GELU activation function processing, it is further mapped to a 1024-dimensional hidden space through the second layer.

[0087] In the spatial dependency modeling module, a decoder structure based on Transformer is used to model the spatial and temporal features. The module contains 6 identical decoding units, each of which is composed of multi-head self-attention mechanism (8 heads) and feedforward network. The hidden features are reshaped into sequence form (46 regression output layers).

[0088] The dual-channel output module is designed for the overall architecture and the adaptive module of the method. The decoder output features are simultaneously sent to two linear transformation layers that are isolated from each other:

[0089] For the high-bit main term output head, the features are mapped to high-precision coordinate prediction through a weight matrix

[0090]

[0091] For the low-bit residual term, an independent weight matrix is used to generate the residual compensation term:

[0092]

[0093] The two output heads share the same feature input , but achieve differentiated prediction targets through completely decoupled weight parameters. This design ensures that the high-precision main term focuses on capturing macro regression values, while the residual term fine-tunes microscopic errors.

[0094] In the third step, corresponding to step S3, a mixed precision training strategy is implemented. During training, the loss function is calculated through a dynamic precision recovery mechanism. Specifically, the model's predicted and are converted to double precision and superimposed, and the L1 loss is calculated with the true value :

[0095]

[0096] During backpropagation, gradient calculation is performed in single-precision environment, but the error signal is transmitted to the encoder through the dual-channel path, allowing the network to learn the distribution rules of the main term and the residual at the same time. To suppress error accumulation during long-term training, a dynamic re-decomposition strategy is introduced: every 20 iterations, the relative amplitude of the residual term is calculated , if , re-decomposition is performed on the current prediction value to generate new main term and residual target, thereby resetting the error baseline.

[0097] ​​Step 4, corresponding to step S4, involves inference deployment and precision mode. During the model inference and prediction phase, two-precision mode and fast mode are available. In double-precision mode, the model outputs two-channel predicted values. and Convert to double precision and then sum them to obtain the final result. Its accuracy is essentially equivalent to native double-precision training. In fast mode, only the high-order principal term is used. The generated results are suitable for scenarios with stringent timeliness requirements, such as real-time visualization. In these scenarios, the accuracy is comparable to that of a conventional single-precision model, but memory usage and inference speed are significantly optimized.

[0098] The aforementioned method for implementing double-precision simulation in single-precision hardware based on floating-point decomposition and reconstruction meets the high-precision computing requirements of single-precision servers. It is implemented entirely based on a single-precision instruction set, requiring no dedicated hardware support and is suitable for NPU clusters. Compared to native double-precision storage, memory usage is reduced by 25%. By numerically decomposing the target data, the original double-precision floating-point data is accurately decomposed into two single-precision components: a high-order primary term and a low-order residual term. A strict dual-channel mapping relationship is established mathematically to ensure that the decomposition error is lower than the inherent rounding error of double-precision floating-point. Double-precision equivalent computation is achieved on a single-precision 32-bit computer chip. Double precision is decomposed into high-order and low-order components using residual methods, and then processed in dual channels by single-precision hardware. Error values ​​are recorded during processing, and a re-decomposition mechanism is triggered when the error is too large. Finally, the calculation results of the two channels are summed to obtain the final double precision simulation result. Using this method, a dual-channel deep learning model with independent weight output heads and gradient routing gating is designed. The main term and residual term are predicted in parallel within the single-precision calculation framework, enabling the model to learn the macroscopic geometric structure and microscopic error compensation simultaneously. Finally, during the training phase, parameters are optimized through double precision loss reconstruction and dynamic re-decomposition strategies. During the inference phase, high-precision results can be synthesized or the main term prediction can be quickly output, achieving a balance between "accuracy and efficiency".

Claims

1. A single-precision hardware implementation double-precision simulation method based on floating-point decomposition reconstruction, using two 32-bit single-precision data to represent a 64-bit double-precision data, establishing a full link of floating-point number decomposition-computation-reconstruction, for constructing an aircraft flow field prediction model graph, comprising a 32-bit single-precision server; characterized in that, First, the prediction model is used to train the floating-point 64-bit double-precision scalar or tensor raw data, and the implementation of the double-precision simulation method comprises the following steps: Step S1, data preprocessing: decompose and convert the floating-point 64-bit double-precision scalar or tensor raw data into two floating-point 32-bit single-precision data, which are 32-bit single-precision data of high bits and 32-bit single-precision data of low bits of 64-bit double-precision data respectively, set an iteration number comparison counter, and set a maximum iteration number according to the error approaching the double-precision limit value; Step S2, calculation graph reconstruction: rewrite the two floating-point 32-bit single-precision data into a double-channel calculation form using a traditional calculation process, and merge and reconstruct the calculation results of the two channels; Step S3, dynamic precision recovery: double-precision synthesis is performed on the calculation results to obtain the calculation results, and the error limit of the calculation results is analyzed; Step S4, adaptive quantization control: an error feedback mechanism is introduced, when the error exceeds the threshold, the dynamic decomposition of the calculation results is triggered, the decomposition results are iterated into two floating-point 32-bit single-precision data, the iteration number comparison counter is added by 1, and the step S2 is returned; when the error does not exceed the threshold, it is judged whether the iteration number reaches the maximum number, if not, the decomposition results are iterated into two floating-point 32-bit single-precision data, the iteration number comparison counter is added by 1, and the step S2 is returned, otherwise, the calculation results are the final results, and the calculation graph reconstruction is completed.

2. The simulation method of claim 1, wherein, In step 1, the scalar or tensor raw data decomposition formula is: ； Wherein, represents the 32-bit single-precision high component of the original data; represents the 32-bit single-precision low component of the original data; The decomposition process is completed in two steps: The first step is to cut off the approximation, convert the original data into single-precision high-bit components, and back to double-precision representation, this process loses low-bit significant digits; The second step is the residual capture, which computes the difference between the original data and the high bits of the previous result and quantizes the difference again to single precision. The decomposition process guarantees that the error is close to the double precision limit.

3. The simulation method of claim 1, wherein, In step S3, the error limit of the dynamic precision recovery is: ； Wherein: The result of the calculation for double precision synthesis is obtained; ； Rewrite the traditional computing flow into a dual-channel computing form: ； Wherein: represents the 32-bit single-precision high component of the original data; Represents the 32-bit single-precision low component of the original data.

4. The simulation method of claim 1, wherein, In step S4, the activation condition of the error feedback mechanism is: ； Wherein: Threshold value ; represents the 32-bit single-precision high component of the original data; Represents the 32-bit single-precision low component of the original data.

5. The simulation method of claim 4, wherein, In step S4, the dynamic decomposition is realized by a dynamic decomposition formula, and the dynamic decomposition formula is: 。

Images