POSIT-based large language model floating point operation optimization method

Abstract

The invention discloses a large language model (LLM) floating point operation optimization method based on POSIT, and the method comprises the following steps: customizing an RISC-V instruction, and enabling the RISC-V instruction to adapt to Point format operation; performing weight quantization on the weight parameter of the LLM by using a Point number; a floating-point number used in the LLM training process is converted into a Point number; a floating-point number used in the LLM reasoning process is converted into a Point number; and carrying out model quantization by using a Posit number. The invention aims to solve the current situation that an IEEE 754 floating-point number format is adopted in the floating-point operation of the current LLM, a Point floating-point number is used for replacing the IEEE 754 floating-point number, and by means of the self-adaptive precision and the dynamic range of the Point floating-point number, the size of the LLM model is reduced and floating-point number calculation is accelerated while various calculation anomalies are avoided and high precision is kept, so that LLM reasoning training is accelerated.

Description

Technical Field

[0001] The present invention mainly relates to the fields of computer architecture design and processor architecture, and in particular to a method for optimizing floating-point operations of a large language model based on POSIT. Background Art

[0002] In recent years, with the increasing popularity of models such as GPT and Llama, large language models (LLMs) have gradually entered everyone's life. LLMs are models trained based on deep learning technology and are specifically used for natural language processing (NLP) tasks. They are known for their large number of parameters and powerful language understanding and generation capabilities, and can perform a variety of language tasks such as text generation, translation, summarization, question answering, dialogue systems, etc.

[0003] For the underlying acceleration of LLM, the current mainstream methods include hardware optimization, model compression, parallel computing, optimization algorithms, and self-attention mechanism acceleration. Norman P. Jouppi from Google accelerated the reasoning training of LLM through the heterogeneity of TPU and CPU; Benoit Jacob Matthew Tang from Google quantified the weights and activation values ​​of the neural network and replaced floating-point calculations with integer calculations to reduce the size of LLM and accelerate LLM reasoning; Mohammad Shoeybi from Nvidia proposed the Megatron-LM framework to accelerate LLM training through model parallel technology; Yang You from the University of Berkeley proposed the LAMB optimizer, which greatly improved the efficiency of LLM training with large batches of data; Sinong Wang from Facebok proposed Linformer, which significantly reduced the computational cost of the traditional Transformer model through the self-attention mechanism of linear complexity.

[0004] The Posit floating-point standard was proposed by John L. Gustafson in 2017. It provides a good trade-off between dynamic range and accuracy, encounters fewer exceptions during operations, has conical accuracy, and supports fused calculations, thereby greatly improving the accuracy of multiplication and accumulation operations.

[0005] In the existing scheme, LLM generates a large number of floating-point calculations during training, reasoning, quantization, etc., and almost all of them adopt the IEEE 754 standard. However, this standard still has many defects, such as rounding and reproducibility issues, signed zero and non-number (NaN) representation problems, etc., and Posit floating-point numbers can well avoid these defects and accelerate the various processes of LLM. Summary of the invention

[0006] The purpose of the present invention is to replace IEEE 754 floating point numbers with Posit floating point numbers in view of the current situation that the floating point calculation of LLM adopts the IEEE 754 floating point format, and by means of the adaptive precision and dynamic range of Posit floating point numbers, various calculation anomalies are avoided, high precision is maintained, and at the same time, the size of the LLM model is reduced, and floating point calculation is accelerated, thereby accelerating LLM reasoning training.

[0007] The present invention uses the RISC-V instruction set to customize corresponding instructions to support Posit number calculation, and modifies the floating-point calculation of each process of LLM into Posit number calculation to accelerate LLM. The specific steps are as follows: Step 1: Customize RISC-V instructions to adapt them to Posit format operations; Step 2: quantize the weight parameters of LLM using Posit numbers; Step 3: Convert the floating point numbers used in the LLM training process into Posit numbers; Step 4: Convert the floating point numbers used in the LLM reasoning process into Posit numbers; Step 5: Use Posit numbers to quantify the model.

[0008] Furthermore, the step 1 modifies some extended instruction fields and CSR registers on the RISC-V basic instructions to adapt them to the Posit format instead of the IEEE 754 format.

[0009] Furthermore, the step 2 converts the weight parameter used by LLM into a Posit number.

[0010] Furthermore, in step three, the matrix operation in LLM training is performed using Posit numbers for adaptive precision and dynamic range operations.

[0011] Furthermore, the step 4 modifies the floating point operation in the LLM reasoning into a low-bit Posit number operation.

[0012] Furthermore, the specific steps of using Posit numbers to quantify the model described in step 5 are as follows: Step 1: Insert quantization and dequantization nodes into the model’s weights and activation functions; Step 2: Use the Posit number for forward propagation (Forward Pass) and backward propagation (Backward Pass), and provide quantization error compensation; Step 3: Perform iterative training to optimize model parameters to adapt to the quantized representation.

[0013] The beneficial effects of the present invention are as follows: (1) The present invention modifies the floating-point calculation of LLM from the IEEE 754 format to the Posit format at the source of the data stream. Through the efficient bit utilization of Posit, higher precision is achieved under the same bit width, thereby maintaining precision while reducing the model size and improving the calculation rate.

[0014] (2) The present invention reduces the probability of error accumulation and calculation anomalies caused by long-term calculations by using smaller rounding errors and larger and more flexible dynamic range of Posit numbers, thereby reducing the number of exception handling steps and optimizing the accuracy of calculations.

[0015] (3) The present invention can be deployed on different RISC-V platforms and is applicable to different LLMs on the market, thus having universality. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is the process of the large language model optimization method based on Posit of the present invention; Figure 2 It is an example of some custom Posit instructions of the present invention; Figure 3 This is an example of the RISC-V floating point control and status register of the present invention; Figure 4 This is the Posit format of the present invention (taking 16 bits as an example). DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0018] Figure 1 The flow chart of a method for optimizing floating-point operations of a large language model based on POSIT of the present invention is shown. As shown in the figure, a method for optimizing floating-point operations of a large language model based on POSIT includes the following steps: S101: Customize RISC-V instructions to adapt them to Posit format operations; S102: quantizing the weight parameters of LLM using Posit numbers; S103: Convert the floating point number used in the LLM training process into a Posit number; S104: converting the floating point number used in the LLM reasoning process into a Posit number; S105: Use Posit numbers to quantify the model.

[0019] Furthermore, the step S101 modifies some extended instruction fields and CSR registers on the RISC-V basic instructions to adapt them to the Posit format instead of the IEEE 754 format.

[0020] Furthermore, the step S102 converts the weight parameter used by LLM into a Posit number.

[0021] Furthermore, the step S103 uses Posit numbers to perform adaptive precision and dynamic range operations on matrix operations in LLM training.

[0022] Furthermore, the step S104 modifies the floating point operation in the LLM reasoning into a low-bit Posit number operation.

[0023] Further, the specific steps of using Posit numbers to quantify the model in step S105 are as follows: Step 1: Insert quantization and dequantization nodes into the model’s weights and activation functions; Step 2: Use the Posit number for forward propagation (Forward Pass) and backward propagation (Backward Pass), and provide quantization error compensation; Step 3: Perform iterative training to optimize model parameters to adapt to the quantized representation. Example

[0024] Llama is a series of large language models developed by Meta. Llama is based on the advanced Transformer architecture and is designed for NLP tasks to generate and understand human language. It is efficient, scalable, and open source. We select Llamam-7B for illustration. The steps of the POSIT-based LLM optimization method implemented on Llama-7B are as follows: Step 1: Customize RISC-V instructions to adapt them to Posit format operations.

[0025] Based on RISC-V I type instructions, customize Posit operation instructions, such as Figure 2 As shown. Keep F to expand the original 32 floating-point registers to store the Posit number, namely P0-P31, and each register has a bit width of 32 bits. Modify the floating-point control and status registers to adapt to the Posit number, such as Figure 3 As shown. Posit has only one rounding mode, so the rounding mode field (rm) is not needed, and it is modified to the Posit length field (pl) to indicate the width of the Posit number currently operated; in the exception flag field (flags), Posit has only the divide by zero exception (DZ), so other flags (NV, OF, UF, NX) are set to 0.

[0026] Step 2: Quantize the weight parameters of LLM using Posit numbers.

[0027] In Llama-7B, weight parameters are usually stored in 32-bit or 16-bit IEEE 754 format, which are quantized into 16-bit or 8-bit Posit numbers. Due to the precision characteristics of the Posit number format, the error introduced in the weight quantization process is relatively small. In order to ensure the performance of the model, error compensation techniques (such as error feedback) are used to further reduce the quantization error.

[0028] Step 3: Convert the floating point numbers used in the LLM training process into Posit numbers.

[0029] During the training process of Llama-7B, the floating-point numbers used in forward propagation, backpropagation, and weight update are modified to Posit numbers to maintain calculation accuracy and reduce the floating-point bit width, improve the efficiency of matrix multiplication and activation function calculation, reduce the rounding error of gradient calculation, and improve the stability of training.

[0030] Step 4: Convert the floating point numbers used in the LLM reasoning process into Posit numbers.

[0031] At the beginning of inference, the model weights quantized by Posit are loaded. After loading, Posit numbers are used for all subsequent calculations. In the forward propagation part of inference, Posit is used for floating-point calculations such as matrix multiplication, activation functions, and nonlinear operations to maintain accuracy while reducing resource consumption; using a lower-bit-width Posit digital format can significantly reduce memory bandwidth usage, thereby accelerating the inference process.

[0032] Step 5: Use Posit numbers to quantify the model.

[0033] (5.1) A quantization node is inserted before each layer of the forward propagation to quantize the IEEE 754 number into a Posit number. A dequantization node is inserted where high-precision operations are required (such as calculating gradients) to restore the Posit number to a floating-point number with a larger bit width to ensure stability and accuracy.

[0034] (5.2) Use Posit numbers for forward propagation and backward propagation calculations, introduce quantization error compensation technology, correct the error caused by quantization each time the gradient is updated, track the error after each quantization, and make appropriate adjustments when updating the model parameters.

[0035] (5.3) After quantizing the nodes with Posit, retrain the model for initial training, track the loss changes of the model to ensure that the quantization error does not significantly reduce the performance. After the initial training, further optimize the model through dynamic learning rate adjustment, regularization, early stopping and other techniques, and perform multiple training iterations to minimize the impact of quantization error on performance.

[0036] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative labor are still within the scope of protection of the present invention.

Claims

1. A floating-point operation optimization method for a large language model based on POSIT, characterized in that: The method for optimizing LLM using Posit format numbers comprises the following steps: Step 1: Customize RISC-V instructions to adapt them to Posit format operations; Step 2: quantize the weight parameters of LLM using Posit numbers; Step 3: Convert the floating point numbers used in the LLM training process into Posit numbers; Step 4: Convert the floating point numbers used in the LLM reasoning process into Posit numbers; Step 5: Use Posit numbers to quantify the model.

2. A POSIT-based large language model floating-point operation optimization method as claimed in claim 1, characterized in that: The step 1 modifies some extended instruction fields and CSR registers on the RISC-V basic instructions to adapt them to the Posit format instead of the IEEE 754 format.

3. A POSIT-based large language model floating-point operation optimization method as claimed in claim 1, characterized in that: The step 2 converts the weight parameter used by LLM into a Posit number.

4. A POSIT-based large language model floating-point operation optimization method as claimed in claim 1, characterized in that: The step three uses Posit numbers to perform adaptive precision and dynamic range operations on matrix operations in LLM training.

5. A POSIT-based large language model floating-point operation optimization method as claimed in claim 1, characterized in that: The step 4 modifies the floating point operation in the LLM reasoning into a low-bit Posit number operation.

6. A POSIT-based large language model floating-point operation optimization method as claimed in claim 1, characterized in that: The specific steps of using Posit numbers to quantify the model in step 5 are as follows: Step 1: Insert quantization and dequantization nodes into the model’s weights and activation functions; Step 2: Use the Posit number for forward propagation (Forward Pass) and backward propagation (Backward Pass), and provide quantization error compensation; Step 3: Perform iterative training to optimize the model parameters to adapt to the quantized representation.

Images