Acceleration method for accelerating multiplication and addition operation in large language model and hardware accelerator

Abstract

The invention relates to the technical field of acceleration operation, in particular to an acceleration method for accelerating multiplication and addition operation in a large language model and a hardware accelerator, and the method comprises the following steps: dividing data in the form of m floating-point numbers into a floating-point number block, m being a preset positive integer; performing block floating point coding on the floating-point number block to obtain a shared index and a mantissa of the floating-point number block, and expanding bit widths of the shared index and the mantissa; performing multiplication and addition calculation based on the expanded sharing index and mantissa; the multiply-add calculation result is decoded and restored into a floating-point number form, and a decoding result is obtained; according to the acceleration method for accelerating the multiplication and addition operation in the large language model and the hardware accelerator, relatively high calculation precision can be kept.

Description

Technical Field

[0001] The present invention relates to the technical field of accelerated computing, and more particularly, to an acceleration method and a hardware accelerator for accelerating multiply-accumulate operations in large language models. Background Art

[0002] Large Language Models (LLMs) have made remarkable progress in the field of Natural Language Processing (NLP). Models such as GPT and T5 perform excellently in tasks such as text generation, translation, and question answering. These models usually have a huge number of parameters. For example, GPT-3 has 175 billion parameters. Such a huge number of parameters makes the training and inference processes of the models require a large amount of computing resources. Traditional computing devices, such as CPUs, are unable to cope when dealing with these large-scale models. Therefore, researchers and engineers have started to explore more efficient hardware solutions to meet the huge demand for computing resources of large language models.

[0003] Data quantization is one of the effective methods to reduce the storage and computing requirements of models. Common quantization methods include fixed-point quantization and block floating-point quantization. Block Floating-Point (BFP) quantization allows a group of values to share an exponent. This quantization method can reduce memory occupancy while maintaining relatively high computing precision. Compared with traditional floating-point numbers, the BFP format reduces the storage space required for each numerical value by sharing the exponent, thereby reducing the memory bandwidth requirements and computing complexity. In addition, BFP quantization can also reduce the complexity of multiply-accumulate operations and improve computing efficiency. For example, the FlexBlock accelerator can maximize core utilization at different precision levels or layer types by supporting multiple BFP modes and allows dynamic precision control. This not only improves the training speed but also significantly enhances the energy efficiency. Specifically, by configuring the FlexBlock to a lower BFP precision, compared with the 16-bit mode, the number of MACs processed by the kernel can increase up to 4 times in the 8-bit mode and up to 16 times in the 4-bit mode. In addition, the FAST system further optimizes the training process through an adaptive training strategy that dynamically adjusts according to the BFP precision of weights, activations, and gradients.

[0004] However, block floating-point quantization also has some problems, especially the reduction of computing precision. Since block floating-point numbers share an exponent, this may lead to a large precision loss during the quantization process, especially when dealing with data with a large dynamic range. Summary of the Invention

[0005] The present invention aims to overcome the defect of the loss of computing precision caused by block floating-point quantization in the above-mentioned prior art, and provides an acceleration method and a hardware accelerator with relatively high computing precision for accelerating multiply-accumulate operations in large language models.

[0006] To solve the above technical problems, the technical solution of the present invention is as follows:

[0007] An acceleration method for accelerating multiplication and addition operations in a large language model, comprising the following steps:

[0008] Divide m floating-point data into a floating-point block, where m is a preset positive integer;

[0009] Perform block floating-point encoding on the floating-point block to obtain the shared exponent and mantissa of the floating-point block, and expand the bit widths of the shared exponent and mantissa;

[0010] Perform multiplication and addition calculations based on the expanded shared exponent and mantissa;

[0011] Decode and restore the multiplication and addition calculation result to floating-point form to obtain the decoding result.

[0012] The present invention also proposes a hardware accelerator for accelerating multiplication and addition operations in a large language model, comprising: an input buffer module, an encoder module, a multiplication and addition operation module, a decoder module, and an output buffer module;

[0013] The input buffer module is used to receive and cache the input floating-point data, and is used to send m floating-point data as a floating-point block to the encoder module;

[0014] The encoder module is used to perform block floating-point encoding on the floating-point block sent by the input buffer module to obtain the shared exponent and mantissa of the floating-point block, and is used to expand the bit widths of the shared exponent and mantissa, and send the expanded shared exponent and mantissa to the multiplication and addition operation module;

[0015] The multiplication and addition operation module contains a multiplication and addition calculator, and the multiplication and addition operation module is used to send the expanded shared exponent and mantissa to the multiplication and addition calculator for multiplication and addition calculations, and is used to send the multiplication and addition calculation result to the decoder module;

[0016] The decoder module is used to decode and restore the data sent by the multiplication and addition operation module to floating-point form, and is used to send the decoding result to the output buffer module;

[0017] The output buffer module is used to put the data sent by the decoder module into the cache, and is used to output the cached data.

[0018] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0019] A method based on block floating-point data quantization compresses the data scale, speeds up the calculation speed, and expands the bit widths of the shared exponent and mantissa of the floating-point blocks. Multiplication and addition calculations are performed based on the expanded shared exponent and mantissa. Utilizing the characteristics of the block floating-point quantization method and the data characteristics after quantization, the bit widths of the exponent and mantissa of the block floating-point are expanded before calculation, which can maintain a high calculation accuracy without affecting the advantage of reducing the storage overhead of the block floating-point. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic flowchart of an acceleration method for accelerating multiplication and addition operations in a large language model proposed in Embodiment 1;

[0021] Figure 2 It is a schematic diagram of the overall structure of a hardware accelerator for accelerating multiplication and addition operations in a large language model proposed in Embodiment 2;

[0022] Figure 3 It is a schematic flowchart of the first encoding execution process of the encoder module proposed in Embodiment 2;

[0023] Figure 4 It is a schematic flowchart of the second encoding execution process of the encoder module proposed in Embodiment 2;

[0024] Figure 5 It is a schematic diagram of the internal structure of the encoder module proposed in Embodiment 2;

[0025] Figure 6 It is a schematic diagram of the structure of the multiplication and addition operation module and its internal PE unit proposed in Embodiment 2. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] The drawings are only for illustrative purposes and should not be construed as a limitation to this embodiment;

[0027] To better illustrate this embodiment, some components in the drawings are omitted, enlarged or reduced, and do not represent the size of the actual product;

[0028] For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0029] The technical solutions of the present invention will be further described below with reference to the drawings and embodiments.

[0030] Embodiment 1

[0031] This embodiment proposes an acceleration method for accelerating multiplication and addition operations in a large language model, Figure 1 which is a schematic flowchart of the acceleration method for accelerating multiplication and addition operations in a large language model proposed in this embodiment;

[0032] In the acceleration method for multiplication and addition operations in large language models proposed in this embodiment, the following steps are included:

[0033] S1: Divide m floating-point data into a floating-point block, where m is a preset positive integer;

[0034] S2: Encode the floating-point block with block floating-point to obtain the shared exponent and mantissa of the floating-point block, and extend the bit widths of the shared exponent and mantissa;

[0035] S3: Perform multiplication and addition calculations based on the extended shared exponent and mantissa;

[0036] S4: Decode the multiplication and addition calculation result back to the floating-point form to obtain the decoding result.

[0037] In the specific implementation process, the data scale is compressed based on the method of block floating-point data quantization to accelerate the calculation speed. By extending the bit widths of the shared exponent and mantissa of the floating-point block and performing multiplication and addition calculations based on the extended shared exponent and mantissa, taking advantage of the characteristics of the block floating-point quantization method and the data characteristics after quantization, the bit widths of the exponent and mantissa of the block floating-point are extended before the calculation, which can maintain a high calculation accuracy without affecting the advantage of reducing the storage overhead of the block floating-point.

[0038] In an alternative embodiment, the step of encoding the floating-point block with block floating-point to obtain the shared exponent and mantissa of the floating-point block includes:

[0039] Take the maximum value of the original exponent E in the floating-point block n and use this maximum value as the shared exponent E of the floating-point block s ;

[0040] Generate a flag bit flag for each floating-point number in the floating-point block according to the relationship between the threshold E T set by the large language model that needs to perform multiplication operations and the original exponent E n . When the original exponent E n is less than the threshold E T , the flag bit flag is 0; when the original exponent E n is greater than or equal to the threshold E T , the flag bit flag is 1;

[0041] Compare the shared exponent E s with the threshold E T . When the shared exponent E s is greater than or equal to the threshold E T , splice the flag bit flag with the original exponent E of the floating-point block n to obtain the spliced mantissa, and record the spliced mantissa as Mn ; when the shared exponent E s is less than the threshold E T , record the original mantissa as M n .

[0042] In an alternative embodiment, the step of expanding the bit widths of the shared exponent and mantissa includes:

[0043] Perform an exclusive OR operation on each of the shared exponent E s and the mantissa M n to obtain an exclusive OR result M n `, and concatenate the mantissa M n with the exclusive OR result M n ` to form {M n , M n `};

[0044] When the flag bit flag is 0, concatenate the shared exponent E s with {M n , M n `} to form the expanded shared exponent and mantissa {E s , M n , M n `};

[0045] When the flag bit flag is 1, perform an exclusive OR operation on each of the original exponent E n and the shared exponent E s to obtain an exclusive OR result E n `, and concatenate the exclusive OR result E n ` with {M n , M n `} to form {E n , M n , M n `}, and then concatenate the shared exponent E s with {E n , M n , M n `} to form the expanded shared exponent and mantissa {E s , E n , M n , M n `}.

[0046] Embodiment 2

[0047] This embodiment proposes a hardware accelerator for accelerating multiply-add operations in large language models, Figure 2 which is a schematic diagram of the overall structure of the hardware accelerator for accelerating multiply-add operations in large language models proposed in this embodiment;

[0048] In the multiply-accumulate operation accelerator based on block floating-point data quantization proposed in this embodiment, it includes: an input buffer module, an encoder module, a multiply-accumulate operation module, a decoder module, and an output buffer module;

[0049] The input buffer module is used to receive and cache the input data in floating-point form, and is used to send m pieces of data in floating-point form as a floating-point block to the encoder module;

[0050] The encoder module is used to perform block floating-point encoding on the floating-point block sent by the input buffer module to obtain the shared exponent and mantissa of the floating-point block, and is used to expand the bit widths of the shared exponent and mantissa, and send the expanded shared exponent and mantissa to the multiply-accumulate operation module;

[0051] The multiply-accumulate operation module contains a multiply-accumulate calculator. The multiply-accumulate operation module is used to send the expanded shared exponent and mantissa to the multiply-accumulate calculator for multiply-accumulate calculation, and is used to send the multiply-accumulate calculation result to the decoder module;

[0052] The decoder module is used to decode and restore the data sent by the multiply-accumulate operation module into floating-point form, and is used to send the decoding result to the output buffer module;

[0053] The output buffer module is used to put the data sent by the decoder module into the cache, and is used to output the cached data.

[0054] In an alternative embodiment, the input buffer module includes a weight buffer unit;

[0055] The weight buffer unit is used to receive and cache the input weight value data in floating-point form, and send m pieces of weight value data in floating-point form as a floating-point block to the encoder module.

[0056] In an alternative embodiment, the input buffer module further includes an activation buffer unit;

[0057] The activation buffer unit is used to receive and cache the input activation value data in floating-point form, and send m pieces of activation value data in floating-point form as a floating-point block to the encoder module.

[0058] In an alternative embodiment, the weight buffer unit contains a buffer area, which is used to receive the input weight value data in floating-point form from the outside. After receiving m floating-point numbers, send the m floating-point numbers as a floating-point block to the multiply-accumulate operation module;

[0059] The activation buffer unit contains a buffer, which is used to receive activation value data in the form of floating-point numbers input from the outside. After receiving m floating-point numbers, the m floating-point numbers are sent to the encoder module as a floating-point block; the buffer also caches a threshold set according to the large language model that needs to perform multiplication operations. When sending the floating-point block to the encoder module, the threshold is sent to the encoder module.

[0060] As an example, regarding the threshold E of the activation buffer module T , it will be selected based on the distribution of activation values of the selected large language model. For the activation value distribution of the large language model, there are a small number of outliers with relatively large values. Although the number of outliers is small, it will have a greater impact on the accuracy when performing block floating-point quantization. Here, the threshold E is selected a priori according to experience for different large language models T to eliminate the influence brought by larger outliers when performing exponential expansion.

[0061] In an optional embodiment, the steps for the encoder module to perform block floating-point encoding on the floating-point block sent by the input buffer module to obtain the shared exponent and mantissa of the floating-point block include:

[0062] Take the maximum value of the original exponents E in the floating-point block n and use this maximum value as the shared exponent E of the floating-point block s ;

[0063] Generate a flag bit flag for each floating-point number in the floating-point block according to the relationship between the threshold E T set according to the large language model that needs to perform multiplication operations and the original exponent E n . When the original exponent E n is less than the threshold E T , the flag bit flag is 0; when the original exponent E n is greater than or equal to the threshold E T , the flag bit flag is 1;

[0064] Compare the shared exponent E s with the threshold E T . When the shared exponent E s is greater than or equal to the threshold E T , splice the flag bit flag with the original exponent E of the floating-point block n to obtain the spliced mantissa, and record the spliced mantissa as M n ; when the shared exponent E s is less than the threshold E T , record the original mantissa as M n ;

[0065] The steps for the encoder module to expand the bit widths of the shared exponent and mantissa include:

[0066] Perform an exclusive OR operation on each of the shared exponent E s and mantissa M n to obtain an exclusive OR result M n `, and concatenate the mantissa M n with the exclusive OR result M n ` to form {M n , M n `};

[0067] When the flag bit flag is 0, concatenate the shared exponent E s with {M n , M n `} to form the expanded shared exponent and mantissa {E s , M n , M n `};

[0068] When the flag bit flag is 1, perform an exclusive OR operation on each of the original exponent E n and the shared exponent E s to obtain an exclusive OR result E n `, and concatenate the exclusive OR result E n ` with {M n , M n `} to form {E n `, M n , M n `}. Then, concatenate the shared exponent E s with {E n `, M n , M n `} to form the expanded shared exponent and mantissa {E s , E n , M n , M n `}.

[0069] As an exemplary illustration, based on the selected large language model, the threshold E T is selected. For example, when selecting the OPT-13b (Open Pretrained Transformers) large language model developed by MetaAI, according to the statistical results of the activation value distribution characteristics of this model, the threshold E T is selected as 5.

[0070] The shared exponent E n is obtained by taking the maximum value of the original exponent E s in the floating-point block. When training with OPT-13b, E s will be compared with the threshold ET Compare with (i.e., 5). If E s < 5, there is no need to generate a flag bit, and directly perform bit-width expansion on the original mantissa M n ; if E s ≥ 5, then complete the bit-width expansion operation of the exponent and mantissa according to the Figure 3 coding execution process; among them, Figure 3 is a schematic diagram of the second coding execution process of the encoder module proposed in this embodiment.

[0071] As an exemplary illustration, Figure 3 is a schematic diagram of the first coding execution process of the encoder module proposed in Embodiment 2; Figure 4 is a schematic diagram of the second coding execution process of the encoder module proposed in Embodiment 2; Figure 3 shows the process of the encoder module performing block floating-point coding on the floating-point block sent by the input buffer module to obtain the shared exponent and mantissa of the floating-point block, and summarizes the process of expanding the bit-width of the shared exponent and mantissa with the word "expansion"; Figure 4 shows the process of the encoder module expanding the bit-width of the shared exponent and mantissa.

[0072] As an exemplary illustration, M n and E n respectively represent the mantissa and exponent of any floating-point number in a floating-point block composed of m floating-point numbers, 1 ≤ n ≤ m, Figure 4 where XOR in

[0073] Figure 5 is a schematic diagram of the internal structure of the encoder module proposed in this embodiment, Figure 5 in which the leftmost is the input mantissa M n , the shared exponent E s , the flag bit flag, and the original exponent E n of each data. M n and E s are XORed to obtain the extended mantissa M[[ID=*]] n `, and then MUX is used for data splicing to obtain the mantissa {M n , M n `} as the output; the flag is connected to the enable terminal of the MUX to determine whether E s performs an expansion operation: if the flag is 0, then directly output E s ; if the flag is 1, then E s and E n are respectively XORed to obtain the extended exponent E n `, and then MUX is used to combine E n ` and E sPerform splicing to obtain the exponent {E s , E n `} as the output.

[0074] In an optional embodiment, the multiply-accumulate calculator includes a PE array composed of n×n PE units, and the multiply-accumulate calculator is used to implement parallel processing of n 2 multiply-accumulate calculations and send the multiply-accumulate calculation results to the decoder module, where n is a positive integer.

[0075] As an exemplary illustration, when the multiply-accumulate calculator of the multiply-accumulate operation module includes a PE array composed of n×n PE units, the multiply-accumulate operation module can be referred to as a systolic array module.

[0076] Figure 6 FIG. is a schematic structural diagram of the multiply-accumulate operation module and its internal PE units proposed in this embodiment. In Figure 6 , E w and E a are respectively the exponent of the weight value and the exponent of the extended activation value, M w and M a are respectively the mantissa of the weight value and the mantissa of the extended activation value, and Psum is the accumulated intermediate result. After receiving the activation value in block floating-point form sent by the encoder module and the weight value sent by the weight buffer module, the PE starts to calculate: ① E1 = E w + E a , M1 = M w × M a ; ② M2 = M1 + Psum; ③ out = M2 << E1. After completing the calculation, the output result is out.

[0077] In an optional embodiment, the PE unit includes a multiplier for implementing the multiplication of the mantissas of the weight value and the activation value, an adder for implementing the addition of the exponents of the weight value and the activation value, an adder for implementing the accumulation of the partial sum, and a shift register for aligning the mantissa to maintain precision.

[0078] As an exemplary illustration, the PE unit can implement the multiply-accumulate calculation Result = ∑(M d × M w ) + Psum in the large language model based on the block floating-point quantization method, and at the same time use the exponent E w + E a to perform a shift operation on the mantissa to maintain precision.

[0079] Embodiment 3

[0080] This embodiment proposes a multiply-accumulate operation acceleration method based on block floating-point data quantization. Applying the multiply-accumulate operation accelerator based on block floating-point data quantization proposed in Embodiment 1, it includes the following steps:

[0081] The input buffer module is used to obtain data in floating-point form, and m pieces of data in floating-point form are divided into a floating-point block;

[0082] The encoder module is used to perform block floating-point encoding on the floating-point block to obtain the shared exponent and mantissa of the floating-point block, and expand the bit widths of the shared exponent and mantissa;

[0083] The multiply-accumulate operation module is used to send the expanded shared exponent and mantissa to a multiply-accumulate calculator for multiply-accumulate calculation;

[0084] The decoder module is used to decode and restore the multiply-accumulate calculation result into floating-point form;

[0085] The output buffer module is used to cache and output the decoding result.

[0086] It can be understood that the multiply-accumulate operation acceleration method based on block floating-point data quantization in this embodiment applies the accelerator in Embodiment 1. The optional items in the above Embodiment 1 also apply to this embodiment, so they will not be described again here.

[0087] This embodiment provides a computer device, including a memory and a processor. Computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by the processor, the processor executes the steps of the multiply-accumulate operation acceleration method based on block floating-point data quantization as described in Embodiment 2.

[0088] The same or similar reference numerals correspond to the same or similar components;

[0089] The terms used to describe the positional relationship in the drawings are only for illustrative purposes and should not be construed as a limitation of this embodiment;

[0090] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, rather than limitations on the implementation manners of the present invention. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to enumerate all the implementation manners here. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the claims of the present invention..

Claims

1. An acceleration method for accelerating multiply-accumulate operations in large language models, characterized in that, It includes the following steps: Divide m floating-point numbers into a floating-point block, where m is a preset positive integer; Perform block floating-point encoding on the floating-point block to obtain the shared exponent and mantissa of the floating-point block, and extend the bit widths of the shared exponent and mantissa; Perform multiplication and addition calculations based on the extended shared exponent and mantissa; Decode and restore the multiplication and addition calculation result into a floating-point form to obtain the decoding result.

2. The acceleration method for accelerating the multiplication and addition operations in the large language model according to claim 1, wherein The steps of performing block floating-point encoding on the floating-point block to obtain the shared exponent and mantissa of the floating-point block include: For the original exponent E in the floating-point block n Take the maximum value and use this maximum value as the shared exponent E for the floating-point block s ; Threshold E set according to the large language model that needs to perform multiplication operations T and the original exponent E n The relationship generates a flag bit flag for each floating-point number in the floating-point block. When the original exponent E n is less than the threshold E T , the flag bit flag is 0; when the original exponent E n is greater than or equal to the threshold E T , the flag bit flag is 1; Compare the shared exponent E s with the threshold E T . When the shared exponent E s is greater than or equal to the threshold E T , splice the flag bit flag with the original exponent E n of the floating-point block to obtain the spliced mantissa, and denote the spliced mantissa as M n ; when the shared exponent E s is less than the threshold E T , denote the original mantissa as M n .

3. The acceleration method for accelerating multiplication and addition operations in a large language model according to claim 2, wherein, The steps of extending the bit widths of the shared exponent and mantissa include: Shared exponent E for floating-point blocks s and mantissa M n Perform an exclusive OR operation on each one to obtain the exclusive OR result M n `, and splice the mantissa M n with the exclusive OR result M n ` to form {M n , M n `}; When the flag bit flag is 0, the shared exponent E s is concatenated with {M n , M n `} to form the extended shared exponent and mantissa {E s , M n , M n `}; When the flag bit flag is 1, the original exponent E n and the shared exponent E s are successively XORed to obtain the XOR result E n `, and the XOR result E n ` is concatenated with {M n , M n `} to form {E n `, M n , M n `}. After that, the shared exponent E s is concatenated with {E n `, M n , M n `} to form the extended shared exponent and mantissa {E s , E n `, M n , M n `}.

4. A hardware accelerator for accelerating multiply-accumulate operations in large language models, which is used to implement the acceleration method for accelerating multiply-accumulate operations in large language models described in any one of claims 1 to 3, characterized in that, It includes: An input buffer module, an encoder module, a multiplication and addition operation module, a decoder module, and an output buffer module; The input buffer module is used to receive and cache the input floating-point data, and is used to send m floating-point data as a floating-point block to the encoder module; The encoder module is used to perform block floating-point encoding on the floating-point block sent by the input buffer module to obtain the shared exponent and mantissa of the floating-point block, and is used to extend the bit widths of the shared exponent and mantissa, and send the extended shared exponent and mantissa to the multiplication and addition operation module; The multiplication and addition operation module contains a multiplication and addition calculator. The multiplication and addition operation module is used to send the extended shared exponent and mantissa to the multiplication and addition calculator for multiplication and addition calculations, and is used to send the multiplication and addition calculation result to the decoder module; The decoder module is used to decode and restore the data sent by the multiplication and addition operation module into a floating-point form, and is used to send the decoding result to the output buffer module; The output buffer module is used to put the data sent by the decoder module into the cache, and is used to output the cached data.

5. The hardware accelerator for accelerating multiply-add operations in a large language model according to claim 4, wherein The input buffer module includes a weight buffer unit; The weight buffer unit is used to receive and cache the input floating-point weight value data, and send m floating-point weight value data as a floating-point block to the encoder module.

6. The hardware accelerator for accelerating multiply-accumulate operations in a large language model according to claim 5, wherein The input buffer module further includes an activation buffer unit; The activation buffer unit is used to receive and cache the input floating-point activation value data, and send m floating-point activation value data as a floating-point block to the encoder module.

7. The hardware accelerator for accelerating multiply-add operations in a large language model according to claim 6, wherein The weight buffer unit contains a buffer area, which is used to receive the input floating-point weight value data from the outside. After receiving m floating-point numbers, send m floating-point numbers as a floating-point block to the multiplication and addition operation module; The activation buffer unit contains a buffer area, which is used to receive the input floating-point activation value data from the outside. After receiving m floating-point numbers, send m floating-point numbers as a floating-point block to the encoder module; the buffer area also caches a threshold set according to the large language model that needs to perform multiplication operations, and sends the threshold to the encoder module while sending the floating-point block to the encoder module.

8. The hardware accelerator for accelerating multiply-add operations in a large language model according to claim 7, characterized in that, The steps of the encoder module performing block floating-point encoding on the floating-point block sent by the input buffer module to obtain the shared exponent and mantissa of the floating-point block include: For the original exponent E in the floating-point block n Take the maximum value and use this maximum value as the shared exponent E of the floating-point block s ; Threshold E set according to the large language model that needs to perform multiplication operations T and the original exponent E n The relationship generates a flag bit flag for each floating point number in the floating point block. When the original exponent E n is less than the threshold E t , the flag bit flag is 0; when the original exponent E n is greater than or equal to the threshold E T , the flag bit flag is 1; Compare the sharing index E s with the threshold E T . When the sharing index E s is greater than or equal to the threshold E T , splice the flag bit flag with the original exponent E n of the floating-point block to obtain the spliced mantissa, and denote the spliced mantissa as M n ; when the sharing index E s is less than the threshold E T , denote the original mantissa as M n ; The steps of the encoder module extending the bit widths of the shared exponent and mantissa include: Shared exponent E of the floating-point block s and mantissa M n Perform an exclusive OR operation on each one to obtain the exclusive OR result M n `, concatenate the mantissa M n with the exclusive OR result M n ` to form {M n , M n `}; When the flag bit flag is 0, the shared exponent E s is concatenated with {M n , M n `} to form the extended shared exponent and mantissa {E s , M n , M n `}; When the flag bit flag is 1, the original exponent E n and the shared exponent E s are successively XOR-operated to obtain the XOR result E n `. Then, the XOR result E n ` is concatenated with {M n , M n `} to form {E n `, M n , M n `}. After that, the shared exponent E s is concatenated with {E n `, M n , M n `} to form the extended shared exponent and mantissa {E s , E n , M n , M n `}.

9. The hardware accelerator for accelerating multiplication and addition operations in a large language model according to any one of claims 5 to 8, characterized in that The multiplier-accumulator calculator includes a PE array composed of n×n PE units, and the multiplier-accumulator calculator is used to implement parallel processing of n 2 multiplier-accumulation calculations and send the multiplier-accumulation calculation results to the decoder module, where n is a positive integer.

10. The hardware accelerator for accelerating multiplication and addition operations in a large language model according to claim 9, wherein, The PE unit includes a multiplier for implementing the multiplication of the mantissas of the weight value and the activation value, an adder for implementing the addition of the exponents of the weight value and the activation value, an adder for implementing the accumulation of partial sums, and a shift register for aligning the mantissas to maintain precision.

Images