Method and device for accelerating large language model and electronic equipment

Abstract

The invention relates to a large language model acceleration method and device and electronic equipment, and the method comprises the steps: obtaining a conversion formula for converting floating-point number multiplication into integer addition according to a multiplication formula of two floating-point numbers, obtaining the related data of a floating-point number multiplier and a multiplicand, determining the exponential offset, and carrying out the exponential offset on the sign bits of the floating-point number multiplier and the multiplicand according to the sign bits of the floating-point number multiplier and the multiplicand. The exponent bits and the exponent offsets are substituted into the conversion formula to obtain approximate sign bits and approximate exponent bits of an approximate result, mantissas of the floating-point number multiplier and the multiplicand are correspondingly added to obtain approximate mantissas of the approximate result, and the approximate sign bits, the approximate exponent bits and the approximate mantissas of the approximate result are combined to obtain a final result. According to the method, the characteristic that the energy consumption of addition is lower than that of multiplication is utilized, and the addition is used for approximating multiplication operation, so that the energy consumption of training and reasoning of a large language model is effectively reduced, and the computing resource consumption of fine-tuning training and reasoning is reduced.

Description

Technical Field

[0001] The present application relates to the technical field of deep learning and natural language processing, and in particular to a method, device and electronic device for accelerating a large language model. Background Art

[0002] With the widespread application of AI technology, large language models are also developing rapidly. In large language models, the model training and reasoning processes require a large number of floating-point precision matrix multiplications. In the field of deep learning and natural language processing, multiplication operations, especially floating-point multiplications, are computationally intensive and energy-intensive operations. Due to the large number of multiplication operations in large language models, the model training and reasoning processes are accompanied by huge energy consumption. The high energy consumption caused by multiplication operations in large language models not only increases operating costs, but also puts pressure on the environment, exacerbating energy consumption and carbon emissions. Summary of the invention

[0003] In view of this, the present application proposes a method for accelerating a large language model to solve the problems reflected in the above-mentioned background technology.

[0004] In one aspect, the present application provides a method for accelerating a large language model, comprising the following steps:

[0005] Based on the multiplication formula of two floating-point numbers, the conversion formula for converting floating-point multiplication into integer addition is obtained;

[0006] Get the relevant data of floating-point multiplier and multiplicand, and determine the exponent offset;

[0007] Substituting the floating-point multiplier, the sign bit, the exponent bit and the exponent offset of the multiplicand into the conversion formula to obtain an approximate sign bit and an approximate exponent bit of the approximate result;

[0008] Add the mantissas of the floating-point multiplier and the multiplicand correspondingly to obtain the approximate mantissas of the approximate result;

[0009] The approximate sign bit, the approximate exponent bit and the approximate mantissa bit of the approximate result are combined to obtain a final result.

[0010] As an optional implementation scheme of the present application, optionally, according to the multiplication formula of two floating-point numbers, a conversion formula for converting floating-point number multiplication into integer addition is obtained, including:

[0011] According to the multiplication formula of the two floating-point numbers, an intermediate formula of an approximate result is obtained;

[0012] According to the intermediate formula, the conversion formula is obtained.

[0013] As an optional implementation scheme of the present application, optionally, the step of obtaining the relevant data of the floating-point multiplier and the multiplicand and determining the exponent offset, wherein the relevant data of the floating-point multiplier and the multiplicand include the sign bit, the exponent bit and the mantissa bit of the floating-point multiplier and the multiplicand, comprises:

[0014] Get the floating-point multiplier, the sign bit, the exponent bit and the mantissa bit of the multiplicand;

[0015] The sign bit, exponent bit and mantissa bit are processed to obtain the exponent offset value.

[0016] As an optional implementation of the present application, optionally, the floating-point multiplier, the sign bit, the exponent bit and the exponent offset of the multiplicand are substituted into the conversion formula to obtain an approximate sign bit and an approximate exponent bit of the approximate result, including:

[0017] Performing an XOR operation on the sign bits of the floating-point multiplier and the multiplicand to obtain the sign bit of the approximate result;

[0018] The exponent offset needs to be moved to a position corresponding to the exponent bit for operation, and the exponent bits and exponent offsets of the floating-point multiplier and multiplicand are calculated to obtain the exponent bit of the approximate result.

[0019] As an optional implementation scheme of the present application, optionally, the exponent offset needs to be moved to a position corresponding to the exponent bit for calculation in the conversion formula.

[0020] As an optional implementation scheme of the present application, optionally, the value of the exponential offset is an exponential offset value obtained by quantitatively analyzing the data, including:

[0021] The exponential offset value is obtained by inputting a certain amount of sample data into a large language model for inference;

[0022] In a large language model, the method for determining the exponential offset value may be used to train the model.

[0023] As an optional implementation scheme of the present application, optionally, it is characterized in that the floating-point multiplier and multiplicand can be inferred in the format of BFloat16 with higher precision under the same power consumption as FP8_e4m3.

[0024] According to the second aspect of the present application, a large language model acceleration device is provided, comprising the following modules:

[0025] The transformation formula module is used to convert the floating point multiplication formula into the integer addition formula;

[0026] The data acquisition module is used to obtain the floating point numbers to be calculated;

[0027] A calculation module, used to calculate the sign bit, exponent bit and mantissa bit of a floating point number;

[0028] The result acquisition module is used to stitch together the results of the step-by-step calculations to obtain the final result.

[0029] As an optional implementation scheme of the present application, optionally, the transformation formula module includes:

[0030] Get floating point multiplication formula module, used to get the multiplication formula of two floating point numbers;

[0031] A preliminary transformation module is used to obtain an approximate result formula of the two floating-point multiplication formulas;

[0032] The final transformation module is used to transform the approximate result formula into an integer addition formula.

[0033] According to the third aspect of the present application, an electronic device is proposed, comprising:

[0034] processor;

[0035] a memory for storing processor-executable instructions;

[0036] Wherein, the processor is configured to implement the acceleration method of a large language model and the acceleration device of a large language model as described in any one of claims 1 to 8 when executing the executable instructions.

[0037] Beneficial effects of this application:

[0038] The present invention converts floating-point multiplication into integer addition, and by directly modifying the exponent bit, reduces the number of instructions for calculation calls, thereby improving calculation efficiency. The present invention can also reduce power consumption while ensuring accuracy. Since general addition operations consume less energy than multiplication operations, power consumption during reasoning and training can be reduced. Under the same power consumption as FP8_e4m3, the BFloat16 format with higher accuracy can be used for reasoning, which can reduce quantization errors and ensure higher accuracy.

[0039] Other features and aspects of the present application will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the present application and, together with the description, serve to explain the principles of the present application.

[0041] Figure 1 A flowchart showing a method for accelerating a large language model according to an embodiment of the present application is shown;

[0042] Figure 2 A block diagram of an acceleration device for a large language model according to an embodiment of the present application is shown; DETAILED DESCRIPTION

[0043] Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0044] Among them, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application or simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0045] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0046] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0047] In addition, in order to better illustrate the present application, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present application can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present application.

[0048] The present invention uses integer addition to approximate floating-point multiplication, and directly uses exponential offset to express the calculation result, completely eliminating the multiplication operation. The reasoning process of determining the exponential offset value in this method has certain similarities with the optimal weights and benchmark models of training scenarios such as fine-tuning in large language models. Therefore, the characteristic of lower energy consumption of addition than multiplication in the present invention can effectively reduce the energy consumption of training and reasoning of large language models, thereby reducing the computing resource consumption of fine-tuning training and reasoning. Under power consumption equivalent to that of the FP8 model, a model with higher running accuracy of BFloat16 can achieve better model effects.

[0049] Example 1

[0050] Figure 1 FIG. 1 is a flow chart showing a method for accelerating a large language model according to Embodiment 1 of the present application. Figure 1 As shown, the flow chart includes:

[0051] S100. According to a multiplication formula of two floating-point numbers, a conversion formula for converting floating-point number multiplication into integer addition is obtained.

[0052] The floating-point number is a numerical type used in computers to represent real numbers. An approximate result formula is first obtained based on the multiplication formula of the two floating-point numbers, and then a conversion formula for converting the floating-point multiplication into integer addition is obtained by modifying the approximate result formula.

[0053] S200, obtaining relevant data of floating-point multiplier and multiplicand, and determining exponent offset;.

[0054] The floating point number is composed of a sign bit, a mantissa and an exponent, and the relevant data of the floating point multiplier and multiplicand are the sign bit, mantissa and exponent of the multiplier and multiplicand. The exponent offset is the difference between the coded value of the exponent domain and the actual exponent value in the floating point representation, which is used to simplify the operation. The exponent offset is determined by quantitative analysis of the data.

[0055] S300, substituting the floating-point multiplier, the sign bit, the exponent bit and the exponent offset of the multiplicand into the conversion formula to obtain an approximate sign bit and an approximate exponent bit of an approximate result.

[0056] The sign bits of the floating-point multiplier and the multiplicand are brought into the conversion formula for exclusive OR operation to obtain an approximate sign bit of an approximate result, and the exponent offset is moved to a position corresponding to the exponent bit, brought into the conversion formula and subtracted from the exponent bit to obtain an approximate exponent bit of the approximate result.

[0057] S400, correspondingly adding the mantissas of the floating-point multiplier and the multiplicand to obtain approximate mantissas of an approximate result.

[0058] In this embodiment, since 1 is automatically added to the mantissa when a floating point number is represented, the approximate mantissa of the approximate result can be obtained by correspondingly adding the mantissas of the floating point multiplier and the multiplicand.

[0059] S500 , combining the approximate sign bit, the approximate exponent bit and the approximate mantissa bit of the approximate result to obtain a final result.

[0060] In this embodiment, the calculated approximate sign bit is combined with the added approximate mantissa bits and the approximate exponent bit portion after the operation with the exponent offset to form an approximate result of the final product.

[0061] As an optional implementation scheme of the present application, optionally, in step S100, based on the multiplication formula of two floating-point numbers, a conversion formula for converting floating-point multiplication into integer addition is obtained, including: based on the multiplication formula of the two floating-point numbers, obtaining an intermediate formula of an approximate result; based on the intermediate formula, obtaining the conversion formula.

[0062] In this implementation, the multiplication of two floating point numbers is expressed as:

[0063]

[0064] Assume x m and m Represent the mantissa of the multiplier and multiplicand respectively. Then the intermediate formula for the approximate result is:

[0065]

[0066] The exponential offset e m It is obtained by quantitative analysis of the data. It is fixed for each tensor. Using a fixed negative integer power of 2 can directly add the offset to the exponent position, which can speed up subsequent calculations. According to the above intermediate water supply and exponent offset e m , we can conclude that the conversion formula for converting floating-point multiplication into integer addition is:

[0067]

[0068] Note that the above formula is a simplified way of writing, because the sign bit is actually a high bit, and the formula writes it to the position of 0. In addition, the exponent offset e m Move to the position corresponding to the exponent to subtract. Since 1 is automatically added to the mantissa when representing a floating point number, the mantissa can be added accordingly.

[0069] As an optional implementation scheme of the present application, optionally, in step S200, relevant data of the floating-point multiplier and the multiplicand are obtained, and the exponent offset is determined, wherein the relevant data of the floating-point multiplier and the multiplicand include the sign bit, exponent bit and mantissa bit of the floating-point multiplier and the multiplicand; this step includes: obtaining the sign bit, exponent bit and mantissa bit of the floating-point multiplier and the multiplicand; processing the sign bit, exponent bit and mantissa bit to obtain the exponent offset value.

[0070] In this embodiment, since the floating point number is composed of a sign bit, an exponent bit and a mantissa bit, the multiplication or addition of the floating point number needs to obtain the floating point multiplier, the sign bit, the exponent bit and the mantissa bit of the multiplicand. The exponent offset value is obtained by quantitatively analyzing the data. The following describes the method for determining the exponent offset value: Since x m and m are all decimals, so the default is between 0 and 1. m and m When it obeys the uniform distribution between [0,1], it can be concluded that x m and m The mean of is 1 / 2, then e m Should be 2. In actual data, x m and m At most, it can only cover, and because data is generally not evenly distributed, there are usually more parts close to 0. So from experience, e m The optimal value of should be slightly larger than 2. If this value exceeds m, it is likely to exceed the representation range of the floating point number, resulting in an out-of-bounds situation. Therefore, the upper limit of the search should not exceed m. So in the following determination of e m In the step, we will e m The search range of is set to [2,m]. Specifically, we can use the following method to obtain a better approximation of e m Method 1: All layers use the same value. Fix the number of bits of m. Prepare a certain number of samples, such as samples of some evaluation tasks GSM8K, generally take a batch of 1024 data. Input into a large language model such as Llama3.1, the model will generate the original output value at each tensor according to the inference step, as well as the e m For each value between [2,m], calculate the approximation using additive approximation multiplication. Then calculate the mean square error of these two values ​​and choose the one that minimizes the mean square error of the entire batch. m The value is used as the default exponent offset for multiplication of all layers. Method 2: Use different values ​​for different layers. For example, the 8B parameter version of Llama3.1 has a total of 32 LlamaDecoder layers. Different values ​​can be set for different layers. mInput a batch of test samples into the Llama3.1 model, and then follow the same steps as the above method to determine the final e that can approximate the original multiplication result for each layer of multiplication data. m , fixed as the value of this layer.

[0071] As an optional implementation scheme of the present application, optionally, in step S300, the sign bit, exponent bit and exponent offset of the floating-point multiplier and multiplicand are substituted into the conversion formula to obtain the approximate sign bit and approximate exponent bit of the approximate result, including: performing an XOR operation on the sign bits of the floating-point multiplier and multiplicand to obtain the approximate sign bit of the approximate result; the exponent offset needs to be moved to the position corresponding to the exponent bit for operation, and the exponent bit and exponent offset of the floating-point multiplier and multiplicand are calculated to obtain the approximate exponent bit of the approximate result.

[0072] In the present embodiment, the sign bits of the floating-point multiplier and the multiplicand are extracted and an exclusive-OR (XOR) operation is performed. The rule of the exclusive-OR operation is: if the sign bits of the two floating-point numbers are the same (both are 0 or both are 1), the approximate sign bit of the product is 0 (indicating a positive number); if the sign bits of the two floating-point numbers are different (one is 0 and the other is 1), the approximate sign bit of the product is 1 (indicating a negative number). The exponent bits of the floating-point multiplier and the multiplicand are extracted, the exponent bits of the floating-point multiplier and the multiplicand are added and the exponent offset is subtracted to obtain the approximate exponent bit part of the approximate result.

[0073] As an optional implementation scheme of the present application, optionally, the exponent offset needs to be moved to a position corresponding to the exponent bit for calculation in the conversion formula.

[0074] In this embodiment, in order to convert multiplication into addition, the calculation position of the exponent offset needs to be moved, and the exponent offset is fixed for each tensor. The offset can be directly added to the exponent bit through a fixed negative integer power of 2, and then the exponent offset is subtracted from the exponent bit to complete the calculation of the exponent bit part of the floating point number, thereby speeding up subsequent calculations, reducing the number of instructions called for calculations, and improving calculation efficiency.

[0075] As an optional implementation scheme of the present application, optionally, the value of the exponential offset is an exponential offset value obtained by quantitatively analyzing the data, including: the exponential offset value is obtained by inputting a certain number of sample data into a large language model for inference; in the large language model, the method for determining the exponential offset value can be used to train the model.

[0076] In this embodiment, the exponential offset value, i.e., the optimal offset value, is obtained by using a large language model for reasoning calculation, such as setting the search range of the exponential offset to [2, m], and fixing the number of digits of m. Prepare a certain number of samples, such as 1024 data in the sample of the evaluation task GSM8K. Input into a large language model such as Llama3.1, the model generates the original output value at each tensor according to the reasoning steps, and calculates the approximate value of the multiplication using addition approximation for each value of the exponential offset between [2, m]. Then calculate the mean square error of the two values, and select the exponential offset value that minimizes the mean square error of the entire batch. Use this value as the default exponential offset for multiplication of all layers. Although the optimal offset value is obtained by calculation in the reasoning process of the large language model, the optimal weight in training scenarios such as fine-tuning is generally similar to the benchmark model. Since general addition operations consume less energy than multiplication operations, the power consumption of the large language model during reasoning and training can be reduced.

[0077] As an optional implementation scheme of the present application, optionally, it is characterized in that the floating-point multiplier and multiplicand can be inferred in the format of BFloat16 with higher precision under the same power consumption as FP8_e4m3.

[0078] In this embodiment, FP8_e4m3 and BFloat16 are two different floating point representation formats.

[0079] FP8_e4m3 is an 8-bit floating point format, suitable for scenarios with high requirements on latency and video memory occupancy, while BFloat16 is a 16-bit floating point format that can improve computing performance while ensuring accuracy, and is widely used in the accelerated training process of machine learning and deep learning. In this implementation, under the same power consumption as FP8_e4m3, the BFloat16 format is used for inference, which can reduce quantization errors and ensure higher accuracy.

[0080] Example 2

[0081] Based on the same principle as the above method, an acceleration device for large language models is also proposed, see Figure 2 , an acceleration device 100 of a large language model according to an embodiment of the present disclosure includes:

[0082] A transformation formula module 110 is used to transform the floating point multiplication formula into an integer addition formula;

[0083] A data acquisition module 120 is used to acquire a floating point number to be calculated;

[0084] The calculation module 130 is used to calculate the sign bit, exponent bit and mantissa bit of the floating point number;

[0085] The result acquisition module 140 is used to combine the results of the step-by-step calculations to obtain the final result.

[0086] As an optional implementation scheme of the present application, optionally, the transformation formula module includes:

[0087] Get floating point multiplication formula module, used to get the multiplication formula of two floating point numbers;

[0088] A preliminary transformation module is used to obtain an approximate result formula of the two floating-point multiplication formulas;

[0089] The final transformation module is used to transform the approximate result formula into an integer addition formula.

[0090] Obviously, those skilled in the art should understand that all or part of the processes in the above-mentioned embodiment method can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned control methods. The modules or steps of the present invention can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. Optionally, they can be implemented by program codes executable by computing devices, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules respectively, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the present invention is not limited to any specific combination of hardware and software.

[0091] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment method can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned control methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, abbreviated as: HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0092] Example 3

[0093] Furthermore, an electronic device is proposed, comprising:

[0094] processor;

[0095] a memory for storing processor-executable instructions;

[0096] Wherein, the processor is configured to implement the acceleration method of the large language model described in Example 1 when executing the executable instructions.

[0097] The electronic device of the embodiment of the present disclosure includes a processor and a memory for storing processor executable instructions, wherein the processor is configured to implement any of the above-mentioned acceleration methods for large language models when executing the executable instructions.

[0098] It should be noted that the number of processors may be one or more. At the same time, the electronic device of the embodiment of the present disclosure may also include an input device and an output device. The processor, memory, input device, and output device may be connected via a bus or in other ways, which are not specifically limited here.

[0099] The memory is a computer-readable storage medium for the acceleration method of a large language model, and can be used to store software programs, computer executable programs, and various modules, such as: the program or module corresponding to the acceleration method of a large language model of the embodiment of the present disclosure. The processor executes various functional applications and data processing of the electronic device by running the software program or module stored in the memory.

[0100] The input device can be used to receive input numbers or signals. The signal can be a key signal related to user settings and function control of the device / terminal / server. The output device can include a display device such as a display screen.

[0101] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for accelerating a large language model, characterized in that: The multiplication operation is approximated by the addition operation, and the method comprises the following steps: Based on the multiplication formula of two floating-point numbers, the conversion formula for converting floating-point multiplication into integer addition is obtained; Get the relevant data of floating-point multiplier and multiplicand, and determine the exponent offset; Substituting the floating-point multiplier, the sign bit, the exponent bit and the exponent offset of the multiplicand into the conversion formula to obtain an approximate sign bit and an approximate exponent bit of the approximate result; Add the mantissas of the floating-point multiplier and the multiplicand correspondingly to obtain the approximate mantissas of the approximate result; The approximate sign bit, the approximate exponent bit and the approximate mantissa bit of the approximate result are combined to obtain a final result.

2. The acceleration method of a large language model as claimed in claim 1, characterized in that: According to the multiplication formula of two floating-point numbers, the conversion formula for converting floating-point multiplication into integer addition is obtained, including: According to the multiplication formula of the two floating-point numbers, an intermediate formula of an approximate result is obtained; According to the intermediate formula, the conversion formula is obtained.

3. The acceleration method of a large language model as claimed in claim 1, characterized in that: The step of obtaining the relevant data of the floating-point multiplier and the multiplicand and determining the exponent offset, wherein the relevant data of the floating-point multiplier and the multiplicand include the sign bit, the exponent bit and the mantissa bit of the floating-point multiplier and the multiplicand, comprises: Get the floating-point multiplier, the sign bit, the exponent bit and the mantissa bit of the multiplicand; The sign bit, exponent bit and mantissa bit are processed to obtain the exponent offset value.

4. The acceleration method of a large language model as claimed in claim 1, characterized in that: Substituting the floating-point multiplier, the sign bit, the exponent bit and the exponent offset of the multiplicand into the conversion formula, the approximate sign bit and the approximate exponent bit of the approximate result are obtained, including: Performing an XOR operation on the sign bits of the floating-point multiplier and the multiplicand to obtain the sign bit of the approximate result; The exponent offset needs to be moved to a position corresponding to the exponent bit for operation, and the exponent bits and exponent offsets of the floating-point multiplier and multiplicand are calculated to obtain the exponent bit of the approximate result.

5. The acceleration method of a large language model as claimed in claim 1, characterized in that: In the conversion formula, the exponent offset needs to be moved to the position corresponding to the exponent bit for calculation.

6. The acceleration method of a large language model as claimed in claim 3, characterized in that: The value of the exponential offset is an exponential offset value obtained by quantitatively analyzing the data, including: The exponential offset value is obtained by inputting a certain amount of sample data into a large language model for inference; In a large language model, the method for determining the exponential offset value may be used to train the model.

7. The acceleration method of a large language model as claimed in claim 3, characterized in that: The floating-point multiplier and multiplicand can be inferred in the higher-precision BFloat16 format under the same power consumption as FP8_e4m3.

8. An acceleration device for a large language model, comprising: The transformation formula module is used to convert the floating point multiplication formula into the integer addition formula; The data acquisition module is used to obtain the floating point numbers to be calculated; A calculation module, used to calculate the sign bit, exponent bit and mantissa bit of a floating point number; The result acquisition module is used to stitch together the results of the step-by-step calculations to obtain the final result.

9. The acceleration device for a large language model as claimed in claim 8, characterized in that Transformation formula module, including: Get floating point multiplication formula module, used to get the multiplication formula of two floating point numbers; A preliminary transformation module is used to obtain an approximate result formula of the two floating-point multiplication formulas; The final transformation module is used to transform the approximate result formula into an integer addition formula.

10. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement the acceleration method of a large language model and the acceleration device of a large language model as described in any one of claims 1 to 8 when executing the executable instructions.

Images