Floating point multiplication and addition instruction fusion method and device, equipment and storage medium
By merging the parameters of two floating-point multiply-accumulate operations into a single operation, a fused instruction is generated, which solves the problem of high GPU computational load caused by multiple floating-point multiply-accumulate instruction calls in the existing technology, and improves the GPU's computing efficiency and hardware resource utilization.
Patent Information
- Application Number
- CN202510888621.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-30
- Publication Date
- 2025-11-04
AI Technical Summary
The existing softmax algorithm involves a lot of calls to floating-point multiply-accumulate instructions for exponential approximation, which results in a heavy computational load on the GPU, especially in resource-constrained embedded GPU environments where computational efficiency is low.
By obtaining the parameters of two consecutive floating-point multiply-accumulate operations, the fusion scaling factor and fusion offset are calculated, and these parameters are combined into a single floating-point multiply-accumulate operation to generate a fusion instruction, reducing instruction pipeline overhead and improving GPU utilization.
By merging two FMA operations into a single FMA instruction, the instruction pipeline overhead in softmax is reduced, improving GPU utilization and computational efficiency while reducing computational latency.
Smart Images
Figure CN120892091A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present disclosure relates to the technical field of computers, and particularly relates to a floating-point multiply-add instruction fusion method and device, equipment and a storage medium. BACKGROUND
[0002] Traditional exponential function operations are usually implemented by calling exp() functions, but the operation is executed by a multi-function unit (MFU) in a GPU, and the throughput is low, which is prone to become the attention mechanism, such as the softmax function in FlashAttention, and there is a problem of low calculation efficiency.
[0003] At present, in order to improve the calculation efficiency, an exponential approximation technique is cited to process the floating-point multiply-add (FMA) instruction, such as the Schraudolph fast exponential method. Although this algorithm can reduce the instruction call, it still needs multiple instruction steps, including multiplication, addition, type conversion, etc. It can be seen that the number of called instructions is still relatively large, and the problem of low execution efficiency is more prominent in resource-limited environments such as embedded GPUs (such as Jetson Orin). SUMMARY
[0004] Therefore, the purpose of the present disclosure is to provide a floating-point multiply-add instruction fusion method, device, equipment and storage medium to solve the problem that the existing softmax exponential approximation calculation calls too many floating-point multiply-add instructions, resulting in a large operation load of the GPU.
[0005] In a first aspect, the embodiments of the present disclosure provide a floating-point multiply-add instruction fusion method, which comprises:
[0006] Obtaining two consecutive floating-point multiply-add operations in an original calculation process, and extracting parameters of the two floating-point multiply-add operations;
[0007] Calculating fusion parameters of the parameters of the two floating-point multiply-add operations, wherein the fusion parameters comprise a fusion scaling factor and a fusion offset;
[0008] Based on the fusion scaling factor and the fusion offset, the two floating-point multiply-add operations are merged to obtain a fusion instruction.
[0009] In a feasible implementation, the extraction of the parameters of the two floating-point multiply-add operations comprises:
[0010] Determining a target element value from a specified position in an input matrix of the two floating-point multiply-add operations;
[0011] Based on a pre-defined model parameter and a normalization algorithm, calculating a scaling factor and a maximum value offset of the first floating-point multiply-add operation under the target element value;
[0012] The exponent parameter and the modulus conversion parameter of the second floating-point multiplication and addition operation are calculated based on a preset displacement bit number, a logarithm constant, a target floating-point precision, and hardware architecture characteristics.
[0013] In an implementation, the fusion parameters of the parameters of the two floating-point multiplication and addition operations include:
[0014] A fusion scaling factor of the parameters of the two floating-point multiplication and addition operations is calculated based on the scaling factor and the exponent parameter.
[0015] A fusion offset of the parameters of the two floating-point multiplication and addition operations is calculated based on the modulus conversion parameter, the exponent parameter, and the maximum value offset.
[0016] In an implementation, the two floating-point multiplication and addition operations are merged based on the fusion scaling factor and the fusion offset to obtain a fusion instruction, including:
[0017] In the original calculation process, a single floating-point multiplication and addition operation is performed based on the fusion scaling factor and the fusion offset, and two consecutive floating-point multiplication and addition operations are replaced based on a result of the performance to obtain a fusion instruction.
[0018] In an implementation, the single floating-point multiplication and addition operation is performed in the original calculation process based on the fusion scaling factor and the fusion offset, including:
[0019] In the original calculation process, the two consecutive floating-point multiplication and addition operations are multiplied by the fusion scaling factor and added to the fusion offset.
[0020] In an implementation, after the single floating-point multiplication and addition operation is performed in the original calculation process based on the fusion scaling factor and the fusion offset, the method further includes:
[0021] The result of the performance is converted into an integer and subjected to a bit operation to generate a new result of the performance.
[0022] In an implementation, the result of the performance is converted into an integer and subjected to a bit operation to generate a new result of the performance, including:
[0023] The floating-point value of the result of the performance is converted into an integer based on the modulus conversion parameter in the parameters of the two floating-point multiplication and addition operations.
[0024] The integer is subjected to a left shift operation and added to a correction term to generate a new result of the performance that meets a preset standard, and the new result of the performance is a floating-point number.
[0025] In a second aspect, the embodiments of the present disclosure provide a floating-point multiplication and addition instruction fusion device, including:
[0026] The acquisition module is configured to acquire two continuous floating-point multiply-add operations in an original calculation process and extract parameters of the two floating-point multiply-add operations.
[0027] The parameter fusion module is configured to calculate fusion parameters of the parameters of the two floating-point multiply-add operations, wherein the fusion parameters include a fusion scaling factor and a fusion offset.
[0028] The instruction merging module is configured to merge the two floating-point multiply-add operations based on the fusion scaling factor and the fusion offset to obtain a fusion instruction.
[0029] In a third aspect, an electronic device is provided, which includes a processor and a memory. The memory stores machine executable instructions capable of being executed by the processor. The processor executes the machine executable instructions to implement the floating-point multiply-add instruction fusion method provided above.
[0030] In a fourth aspect, a computer readable storage medium is provided, which stores computer executable instructions. When the computer executable instructions are invoked and executed by a processor, the computer executable instructions cause the processor to implement the floating-point multiply-add instruction fusion method provided above.
[0031] The embodiments of the present disclosure bring the following beneficial effects:
[0032] The floating-point multiply-add instruction fusion method, device, equipment and storage medium provided above acquire two continuous floating-point multiply-add (FMA) operations in an original calculation process and extract parameters of the two floating-point multiply-add operations. Fusion parameters of the parameters of the two floating-point multiply-add operations are calculated, wherein the fusion parameters include a fusion scaling factor and a fusion offset. The two floating-point multiply-add operations are merged based on the fusion scaling factor and the fusion offset to obtain a fusion instruction.
[0033] In the method, two FMA operations are merged into a single FMA instruction module, and single FMA reduces the instruction pipeline occupation in softmax, thereby improving GPU utilization.
[0034] Other features and advantages of the present disclosure will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present disclosure. The objects and other advantages of the present disclosure will be realized and achieved by means of the structures particularly pointed out in the description, claims, and drawings.
[0035] In order to make the above objectives, features and advantages of the present disclosure more apparent, the following will specifically describe preferred embodiments in combination with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS
[0036] In order to make the technical solutions in the specific embodiments of the present disclosure or the prior art clearer, the accompanying drawings needed in the specific embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description are some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without any creative work on the premise of the accompanying drawings.
[0037] Figure 1 An embodiment schematic diagram of the floating-point multiply-add instruction fusion method provided by the embodiments of the present disclosure is shown in the following figure.
[0038] Figure 2 Another embodiment schematic diagram of the floating-point multiply-add instruction fusion method provided by the embodiments of the present disclosure is shown in the following figure.
[0039] Figure 3 An embodiment schematic diagram of the floating-point multiply-add instruction fusion device provided by the embodiments of the present disclosure is shown in the following figure.
[0040] Figure 4 Another embodiment schematic diagram of the floating-point multiply-add instruction fusion device provided by the embodiments of the present disclosure is shown in the following figure.
[0041] Figure 5 An embodiment schematic diagram of an electronic device provided by the embodiments of the present disclosure is shown in the following figure. EMBODIMENT
[0042] In order to make the technical solutions in the specific embodiments of the present disclosure or the prior art clearer, the accompanying drawings needed in the specific embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description are some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without any creative work on the premise of the accompanying drawings.
[0043] The floating-point multiply-add instruction fusion method in one of the embodiments of the present disclosure can run on a terminal device or a server. The terminal device can be a local terminal device. When the floating-point multiply-add instruction fusion method runs on the server, the method can be implemented and executed based on a cloud interaction system, wherein the cloud interaction system includes a server and a client device.
[0044] For the convenience of understanding, the specific process of the present embodiment will be described below. Please refer to Figure 1 One embodiment of the floating-point multiply-add instruction fusion method in the present embodiment is used to process the call of the GPU and the FMA operation in the autonomous vehicle, and the method specifically includes the following steps:
[0045] 101, obtaining two consecutive floating-point multiply-add operations in the original calculation process, and extracting the parameters of the two floating-point multiply-add operations;
[0046] It should be noted that the two consecutive floating-point multiply-add operations can be connected as two separate FMA operations that are time-continuous, which are a combination of multiply and add operations, and two consecutive times are two separate multiply and add operations.
[0047] Specifically, the FMA operations in the softmax calculation process are collected from the vehicle GPU through a tracking tool, and two consecutive FMA operations are converted into a matrix, and then element extraction is performed according to the dimensions in the matrix to obtain the parameters of the two floating-point multiply-add operations.
[0048] In this embodiment, the parameters of the two floating-point multiply-add operations include the scaling factor and the maximum value offset of the first FMA operation, and the exponential operation and the modulus conversion parameter of the second FMA operation.
[0049] Specifically, the target element value is determined from the specified position in the input matrix of the two floating-point multiply-add operations; based on the pre-defined model parameters and the normalization algorithm, the scaling factor and the maximum value offset of the first floating-point multiply-add operation under the target element value are calculated; based on the pre-set bit shift number, logarithmic constant, target floating-point precision and hardware architecture characteristics, the exponential parameter and modulus conversion parameter of the second floating-point multiply-add operation are calculated.
[0050] 102, the fusion parameters of the parameters of the two floating-point multiply-add operations are calculated, and the fusion parameters include a fusion scaling factor and a fusion offset;
[0051] In this embodiment, the mathematical equivalent exchange principle is used to calculate that in the softmax calculation of the attention mechanism, in order to ensure numerical stability, the original process needs to perform two floating-point multiply-add (FMA) operations:
[0052] The first FMA operation: scaling the input matrix elements and subtracting the maximum value;
[0053] The second FMA operation: applying the exponential parameter and adding the modulus conversion parameter;
[0054] Through mathematical equivalent transformation (distribution law), the two operations are combined into a single FMA.
[0055] Wherein, the calculation of the fusion scaling factor is to first calculate the exponential parameter, then multiply the exponential parameter with the scaling factor to obtain the fusion scaling factor; the calculation of the fusion offset is to first calculate the maximum value offset, which is specifically calculated based on the maximum value of the row or column in the matrix, and then linearly combined with the modulus conversion parameter.
[0056] 103, based on the fusion scaling factor and the fusion offset, the two floating-point multiply-add operations are combined to obtain a fusion instruction.
[0057] The fusion instruction refers to the control of the two consecutive FMA operations together by means of software command, specifically by means of the fusion scaling factor and the fusion offset calculated above.
[0058] It can be understood that the single floating-point multiply-add operation is performed, the input element, the fusion scaling factor and the fusion offset generate an intermediate result; the intermediate result is converted into an integer and a bit operation is performed to generate a final exponent value, so as to realize the fusion of two floating-point multiply-add operations.
[0059] By executing the above-mentioned scheme, the fusion parameters of two floating-point multiply-add operations are calculated by mathematical transformation, and the fusion based on the fusion parameters is performed, thereby reducing the number of FMA operations, reducing the FMA calling load of the GPU, reducing the calculation delay and improving the utilization of the corresponding hardware.
[0060] Please refer to Figure 2 Another embodiment of the floating-point multiply-add instruction fusion method in the embodiment specifically includes the following steps:
[0061] 201, obtaining two consecutive floating-point multiply-add operations in the original calculation flow, and extracting parameters of the two floating-point multiply-add operations;
[0062] In this embodiment, the input element, the scaling factor, the maximum value offset, the exponent parameter and the modulus conversion parameter are extracted from the two floating-point multiply-add operations, and these parameters are specifically implemented by the following method:
[0063] The target element value is determined from the specified position of the input matrix of the two floating-point multiply-add operations;
[0064] Based on the pre-defined model parameters and the normalization algorithm, the scaling factor and the maximum value offset of the first floating-point multiply-add operation under the target element value are calculated;
[0065] Based on the pre-set bit shift number, the logarithmic constant, the target floating-point precision and the hardware architecture characteristics, the exponent parameter and the modulus conversion parameter of the second floating-point multiply-add operation are calculated.
[0066] Based on the pre-defined model parameters and the normalization algorithm, the scaling factor and the maximum value offset of the first floating-point multiply-add operation under the target element value are calculated, including:
[0067] The current element value obtained from the specified position of the input matrix is taken as the input parameter of the first floating-point multiply-add operation; the scaling factor is determined according to the pre-defined model parameters or the dynamic calculation rule; the maximum value of the input matrix is calculated through the maximum value normalization process, and the maximum value offset is generated.
[0068] The exponent parameter and the modulus conversion parameter of the second floating-point multiplication and addition operation are calculated based on the preset displacement bit number, logarithm constant, target floating-point precision and hardware architecture characteristics, including:
[0069] The first floating-point multiplication and addition operation is performed to generate an intermediate result and temporarily store; the exponent parameter of the second floating-point multiplication and addition operation is calculated according to the predetermined displacement bit number and logarithm constant; and the modulus conversion parameter is selected based on the target floating-point precision and hardware architecture characteristics.
[0070] In another embodiment, in order to avoid the influence of floating-point overflow on calculation accuracy when calculating the fusion parameter, the input matrix element value is also limited to the range of -6.0 to 6.0; and the floating-point overflow is avoided by subtracting the maximum value in the input matrix.
[0071] 202, a fusion scaling factor of the parameters of the two floating-point multiplication and addition operations is calculated based on the scaling factor and the exponent parameter;
[0072] It can be understood that the fusion scaling factor actually combines the scaling effects of the two operations, specifically, the scaling factor of the first floating-point multiplication and addition operation is multiplied by the exponent parameter of the second floating-point multiplication and addition operation to obtain the fusion scaling factor.
[0073] 203, a fusion offset of the parameters of the two floating-point multiplication and addition operations is calculated based on the modulus conversion parameter, the exponent parameter and the maximum value offset;
[0074] It can be understood that the fusion offset actually unifies the two offsets into a single compensation, specifically, the product of the modulus conversion parameter of the second floating-point multiplication and addition operation and the maximum value offset of the first floating-point multiplication and addition operation and the exponent parameter is subtracted to obtain the fusion offset.
[0075] 204, in the original calculation process, a single floating-point multiplication and addition operation is performed based on the fusion scaling factor and the fusion offset;
[0076] This step mainly uses the fusion scaling factor and the fusion offset to perform a single floating-point multiplication and addition operation in the calculation loop; specifically, in the original calculation process, the two consecutive floating-point multiplication and addition operations are multiplied by the fusion scaling factor and added to the fusion offset.
[0077] In another embodiment, after the single floating-point multiplication and addition operation is performed based on the fusion scaling factor and the fusion offset in the original calculation process, the method further includes:
[0078] The result of the execution is converted into an integer and a bit operation is performed to generate a new result of the execution.
[0079] Specifically, the floating point value of the execution result is converted into an integer by a modulus conversion parameter in the parameters of two floating point multiplication and addition operations; the integer is left shifted and a correction term is added to generate a new execution result meeting a preset standard, and the new execution result is a floating point number.
[0080] It should be noted that when performing the bit operation, it is necessary to determine whether the hardware supports the FP16 data type, if so, the bit shift number is adjusted to be FP16_SHIFT_1 = 6, and the modulus conversion parameter is 0x1C00; two FP16 values are processed in parallel by a half2 instruction to improve the throughput.
[0081] Wherein, when performing the integer conversion, the flow is:
[0082] After the FP16 floating point result is converted into an integer, the modulus 0x1C00 is subtracted;
[0083] The integer is left shifted by 4 bits and a correction term 0x3C00 is added to generate a final FP16 exponent value.
[0084] 205, replacing the two consecutive floating point multiplication and addition operations based on the execution result to obtain a fused instruction.
[0085] The floating point multiplication and addition instruction fusion method combines two FMA operations into a single operation by the mathematical equivalent exchange principle, generates a fusion scaling factor and a fusion offset by linear combination, to realize instruction fusion, that is, to adapt to different precisions and GPU architectures, improve the calculation efficiency, reduce the calculation complexity, and significantly improve the hardware resource utilization and the inference speed.
[0086] Corresponding to the above method embodiment, referring to Figure 3 a schematic diagram of a floating point multiplication and addition instruction fusion device is shown, which specifically comprises:
[0087] The acquisition module 310 is configured to acquire two consecutive floating point multiplication and addition operations in an original calculation flow, and extract parameters of the two floating point multiplication and addition operations;
[0088] The parameter fusion module 320 is configured to calculate fusion parameters of the parameters of the two floating point multiplication and addition operations by the mathematical equivalent exchange principle, wherein the fusion parameters include a fusion scaling factor and a fusion offset;
[0089] The instruction merging module 330 is configured to merge the two floating point multiplication and addition operations based on the fusion scaling factor and the fusion offset to obtain a fused instruction.
[0090] The aforementioned floating-point multiply-accumulate instruction fusion device calculates the fusion scaling factor and fusion offset of two consecutive floating-point multiply-accumulate operations using the principle of mathematical equivalence commutation. Based on these factors, it merges the two operations into a single fusion instruction. This mathematical equivalence transformation combines two FMA operations into a single instruction, addressing the problem in existing softmax algorithms where exponential approximation calculations involve numerous calls to floating-point multiply-accumulate instructions, leading to high GPU computational load, computational latency, and low hardware utilization.
[0091] Please see Figure 4 Another embodiment of the floating-point multiply-accumulate instruction fusion device in this application includes:
[0092] The acquisition module 310 is used to acquire two consecutive floating-point multiply-accumulate operations in the original calculation process and extract the parameters of the two floating-point multiply-accumulate operations;
[0093] The parameter fusion module 320 is used to calculate the fusion parameters of the parameters of two floating-point multiply-add operations, wherein the fusion parameters include a fusion scaling factor and a fusion offset;
[0094] The instruction merging module 330 is used to merge two floating-point multiply-accumulate operations based on the merging scaling factor and the merging offset to obtain a merging instruction.
[0095] Optionally, the acquisition module 310 includes:
[0096] The determining unit 311 is used to determine the target element value from a specified position in the input matrix of the two floating-point multiply-add operations;
[0097] The calculation unit 312 is used to calculate the scaling factor and maximum value offset of the first floating-point multiply-accumulate operation under the target element value based on predefined model parameters and normalization algorithm; and to calculate the exponent parameters and analog-to-digital conversion parameters of the second floating-point multiply-accumulate operation based on preset displacement bits, logarithmic constant, target floating-point precision and hardware architecture characteristics.
[0098] Optionally, the parameter fusion module 320 includes:
[0099] The first fusion unit 321 is used to calculate the fusion scaling factor of the parameters of the two floating-point multiply-add operations based on the scaling factor and the exponent parameter.
[0100] The second fusion unit 322 is used to calculate the fusion offset of the parameters of the two floating-point multiply-accumulate operations based on the analog-to-digital conversion parameters, the exponent parameters, and the maximum value offset.
[0101] Optionally, the instruction merging module 330 is specifically used for:
[0102] In the original calculation flow, a single floating-point multiply-add operation is performed based on the fusion scaling factor and the fusion offset, and two consecutive floating-point multiply-add operations are replaced based on the result after the execution to obtain the fusion instruction.
[0103] Optionally, the instruction merging module 330 is specifically used for:
[0104] In the original calculation flow, two consecutive floating-point multiply-add operations are multiplied by the fusion scaling factor and added to the fusion offset.
[0105] Optionally, the instruction merging module 330 is specifically used for:
[0106] The result of the execution is converted to an integer and a bit operation is performed to generate a new result of the execution.
[0107] Optionally, the instruction merging module 330 is specifically used for:
[0108] Based on the modulus conversion parameter in the parameters of the two floating-point multiply-add operations, the floating-point value of the result of the execution is converted to an integer;
[0109] The integer is left-shifted and a correction term is added to generate a new result of the execution that meets a preset standard, and the new result of the execution is a floating-point number.
[0110] The embodiment also provides an electronic device including a processor and a memory, the memory storing machine executable instructions capable of being executed by the processor, and the processor executes the machine executable instructions to implement the floating-point multiply-add instruction fusion method described above. The electronic device can be a server or a terminal device.
[0111] Referring to Figure 5 The electronic device includes a processor 500 and a memory 501, the memory 501 storing machine executable instructions capable of being executed by the processor 500, and the processor 500 executes the machine executable instructions to implement the floating-point multiply-add instruction fusion method provided by the above embodiment.
[0112] Further, Figure 5 The electronic device shown also includes a bus 502 and a communication interface 503, and the processor 500, the communication interface 503 and the memory 501 are connected through the bus 502.
[0113] The memory 501 can include a random access memory (RAM) and can also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 503 (which can be wired or wireless), and the Internet, a wide area network, a local area network, a metropolitan area network, etc. can be used. The bus 502 can be an ISA bus, a PCI bus, an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For convenience of representation, Figure 5 Only one bidirectional arrow is used to represent the system in the figure, but it does not mean that there is only one bus or one type of bus.
[0114] The processor 500 can be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware in the processor 500 or the instruction in the form of software. The above processor 500 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. Each method, step and logic block disclosed in the embodiment can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiment can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium in the art. The storage medium is located in the memory 501, and the processor 500 reads the information in the memory 501, and combines the hardware to complete the steps of the floating-point multiplication and addition instruction fusion method provided by the above embodiment.
[0115] The embodiment also provides a computer readable storage medium, the computer readable storage medium stores computer executable instructions, when the computer executable instructions are called and executed by the processor, the computer executable instructions cause the processor to implement the floating-point multiplication and addition instruction fusion method provided by the above embodiment.
[0116] The floating-point multiply-add instruction fusion method, device, electronic equipment and computer program product of the storage medium provided by the embodiment include a computer readable storage medium storing program codes, the program codes include instructions for executing the method described in the foregoing method embodiments, and specific implementation can be referred to the method embodiments, which will not be described here.
[0117] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.
[0118] In addition, in the description of the embodiment, unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connection" should be understood broadly, for example, it can be fixed connection, or detachable connection, or integral connection; it can be mechanical connection, or electrical connection; it can be direct connection, or indirect connection through intermediate medium, or internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the disclosure can be understood according to the specific circumstances.
[0119] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the disclosure essentially or the parts that make contributions to the prior art or parts of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in various embodiments of the disclosure. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and various program code storage media.
[0120] In the description of the disclosure, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the disclosure and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, therefore it cannot be understood as a limitation on the disclosure. In addition, the terms "first", "second", "third" are only for the purpose of description, and cannot be understood as indicating or implying relative importance.
[0121] Finally, it should be noted that the above examples are merely specific embodiments of the present disclosure, used to illustrate the technical solutions of the present disclosure, and are not intended to limit the present disclosure. The protection scope of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the foregoing examples, those skilled in the art should understand that any person skilled in the art can still make modifications or easily think of changes to the technical solutions recorded in the foregoing examples, or make equivalent replacements to some of the technical features, within the technical scope disclosed by the present disclosure. Such modifications, changes or replacements do not cause the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the present embodiment, and should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.
Claims
1. A method for fusing floating-point multiply-accumulate instructions, characterized in that, The method includes: Obtain two consecutive floating-point multiply-accumulate operations in the original calculation process, and extract the parameters of the two floating-point multiply-accumulate operations; Calculate the fusion parameters of the parameters of the two floating-point multiply-accumulate operations, wherein the fusion parameters include a fusion scaling factor and a fusion offset; Based on the fusion scaling factor and the fusion offset, the two floating-point multiply-accumulate operations are merged to obtain the fusion instruction.
2. The floating-point multiply-accumulate instruction fusion method according to claim 1, characterized in that, The extraction of parameters for the two floating-point multiplication and addition operations includes: Determine the target element value from a specified position in the input matrix of two floating-point multiply-accumulate operations; Based on predefined model parameters and normalization algorithms, the scaling factor and maximum value offset of the first floating-point multiply-accumulate operation under the target element value are calculated; Based on the preset number of bits for displacement, logarithmic constant, target floating-point precision, and hardware architecture characteristics, the exponent parameters and analog-to-digital conversion parameters for the second floating-point multiply-accumulate operation are calculated.
3. The floating-point multiply-accumulate instruction fusion method according to claim 2, characterized in that, The fusion parameters, which calculate the parameters of the two floating-point multiply-add operations, include: Based on the scaling factor and the exponent parameter, the fusion scaling factor of the parameters of the two floating-point multiply-accumulate operations is calculated; Based on the analog-to-digital conversion parameters, the exponent parameters, and the maximum value offset, the fused offset of the parameters for the two floating-point multiply-accumulate operations is calculated.
4. The floating-point multiply-accumulate instruction fusion method according to claim 1, characterized in that, The process of merging two floating-point multiply-accumulate operations based on the fusion scaling factor and the fusion offset to obtain a fusion instruction includes: In the original calculation process, a single floating-point multiply-accumulate operation is performed based on the fusion scaling factor and the fusion offset, and the result of the operation is used to replace the two consecutive floating-point multiply-accumulate operations to obtain the fusion instruction.
5. The floating-point multiply-accumulate instruction fusion method according to claim 4, characterized in that, The step of performing a single floating-point multiply-add operation based on the fusion scaling factor and the fusion offset in the original calculation process includes: In the original calculation process, two consecutive floating-point multiply-add operations are multiplied by the fusion scaling factor and added to the fusion offset.
6. The floating-point multiply-accumulate instruction fusion method according to claim 5, characterized in that, In the original calculation process, after performing a single floating-point multiply-accumulate operation based on the fusion scaling factor and the fusion offset, the process further includes: The result of the execution is converted into an integer and bitwise operations are performed to generate a new result.
7. The floating-point multiply-accumulate instruction fusion method according to claim 6, characterized in that, The step of converting the execution result into an integer and performing bit operations to generate a new execution result includes: Based on the analog-to-digital conversion parameter in the parameters of the two floating-point multiplication and addition operations, the floating-point value of the result is converted into an integer; The integer is left-shifted and a correction term is added to generate a new execution result that conforms to a preset standard. The new execution result is a floating-point number.
8. A floating-point multiply-accumulate instruction fusion device, characterized in that, The device includes: The acquisition module is used to acquire two consecutive floating-point multiply-accumulate operations in the original calculation process and extract the parameters of the two floating-point multiply-accumulate operations; The parameter fusion module is used to calculate the fusion parameters of the parameters of two floating-point multiply-accumulate operations, wherein the fusion parameters include a fusion scaling factor and a fusion offset; The instruction merging module is used to merge two floating-point multiply-accumulate operations based on the merging scaling factor and the merging offset to obtain a merging instruction.
9. An electronic device, characterized in that, The method includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor, the processor executing the machine-executable instructions to implement the floating-point multiply-accumulate instruction fusion method according to any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the floating-point multiply-accumulate instruction fusion method according to any one of claims 1-7.