Data processing method and processor

Abstract

The present application relates to the technical field of data processing, and provides a data processing method and a processor. The processor is used for executing training or inference of an artificial intelligence model. The processor comprises an acquisition module, a dot-product module, and an accumulation module. The acquisition module is used for acquiring K pairs of floating-point numbers, the K pairs of floating-point numbers being respectively obtained by combining components at identical positions in two K-dimensional floating-point number vectors to be subjected to a dot-product operation, the K pairs of floating-point numbers being divided into K1 first floating-point arrays, and first floating-point array i comprising Pi pairs of floating-point numbers, where 1≤i≤K1. The dot-product module is used for performing, for first floating-point array i, a dot-product operation on the Pi pairs of floating-point numbers within the first floating-point array i to obtain K1 first floating-point numbers. The accumulation module is used for accumulating the K1 first floating-point numbers and obtaining a second floating-point number. The present application can reduce the area and power consumption of the processor, and effectively improve the energy efficiency of matrix multiplication operations. The present application is used for performing a dot-product operation on floating-point number vectors.

Description

Data processing method and processor

[0001] The present application claims priority to the Chinese patent application No. 202510237913.3, filed on February 27, 2025, and entitled "Data processing method and processor", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of data processing, in particular to a data processing method and a processor. BACKGROUND

[0003] Matrix multiplication is the most basic and most heavily loaded operation task of current artificial intelligence (AI) technology, and is crucial to the training and inference of deep learning. In the training forward and inference process, each layer of the neural network needs to implement the multiplication of the activation value matrix and the weight matrix to obtain new activation values. In the training backward, each layer of the neural network needs to implement the calculation of two matrix multiplications.

[0004] With the development of large language models (LLMs), the matrix multiplication operation of the network presents an exponential growth. Since AI technology has a high demand for electricity, and a large number of computing activities of AI technology are based on matrix multiplication operations, in order to improve the energy efficiency of AI technology, how to reduce the power consumption of matrix multiplication operations has become a problem to be solved. SUMMARY

[0005] The present application provides a data processing method and a processor, which can reduce the area and power consumption of the processor and effectively improve the energy efficiency of matrix multiplication operations.

[0006] In a first aspect, the present application provides a processor for executing the training or inference of an artificial intelligence model, the processor comprising an acquisition module, a point multiplication module and an accumulation module; the acquisition module is configured to acquire K pairs of floating-point numbers, the K pairs of floating-point numbers being obtained by combining components at the same positions in two K-dimensional floating-point number vectors to be multiplied, and the K pairs of floating-point numbers being divided into K1 first floating-point arrays, the first floating-point array i including Pi pairs of floating-point numbers, 1≤i≤K1; the point multiplication module is configured to multiply the Pi pairs of floating-point numbers in the first floating-point array i to obtain K1 first floating-point numbers; and the accumulation module is configured to accumulate the K1 first floating-point numbers to obtain a second floating-point number.

[0007] In the above, the two K-dimensional floating-point number vectors are respectively a row element and a column element in matrix multiplication. The K-dimensional floating-point number vector includes K floating-point number elements.

[0008] In the present application, the multiplication of K pairs of floating-point numbers is divided into two stages. The first stage of multiplication operation completes the multiplication of all pairs of floating-point numbers in each first floating-point array, and the second stage of accumulation operation completes the accumulation of K1 floating-point numbers. In the first stage of multiplication operation, the exponent of Pi floating-point products is found to find the maximum exponent. The maximum exponent is found quickly, and the addition process of floating-point products is also fast, with small time delay. Therefore, the area and power consumption of the processor are reduced, and the energy efficiency of the matrix multiplication operation is effectively improved.

[0009] In a possible implementation, the K pairs of floating-point numbers can be unevenly divided into K1 first floating-point arrays.

[0010] In a possible implementation, the K pairs of floating-point numbers can be evenly divided into K1 first floating-point arrays. That is, each first floating-point array includes K0 pairs of floating-point numbers, and K=K0*K1.

[0011] In a possible implementation, K0≥K1. In the present application, K0≥K1, so that the number of floating-point numbers accumulated by each sub-multiplication module in the first stage of multiplication operation is greater than or equal to the number of floating-point numbers accumulated in the second stage of accumulation operation, thereby further reducing the area and power consumption of the processor.

[0012] In a possible implementation, the decimal bit width involved in the multiplication process performed by the multiplication module is less than or equal to the decimal bit width involved in the accumulation process performed by the accumulation module. For example, the multiplication module is specifically configured to: multiply Pi pairs of floating-point numbers in the first floating-point array i to obtain Pi floating-point products; perform exponentiation, shifting, and rounding on the Pi floating-point products to obtain Pi first significant bits with a first bit width; and accumulate the Pi first significant bits to obtain K1 first floating-point numbers. The accumulation module is specifically configured to: perform exponentiation, shifting, and rounding on the K1 first floating-point numbers to obtain K1 second significant bits with a second bit width; and accumulate the K1 second significant bits to obtain a second floating-point number.

[0013] In a possible implementation, the decimal bit width involved in the multiplication process performed by the multiplication module is less than or equal to the decimal bit width involved in the accumulation process performed by the accumulation module. For example, the multiplication module is specifically configured to: multiply Pi pairs of floating-point numbers in the first floating-point array i to obtain Pi floating-point products; perform exponentiation, shifting, and rounding on the Pi floating-point products to obtain Pi first significant bits with a first bit width; and accumulate the Pi first significant bits to obtain K1 first floating-point numbers. The accumulation module is specifically configured to: perform exponentiation, shifting, and rounding on the K1 first floating-point numbers to obtain K1 second significant bits with a second bit width; and accumulate the K1 second significant bits to obtain a

[0014] In this embodiment, calculating the dot product of all pairs of floating-point numbers within K1 first floating-point arrays is called the first-level dot product operation, and accumulating the K1 first floating-point numbers to obtain the second floating-point number is called the second-level accumulation operation. The first-level dot product operation involves parallel accumulation and compression of a relatively large number of floating-point numbers, and its overhead is usually large. The second-level accumulation operation, on the other hand, accumulates a small number of floating-point numbers, with lower parallelism and usually lower overhead. Therefore, setting the first bit width smaller than the second bit width can prevent the overhead of the first-level dot product operation from becoming too large, while also ensuring the accuracy of the second-level accumulation operation. This effectively reduces the overhead of the first-level dot product operation while maintaining the accuracy requirements of matrix multiplication, thereby reducing the processor area and power consumption, and effectively improving the energy efficiency of matrix multiplication operations. Furthermore, setting the first bit width smaller also allows the addition process of sub-dot product module i on Pi floating-point products in the first-level dot product operation to be faster and with lower latency.

[0015] In one possible implementation, the accumulation module includes L levels of accumulation units. The input floating-point number of the j-th level accumulation unit is divided into Nj second floating-point arrays, where 1 ≤ j < L. The j-th level accumulation unit is used to respectively divide N... j All floating-point numbers in the second floating-point array are summed into a third floating-point number, and N is output to the (j+1)th level accumulation unit. j The Lth stage is a third floating-point number; the Lth stage is an accumulation unit used to accumulate all the input third floating-point numbers into a single floating-point number to obtain the second floating-point number. In this application, the accumulation process is further segmented, which can further reduce the processor area and power consumption.

[0016] In one possible implementation, the decimal place width involved in the dot product process performed by the dot product module is less than or equal to the decimal place width involved in the accumulation process performed by the accumulation module. Furthermore, the decimal place width involved in the accumulation process performed by the j-th level accumulation unit is less than or equal to the decimal place width involved in the accumulation process performed by the (j+1)-th level accumulation unit.

[0017] For example, the dot product module is specifically used to: multiply Pi floating-point numbers in the first floating-point array i by Pi floating-point numbers to obtain Pi floating-point products; perform alignment, shifting, and rounding on the Pi floating-point products to obtain Pi third significant bits with a decimal width of the third bit width; and accumulate the Pi third significant bits to obtain K1 first floating-point numbers. The j-th level accumulation unit is specifically used to: multiply N... j All floating-point numbers within each of the second floating-point arrays are aligned, shifted, and rounded to obtain a decimal place width of B. j Multiple fourth significant bits; summing multiple fourth significant bits to obtain N. j The Lth level accumulation unit is specifically used to: realign, shift, and round all input third-level floating-point numbers to obtain a fractional part with a width of B. La plurality of fourth significant bits; and accumulating according to the plurality of fourth significant bits to obtain the second floating-point number.

[0018] wherein the third bit width is a fractional bit width involved in a multiplication process performed by the point multiplication module, B j is a fractional bit width involved in an accumulation process performed by the jth accumulation unit, B L is a fractional bit width involved in an accumulation process performed by the Lth accumulation unit. The third bit width is less than or equal to B j , B j is less than or equal to B j+1 . When j = L-1, B j+1 is B L .

[0019] In the embodiments of the present application, the smaller the level of the accumulation unit is, the more floating-point numbers the accumulation unit needs to accumulate, and the greater the accumulation overhead is. Therefore, the smaller the level of the accumulation unit is, the smaller the fractional bit width involved in the accumulation process performed by the accumulation unit is, which can reduce the area and power consumption of the processor on the basis of guaranteeing the accuracy requirement of the matrix multiplication, and effectively improve the energy efficiency of the matrix multiplication operation. Moreover, the smaller the fractional bit width involved in the accumulation process performed by the accumulation unit is, the faster the accumulation operation of the accumulation unit is, and the smaller the time delay is.

[0020] In a possible implementation, the accumulation module is further configured to add the second floating-point number and a source floating-point scalar to obtain a target floating-point scalar.

[0021] In a second aspect, the present application provides a data processing method applied to a processor, the processor being configured to perform training or inference of an artificial intelligence model, the processor comprising an acquisition module, a point multiplication module, and an accumulation module; the method comprising: the acquisition module acquiring K pairs of floating-point numbers, the K pairs of floating-point numbers being obtained by combining components at the same positions in two K-dimensional floating-point vectors to be multiplied, the K pairs of floating-point numbers being divided into K1 first floating-point groups, the first floating-point group i comprising Pi pairs of floating-point numbers, 1≤i≤K1; the point multiplication module performing point multiplication on the Pi pairs of floating-point numbers in the first floating-point group i to obtain K1 first floating-point numbers; and the accumulation module accumulating the K1 first floating-point numbers to obtain a second floating-point number.

[0022] In a possible implementation, each first floating-point group comprises K0 pairs of floating-point numbers, and K = K0*K1.

[0023] In a possible implementation, K0≥K1.

[0024] In one possible implementation, the dot product module performs dot product on Pi floating-point numbers within the first floating-point array i to obtain K1 first floating-point numbers. This process includes: multiplying Pi floating-point numbers within the first floating-point array i to obtain Pi floating-point products; aligning, shifting, and rounding the Pi floating-point products to obtain Pi first significant bits with a decimal width equal to the first bit width; and accumulating the Pi first significant bits to obtain K1 first floating-point numbers. The accumulation module then accumulates the K1 first floating-point numbers to obtain a second floating-point number. This process includes: aligning, shifting, and rounding the K1 first floating-point numbers to obtain K1 second significant bits with a decimal width equal to the second bit width; and accumulating the K1 second significant bits to obtain a second floating-point number. Here, the first bit width is less than or equal to the second bit width.

[0025] In one possible implementation, the accumulation module includes L levels of accumulation units. The input floating-point number of the j-th level accumulation unit is divided into Nj second floating-point arrays, where 1 ≤ j < L. The process of the accumulation module accumulating K1 first floating-point numbers to obtain the second floating-point number includes: the j-th level accumulation unit accumulating N... j All floating-point numbers in the second floating-point array are summed into a third floating-point number, and N is output to the (j+1)th level accumulation unit. j The Lth stage accumulator unit accumulates all the input third floating-point numbers into a single floating-point number, thus obtaining the second floating-point number.

[0026] In one possible implementation, the dot product module performs dot product on Pi floating-point numbers within the first floating-point array i to obtain K1 first floating-point numbers. This process includes: multiplying Pi floating-point numbers within the first floating-point array i to obtain Pi floating-point products; performing alignment, shifting, and rounding on the Pi floating-point products to obtain Pi third significant bits with a third-wide decimal place; accumulating the Pi third significant bits to obtain K1 first floating-point numbers; and the j-th stage accumulation unit then multiplies N... j All floating-point numbers in the second floating-point array are summed into a third floating-point number, and N is output to the (j+1)th level accumulation unit. j The process of generating a third floating-point number includes: converting N... j All floating-point numbers within each of the second floating-point arrays are aligned, shifted, and rounded to obtain a decimal place width of B. j Multiple fourth significant bits; summing multiple fourth significant bits to obtain N. j The process of the Lth stage accumulation unit summing all the input third floating-point numbers into a single floating-point number to obtain the second floating-point number includes: aligning, shifting, and rounding all the input third floating-point numbers to obtain a fractional-bit width of B. LThe multiple fourth significant bits are summed to obtain the second floating-point number; where the third bit width is less than or equal to B. j B j Less than or equal to B j+1 .

[0027] In one possible implementation, the method further includes: the accumulation module adds the second floating-point number to the source floating-point scalar to obtain the target floating-point scalar.

[0028] Thirdly, this application provides a data processing apparatus, the apparatus comprising: one or more processors; a memory for storing one or more computer programs or instructions; and, when the one or more computer programs or instructions are executed by the one or more processors, causing the one or more processors to implement the method as described in any of the second aspects.

[0029] Fourthly, this application provides a data processing apparatus, including a processor for performing the method as described in any one of the second aspects.

[0030] Fifthly, this application provides a data processing apparatus, which includes: a processing circuit and an interface circuit; wherein the interface circuit is used to couple with a memory external to the data processing apparatus and to provide a communication interface for the processing circuit to access the memory; the processing circuit is used to execute program instructions in the memory to implement the method as described in any of the second aspects.

[0031] In specific implementation, the data processing device can be a chip, the input circuit can be an input pin, the output circuit can be an output pin, and the processing circuit can be a transistor, gate circuit, flip-flop, and various logic circuits. The input signal received by the input circuit can be received and input by, for example, but not limited to, a receiver, and the signal output by the output circuit can be output to, for example, but not limited to, a transmitter and transmitted by the transmitter. Furthermore, the input circuit and the output circuit can be the same circuit, which is used as the input circuit and the output circuit at different times. This application does not limit the specific implementation of the processor and various circuits.

[0032] In a sixth aspect, this application provides a computer-readable storage medium storing program code, which, when executed by a processor, implements the method as described in any one of the second aspects.

[0033] In a seventh aspect, this application provides a chip comprising: at least one processor. The at least one processor is configured to perform the method as described in any one of the second aspects.

[0034] Optionally, the chip also includes memory. At least one processor is used to execute code in the memory, and when the at least one processor executes the code, it causes the chip to implement the method as described in any one of the second aspects.

[0035] Alternatively, the chip described above can also be an integrated circuit.

[0036] Eighthly, this application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the method as described in any one of the second aspects. Attached Figure Description

[0037] Figure 1 is a diagram of the IEEE 754 floating-point data format provided in an embodiment of this application;

[0038] Figure 2 is a flowchart illustrating a data processing method provided in an embodiment of this application;

[0039] Figure 3 is a flowchart of a data processing method provided in an embodiment of this application;

[0040] Figure 4 is a block diagram of a data processing device provided in an embodiment of this application;

[0041] Figure 5 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0042] Figure 6 is a schematic diagram of the structure of a data processing device provided in an embodiment of this application. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0044] The terms "first," "second," etc., used in the specification, embodiments, claims, and drawings of this application are for distinguishing purposes only and should not be construed as indicating or implying relative importance or order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as including a series of steps or units. A method, system, product, or apparatus is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0045] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0046] This application provides a data processing method for performing dot product of two floating-point vectors. For example, it can be used to perform dot product of two floating-point vectors in training or inference scenarios of artificial intelligence models. This method can be used for general matrix multiplication (GEMM) calculations. GEMM calculations can employ a block fused multiply add (Block FMA) architecture. The block fused multiply add architecture has high energy efficiency and high computing power, and can be applied to artificial intelligence (AI) chips and high-performance computing (HPC) chips. For example, it can be applied to graphics processing units (GPUs), neural network processing units (NPUs), or tensor processing units (TPUs).

[0047] The core process of the block-fused multiply-add architecture is computation, specifically including: multiplying two floating-point vectors by a dot product, and then adding the result of the dot product to a scalar (which can be represented as C) to obtain the updated scalar (which can be represented as the updated C). Thus, the block-fused multiply-add architecture can complete the calculation of one row and one column of elements in a matrix multiplication. Through multiple parallel block-fused multiply-add architectures or multiple computations of the block-fused multiply-add architecture, it can complete part or all of the computation process in matrix multiplication.

[0048] General matrix multiplication includes, but is not limited to: multiplying the error matrix and the activation matrix to obtain the weight gradient, or multiplying the error matrix and the weight matrix to obtain a new error matrix, etc.

[0049] The floating-point number formats supported by the embodiments of this application include, but are not limited to: double precision floating point (FP64), single precision floating point (FP32), half precision floating point (FP16), 8-bit floating point (FP8), BF16, HiFloat8, etc.

[0050] The Institute of Electrical and Electronics Engineers (IEEE) has developed the IEEE 754 standard for binary floating-point arithmetic, defining floating-point data representation methods such as FP64, FP32, and FP16. Please refer to Figure 1, which illustrates the IEEE 754 floating-point data format provided in this embodiment. IEEE 754 floating-point data includes a sign field (bit sign, S), an exponent field (bits exponent, E), and a mantissa field (bits mantissa, M). Specifically, for FP64 data, the sign field is 1 bit, the exponent field is 11 bits, and the mantissa field is 52 bits; for FP32 data, the sign field is 1 bit, the exponent field is 8 bits, and the mantissa field is 23 bits; for FP16 data, the sign field is 1 bit, the exponent field is 5 bits, and the mantissa field is 10 bits. The conversion formula between IEEE 754 floating-point data and its decimal value is: Value = (-1) sign *2exponent-bias*(1+mantissa), where bias is a different constant for floating-point data such as FP64, FP32 and FP16.

[0051] For example, please refer to Table 1, which shows the data format for FP16.

[0052] Table 1. Data Format Table for FP16

[0053] FP16 includes a sign field S, an exponent field E, and a mantissa field M. The sign field of FP16 determines whether the FP16 data is positive or negative, where 0 represents a positive number and 1 represents a negative number. The exponent field of FP16 is a power of 2, which can be used to weight FP16. The mantissa field of FP16 is a binary fraction. The conversion steps of FP16 to decimal data are as follows: (1) If the sign field of FP16 is 0, then the FP16 is a positive number. (2) The encoding value of the exponent field of FP16 is 10001, which represents 17 in decimal. After removing the bias in FP16 (for FP16 data, bias = 15), the exponent field of FP16 is 2 to the power of 2. (3) The mantissa field of FP16 is 0000000010. Since the exponent field is not all "0" or all "1", the decimal data represented by the mantissa field is 1.001953125. Based on the conversion formula between floating-point data and decimal values, the decimal value corresponding to this FP16 data is 4.0078125.

[0054] For example, please refer to Table 2, which shows the data format for HiFloat8.

[0055] Table 2. Data Format Table for HiFloat8

[0056] In this context, 8 represents the total bit width of the HiFloat8 data, which can vary. The sign field occupies one bit, with 0 representing a positive number and 1 representing a negative number, or vice versa. The dot field occupies 2 or 4 bits and can represent 5 different pieces of information; the value of D can be 0, 1, 2, 3, or 4. The DML symbol indicates that the HiFloat8 data lacks an exponent field and requires unconventional parsing. Otherwise, conventional parsing should be used. The dot field uses an unconventional prefix encoding: 4 bits are used to encode the small value 0 and the DML symbol, 3 bits are used to encode the medium value 1, and 2 bits are used to encode the large values ​​2, 3, and 4. The bit width of the exponent field varies depending on the value of D, and the mantissa field occupies the remaining bit width.

[0057] The standard analytical expression is: X = (-1) S ×2 E ×1.M, the unconventional analytical expression is: X=(-1) S ×2 M-23 ×1.0.

[0058] When D is 0, the value of the exponent field is 0. When D is not 0, the exponent field uses signed magnitude encoding, that is, the sign bit follows the true form (TF). The encoding of the exponent field is Ei = {Se, 1'b1, TF[2:end]}, where Se is the sign bit of the exponent. The highest bit 1'b1 of TF is hidden and not stored, so the encoding value of the exponent field is Es = {Se, TF[2:end]}. The decimal encoding value of the exponent field is Ev = Ei + Ec.

[0059] HiFloat(N,5,Ec) can be configured as HiFloat(8,5,0), abbreviated as HiF8, or other values.

[0060] Please refer to Figure 2, which is a flowchart illustrating a data processing method provided in an embodiment of this application. This method can be applied to a processor, which can be a circuit with a block fusion multiply-accumulate architecture. For example, the processor may include a matrix multiplication acceleration core in a chip for performing general matrix multiplication calculations. For instance, it can be a tensor core in a GPU chip or a cube core in an NPU chip. The processor includes an acquisition module, a dot product module, and an accumulation module. The method may include the following processes:

[0061] 101. The acquisition module obtains K pairs of floating-point numbers. Each pair of floating-point numbers is obtained by combining the components at the same position in the two K-dimensional floating-point vectors to be multiplied. The K pairs of floating-point numbers are divided into K1 first floating-point arrays. The first floating-point array i includes Pi pairs of floating-point numbers, 1≤i≤K1.

[0062] Where K1≥2, K can be an integer power of 2, such as 8, 16, 32 or 64. Pi and K1 are both integers.

[0063] Two K-dimensional floating-point vectors represent a row and a column element of a matrix multiplication operation, respectively. A K-dimensional floating-point vector contains K floating-point elements. These two K-dimensional floating-point vectors can also be referred to as the input or source operands for the multiply-accumulate operation, or the input or source operands for the matrix multiplication operation. The K pairs of floating-point numbers can be FP64, FP32, FP16, FP8, BF16, or HiFloat8, etc. This application does not limit the format of the K pairs of floating-point numbers.

[0064] For example, two K-dimensional floating-point vectors can be represented as A. K and B K A K ={a1, a2, ..., a K}, B K ={b1, b2, ..., b K}. a1 and b1 form a pair of floating-point numbers, a2 and b2 form a pair of floating-point numbers, ..., a K and b K Combined into a pair of floating-point numbers.

[0065] In one possible implementation, the K pairs of floating-point numbers can be unequally divided into K1 first floating-point arrays, and there are two first floating-point arrays that contain different numbers of floating-point pairs (that is, there are two first floating-point arrays with different Pi).

[0066] In one possible implementation, K pairs of floating-point numbers can be equally divided into K1 first floating-point arrays, where each of the K1 first floating-point arrays contains the same number of floating-point pairs (i.e., any two first floating-point arrays have the same Pi). For example, each first floating-point array can include K0 pairs of floating-point numbers, i.e., K = K0 * K1, where K0 is an integer.

[0067] In one possible implementation, K0 ≥ K1. For example, K is 64, and K0 and K1 can both be 8; or K0 can be 16, and K1 can be 4; or K0 can be 32, and K1 can be 2. This allows the number of floating-point numbers accumulated by each sub-dot multiplication module in the first-level dot multiplication operation to be greater than or equal to the number of floating-point numbers accumulated in the second-level accumulation operation, thereby further reducing the processor's area and power consumption.

[0068] 102. The dot product module performs dot product on the floating-point numbers Pi in the first floating-point array i to obtain K1 first floating-point numbers.

[0069] The dot product module performs dot product on the floating-point numbers Pi in the first floating-point array i for i = 1, 2, ..., K1. Through this process, all dot product operations on the floating-point numbers in each first floating-point array result in a first floating-point number, thus yielding K1 floating-point numbers. In one possible implementation, the dot product operations on the floating-point numbers in the K1 first floating-point arrays can be performed in parallel. For example, the dot product module includes K1 sub-dot product modules, which perform all dot product operations on the floating-point numbers in the K1 first floating-point arrays in parallel.

[0070] For example, the dot product module can multiply Pi floating-point numbers in the first floating-point array i by Pi floating-point numbers to obtain Pi floating-point products. Then, each of the Pi floating-point products is aligned, shifted, and rounded to obtain Pi first significant bits with a decimal place width equal to the width of the first bit. Finally, the Pi first significant bits are summed to obtain the first floating-point number corresponding to the first floating-point array i, thus obtaining K1 first floating-point numbers.

[0071] Here, "aligning exponents" refers to determining the exponent difference among the Pi floating-point products and unifying them to the largest exponent of the Pi floating-point products. The exponent difference of the floating-point products is the difference between the exponent of the floating-point product and the largest exponent. "Shifting" refers to right-shifting the mantissa of floating-point products whose exponents are not equal to the largest exponent by the corresponding number of bits according to the exponent difference, so as to ensure that the size of the floating-point product remains unchanged. The accumulation process of the Pi first significant bits can be a compression process of fixed-point addition, during which high-order bits will carry over. Finally, the result of the accumulation of the Pi first significant bits is combined with the largest exponent to form the first floating-point number.

[0072] Please refer to Figure 3, which is a flowchart of a data processing method provided in an embodiment of this application. Figure 3 shows K1 sub-dot multiplication modules, which calculate the dot product of two floating-point vectors of length K0 in parallel. The specific process of K0 dot product of floating-point numbers in the first floating-point array i includes: sub-dot multiplication module i completes the multiplication of K0 floating-point numbers, obtaining K0 floating-point products. Then, the maximum exponent Emax_i among the exponents of the K0 floating-point products is found, and the difference between the exponent of the K0 floating-point products and Emax_i is calculated to unify the K0 floating-point products to Emax_i. The mantissa of the floating-point product (i.e., the result of multiplying the mantissas of the pair of floating-point numbers corresponding to the floating-point product) is shifted according to the exponent difference of each floating-point product. The shifted data is rounded and retained in S1PN format (i.e., the aforementioned first valid bit), where S1PN is in two's complement form. Then, the K0 S1PN format data are processed through a fixed-point addition tree to obtain the STPN format addition result, where STPN is in two's complement form. Finally, the STPN format addition result is combined with the maximum exponent Emax_i to form the first floating-point number 2 corresponding to the first floating-point array i. Emax_i *STPN.

[0073] In this embodiment, SXPY is a bit-width expression, where S represents the sign bit, X represents the integer bit width, P represents the decimal point, and Y represents the decimal bit width. For S1PN (X = 1, Y = N), "1" indicates that the integer bit width is 1, and "N" indicates that the decimal bit width is N. For STPN (X = T, Y = N), "T" indicates that the integer bit width is 1 (including carry generated by addition), and "N" indicates that the decimal bit width is N.

[0074] Where N is the first width mentioned above. If K0 is not an integer power of 2, the upper limit can be taken. In this embodiment, each pair of floating-point numbers can be multiplied in sign-magnitude or two's complement form, and the K0 floating-point products can be added in two's complement form. This embodiment does not limit this.

[0075] For example, suppose K = 32, K1 = 2, and each first floating-point array includes 16 pairs of floating-point numbers (K0 = 16). The dot product of the two 32-dimensional floating-point vectors is divided into two stages. For each first floating-point array, the dot product module performs dot product on the 16 pairs of floating-point numbers in the first floating-point array, resulting in 16 floating-point products. Then, each of the 16 floating-point products is aligned, shifted, and rounded to obtain 16 first significant bits in S5PN format. These 16 first significant bits are then accumulated to obtain one first floating-point number. Through this process, a total of two first floating-point numbers are obtained.

[0076] 103. The accumulation module accumulates K1 first floating-point numbers to obtain the second floating-point number.

[0077] In one possible implementation, the accumulation module can directly accumulate K1 first floating-point numbers into a single floating-point number to obtain the second floating-point number. For example, each of the K1 first floating-point numbers can be aligned, shifted, and rounded to obtain K1 second significant bits with a decimal width equal to the second bit width. Finally, the second floating-point number is obtained by accumulating the K1 second significant bits.

[0078] Here, "aligning exponents" refers to determining the exponent difference among the K1 first floating-point numbers and unifying them to the largest exponent. The exponent difference of the first floating-point numbers is the difference between the exponent of the first floating-point number and the largest exponent. "Shifting" refers to right-shifting the mantissa of a first floating-point number whose exponent is not equal to the largest exponent by the corresponding number of bits according to the exponent difference, ensuring that the size of the first floating-point number remains unchanged. The accumulation process of the K1 second significant bits can be a compression process of fixed-point addition, during which high-order bits will carry over. Finally, the result of the accumulation of the K1 second significant bits is combined with the largest exponent to form the second floating-point number.

[0079] In this implementation, the first width (e.g., N in Figure 3) can be less than or equal to the second width.

[0080] For example, please continue to refer to Figure 3, the accumulation module searches for K1 first floating-point numbers 2. Emax_1 *STPN、2 Emax_2 *STPN、…、2 Emax_K1The maximum exponent Emax in the *STPN exponent is used to calculate the difference between the exponents of K1 first floating-point numbers and Emax to unify the K1 first floating-point numbers to Emax. Then, the mantissa of each first floating-point number is shifted according to the exponent difference. The shifted data is rounded and retained in STPN* format (i.e., the aforementioned second significant bit), where STPN* is in two's complement form. In STPN*, "T" indicates the integer bit width is T, and "N*" indicates the fractional bit width is N*. Next, the K1 STPN* format data are processed through a fixed-point addition tree to obtain the addition result in ST*PN* format, where ST*PN* is in two's complement form. In ST*PN*, "T*" indicates the integer bit width is T* (including carry generated by the addend), and "N*" indicates the fractional bit width is N*. Finally, the ST*PN* format addition result is combined with the maximum exponent Emax to form the second floating-point number 2. Emax *ST*PN*.

[0081] Where N* is the second bit width mentioned above, and N can be less than or equal to N*. If K is not an integer power of 2, the upper limit can be taken.

[0082] For example, suppose that in the aforementioned process 102, for each first floating-point array, 16 first significant bits in S5PN format are obtained, and one first floating-point number is obtained by accumulating the 16 first significant bits. Then, in this process 103, the two first floating-point numbers are aligned, shifted, and rounded to obtain two second significant bits in S6PNF format, and the second floating-point number is obtained by accumulating the two second significant bits. Here, N can be less than or equal to NF.

[0083] In this embodiment, calculating the dot product of all pairs of floating-point numbers within K1 first floating-point arrays is called the first-level dot product operation, and accumulating the K1 first floating-point numbers to obtain the second floating-point number is called the second-level accumulation operation. The first-level dot product operation involves parallel accumulation and compression of a relatively large number of floating-point numbers, and its overhead is usually large. The second-level accumulation operation, on the other hand, accumulates a small number of floating-point numbers, with lower parallelism and usually lower overhead. Therefore, setting the first bit width smaller than the second bit width can prevent the overhead of the first-level dot product operation from becoming too large, while also ensuring the accuracy of the second-level accumulation operation. This effectively reduces the overhead of the first-level dot product operation while maintaining the accuracy requirements of matrix multiplication, thereby reducing the processor area and power consumption, and effectively improving the energy efficiency of matrix multiplication operations. Furthermore, setting the first bit width smaller also allows the addition process of sub-dot product module i on Pi floating-point products in the first-level dot product operation to be faster and with lower latency.

[0084] In one possible implementation, K1 first floating-point numbers can be accumulated in stages to obtain a second floating-point number. For example, the accumulation module may include L levels of accumulation units. The input floating-point number of the j-th level accumulation unit is divided into Nj second floating-point arrays, where 1 ≤ j < L. The j-th level accumulation unit accumulates all the floating-point numbers in each of the Nj second floating-point arrays into a third floating-point number and outputs Nj third floating-point numbers to the (j+1)-th level accumulation unit. The L-th level accumulation unit accumulates all the input third floating-point numbers into a single floating-point number to obtain the second floating-point number.

[0085] Each accumulator unit other than level L executes the process performed by level j. The input floating-point numbers of level 1 accumulator units are K1 first floating-point numbers. The input floating-point numbers of level 2 to level L-1 accumulator units are the output floating-point numbers of the previous level accumulator unit.

[0086] The input floating-point number of the j-th level accumulation unit can be divided into Nj second floating-point arrays in an uneven manner or equally. This application embodiment does not limit the division method.

[0087] For example, the j-th level accumulation unit can accumulate N j All floating-point numbers within each of the second floating-point arrays are aligned, shifted, and rounded to obtain a decimal place width of B. j The multiple third significant bits are then summed to obtain N. j A third floating-point number.

[0088] In this process, "aligning exponents" refers to determining the exponent difference of each floating-point number in the second floating-point array, unifying each floating-point number in the second array to the largest exponent of all floating-point numbers in the array. The exponent difference of a floating-point number is the difference between its exponent and the largest exponent of all floating-point numbers in the second floating-point array. "Shifting" refers to right-shifting the mantissa of a floating-point number in the second floating-point array, for which the exponent is not equal to the largest exponent, according to the exponent difference, to ensure that the size of the floating-point number remains unchanged. The accumulation process of multiple third significant bits can be a compression process of fixed-point addition, during which high-order bits will carry over. Finally, the accumulation result of multiple third significant bits is combined with the largest exponent to form a third floating-point number.

[0089] For example, the Lth-level accumulation unit can perform alignment, shifting, and rounding on each input floating-point number to obtain a fractional part width of B. L The multiple fourth significant bits are then summed to obtain the second floating-point number.

[0090] Here, "aligning exponents" refers to determining the exponent difference for each input floating-point number, unifying each input floating-point number to the maximum exponent of all input floating-point numbers. The exponent difference of the input floating-point numbers is the difference between the exponent of the input floating-point number and the maximum exponent of all input floating-point numbers. "Shifting" refers to right-shifting the mantissa of an input floating-point number whose exponent is not equal to the maximum exponent by the corresponding number of bits according to the exponent difference, so as to ensure that the size of the floating-point number remains unchanged. The accumulation process of multiple fourth significant bits can be a compression process of fixed-point addition, during which high-order bits will carry over. Finally, the accumulation result of multiple fourth significant bits is combined with the maximum exponent to form the second floating-point number.

[0091] In this implementation, the first width can be less than or equal to B. j B j Less than or equal to B j+1 When j = L-1, B j+1 That is B L .

[0092] In this embodiment, the smaller the level of the accumulation unit, the more floating-point numbers it needs to accumulate, resulting in greater accumulation overhead. Therefore, the smaller the level of the accumulation unit, the smaller the decimal part width of the output floating-point numbers involved in the accumulation process. This can reduce the processor area and power consumption while ensuring the precision requirements of matrix multiplication, effectively improving the energy efficiency of matrix multiplication operations. Furthermore, a smaller decimal part width of the output floating-point numbers involved in the accumulation process of the accumulation unit can also make the accumulation operation of the accumulation unit faster and with lower latency.

[0093] The tiered accumulation method can be applied when K is relatively large, such as K = 256, 260, 278, 300, etc. Taking K = 256 as an example, the 256 pairs of floating-point numbers can be divided into 32 first floating-point arrays (K1 = 32), and each first floating-point array can include 8 pairs of floating-point numbers (K0 = 8). Following procedures 101 and 102, 32 first floating-point numbers are obtained. Then, the accumulation of these 32 first floating-point numbers is divided into two levels. The input floating-point numbers (i.e., the 32 first floating-point numbers) of the first-level accumulation unit are divided into 8 second floating-point arrays. The first-level accumulation unit accumulates all the floating-point numbers in each of the 8 second floating-point arrays into a third floating-point number and outputs the 8 third floating-point numbers to the second-level accumulation unit. The second-level accumulation unit accumulates the 8 input third floating-point numbers into a single floating-point number, obtaining the second floating-point number.

[0094] The specific operation process of the first-level accumulation unit can refer to the operation process of the j-th level accumulation unit described above, and the specific operation process of the second-level accumulation unit can refer to the operation process of the L-th level accumulation unit described above. These details are not repeated here. The 32 first floating-point numbers can be unevenly divided into 8 second floating-point arrays, or they can be evenly divided into 8 second floating-point arrays (i.e., each second floating-point array includes 4 floating-point numbers). This application does not limit the division method.

[0095] In this embodiment, after obtaining the second floating-point number, the accumulation module can add the second floating-point number to the source floating-point scalar to obtain the target floating-point scalar, thereby updating the source floating-point scalar. The format of the source or target floating-point scalar can be FP64, FP32, FP16, FP8, BF16, or HiFloat8, etc. For example, assuming that two second significant bits in S6PNF format are obtained in this process, and the second floating-point number is obtained by accumulating the two second significant bits. Finally, the second floating-point number can be added to the source floating-point scalar in FP32 format to obtain the third significant bit in S7PNF format, and the target floating-point scalar is obtained based on the third significant bit.

[0096] Multiple iterations following the aforementioned process can achieve multiple updates to the floating-point scalar. After obtaining the target floating-point scalar in the last iteration, the format of the target floating-point scalar can be converted, or the format of the target floating-point scalar can be kept unchanged; this embodiment of the application does not limit this.

[0097] In summary, the data processing method provided in this application involves the following steps: First, the acquisition module acquires K pairs of floating-point numbers. Each pair of floating-point numbers is obtained by combining the components at the same position in two K-dimensional floating-point vectors to be multiplied. The K pairs of floating-point numbers are divided into K1 first floating-point arrays. Each first floating-point array i contains Pi pairs of floating-point numbers, where 1 ≤ i ≤ K1. Then, the multiplication module performs multiplication on the Pi pairs of floating-point numbers in the first floating-point array i to obtain K1 first floating-point numbers. Finally, the accumulation module accumulates the K1 first floating-point numbers to obtain the second floating-point number. In this embodiment, the dot product of K pairs of floating-point numbers is divided into two levels. The first level of dot product operation completes the dot product of all pairs of floating-point numbers in each first floating-point array, and the second level of accumulation operation completes the accumulation of K1 floating-point numbers. In the first level of dot product operation, the maximum exponent is searched for among the exponents of Pi floating-point products. The search for the maximum exponent is faster, the addition of floating-point products is also faster, and the latency is smaller, thereby reducing the area and power consumption of the processor and effectively improving the energy efficiency of matrix multiplication operations.

[0098] The order of the methods provided in the embodiments of this application can be adjusted appropriately, and the process can also be added or removed as appropriate. Any variations that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application, and the embodiments of this application do not limit this.

[0099] The foregoing primarily describes the data processing method provided in the embodiments of this application from the perspective of the device. It is understood that, in order to achieve the above functions, the device includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0100] This application embodiment can divide the device into functional modules according to the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one device. The integrated modules can be implemented in hardware or software functional modules. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0101] Figure 4 is a block diagram of a data processing device provided in an embodiment of this application. When each functional module is divided according to its corresponding function, the data processing device 200 may include an acquisition module 201, a dot multiplication module 202, and an accumulation module 203. Exemplarily, the data processing device may be a device, or a chip within the device, or other combined devices or components having the aforementioned data processing device functions. For example, the data processing device 200 may be a processor (or processing circuit), such as a baseband processor, which may include one or more central processing units (CPUs). The data processing device 200 may also be a processor (or processing circuit) of a chip system, which may include one or more central processing units.

[0102] For example, the acquisition module 201 is used to acquire K pairs of floating-point numbers, which are obtained by combining the components at the same position in two K-dimensional floating-point vectors to be multiplied. The K pairs of floating-point numbers are divided into K1 first floating-point arrays, where the first floating-point array i includes Pi pairs of floating-point numbers, 1≤i≤K1; the multiplication module 202 is used to multiply the Pi pairs of floating-point numbers in the first floating-point array i to obtain K1 first floating-point numbers; the accumulation module 203 is used to accumulate the K1 first floating-point numbers to obtain second floating-point numbers.

[0103] Combining the above scheme, each first floating-point array includes K0 pairs of floating-point numbers, where K = K0 * K1.

[0104] Combining the above schemes, K0≥K1.

[0105] Combining the above scheme, the dot product module 202 is specifically used to: multiply the Pi floating-point numbers in the first floating-point array i by Pi floating-point numbers to obtain Pi floating-point products; perform alignment, shifting, and rounding on the Pi floating-point products to obtain Pi first significant bits with a decimal width equal to the first significant bit width; and accumulate the Pi first significant bits to obtain K1 first floating-point numbers. The accumulation module 203 is specifically used to: perform alignment, shifting, and rounding on the K1 first floating-point numbers to obtain K1 second significant bits with a decimal width equal to the second significant bit width; and accumulate the K1 second significant bits to obtain the second floating-point number. The first significant bit width is less than or equal to the second significant bit width.

[0106] Based on the above scheme, the accumulation module 203 may include L levels of accumulation units. The input floating-point number of the j-th level accumulation unit is divided into Nj second floating-point arrays, 1 ≤ j < L; the j-th level accumulation unit is used to respectively divide N... j All floating-point numbers in the second floating-point array are summed into a third floating-point number, and N is output to the (j+1)th level accumulation unit. j The Lth level accumulator unit is used to accumulate all the input third floating-point numbers into a single floating-point number to obtain the second floating-point number.

[0107] Combining the above scheme, the dot product module 202 is specifically used for: multiplying Pi floating-point numbers in the first floating-point array i to obtain Pi floating-point products; performing alignment, shifting, and rounding on the Pi floating-point products to obtain Pi third significant bits with a decimal width of the third bit width; and accumulating the Pi third significant bits to obtain K1 first floating-point numbers. The j-th stage accumulation unit is specifically used for: multiplying N... j All floating-point numbers within each of the second floating-point arrays are aligned, shifted, and rounded to obtain a decimal place width of B. j Multiple fourth significant bits; summing multiple fourth significant bits to obtain N. jThe Lth level accumulation unit is specifically used to: realign, shift, and round all input third-level floating-point numbers to obtain a fractional part with a width of B. L The multiple fourth significant bits are summed to obtain the second floating-point number. The third bit width is less than or equal to B. j B j Less than or equal to B j+1 .

[0108] In conjunction with the above scheme, the accumulation module 203 is also used to add the second floating-point number to the source floating-point scalar to obtain the target floating-point scalar.

[0109] Figure 5 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 300 can be a terminal device or a chip or functional module in a terminal device. As shown in Figure 5, the electronic device 300 includes a processor 301, a transceiver 302, and a communication line 303.

[0110] The processor 301 is used to execute any step in the aforementioned method embodiments, and when performing processes such as obtaining K pairs of floating-point numbers, it can selectively call the transceiver 302 and the communication line 303 to complete the corresponding operations.

[0111] Furthermore, the electronic device 300 may also include a memory 304. The processor 301, memory 304, and transceiver 302 can be connected via a communication line 303.

[0112] Transceiver 302 is used to communicate with other devices or other communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. Transceiver 302 can be a module, circuit, transceiver, or any device capable of enabling communication.

[0113] The transceiver 302 is mainly used for sending and receiving data, etc., and may include a transmitter and a receiver to send and receive data, etc. respectively. Operations other than sending and receiving data are implemented by the processor, such as multiplying Pi in the first floating-point array i by floating-point numbers to obtain K first floating-point numbers, etc.

[0114] Communication line 303 is used to transmit information between the various components included in electronic device 300.

[0115] In one design, the processor can be viewed as a logic circuit, and the transceiver as an interface circuit.

[0116] Memory 304 is used to store instructions. These instructions can be computer programs.

[0117] It should be noted that the memory 304 can exist independently of the processor 301, or it can be integrated with the processor 301. The memory 304 can be used to store instructions, program code, or some data, etc. The memory 304 can be located inside or outside the electronic device 300, without limitation. The processor 301 is used to execute the instructions stored in the memory 304 to implement the method provided in the above embodiments of this application.

[0118] In one example, processor 301 may include one or more processors, such as processor 0 and processor 1 in Figure 5.

[0119] As an alternative implementation, the electronic device 300 may include multiple processors, for example, in addition to processor 301 in FIG. 5, it may also include processor 307.

[0120] As an optional implementation, the electronic device 300 also includes an output device 305 and an input device 306. For example, the input device 306 is a device such as a keyboard, mouse, microphone, or joystick, and the output device 305 is a device such as a display screen or speaker.

[0121] It should be noted that the electronic device 300 can be a chip system or a device with a structure similar to that shown in Figure 5. The chip system can be composed of chips or may include chips and other discrete components. Actions, terms, etc., involved in the various embodiments of this application can be referred to mutually without limitation. The message names or parameter names in the messages used for interaction between devices in the embodiments of this application are merely examples; other names may be used in specific implementations without limitation. Furthermore, the composition structure shown in Figure 5 does not constitute a limitation on the electronic device 300. In addition to the components shown in Figure 5, the electronic device 300 may include more or fewer components than shown in Figure 5, or combine certain components, or have different component arrangements.

[0122] The processor and transceiver described in this application can be implemented on integrated circuits (ICs), analog ICs, radio frequency integrated circuits, mixed-signal ICs, application-specific integrated circuits (ASICs), printed circuit boards (PCBs), electronic devices, etc. The processor and transceiver can also be manufactured using various IC process technologies, such as complementary metal oxide semiconductors (CMOS), n-metal-oxide-semiconductor (NMOS), p-type metal oxide semiconductors (PMOS), bipolar junction transistors (BJTs), bipolar CMOS (BiCMOS), silicon germanium (SiGe), gallium arsenide (GaAs), etc.

[0123] Figure 6 is a schematic diagram of a data processing device provided in an embodiment of this application. This data processing device is applicable to the scenarios shown in the above-described method embodiments. For ease of explanation, Figure 6 only shows the main components of the data processing device, including a processor, memory, control circuit, and input / output devices. The processor is mainly used to process communication protocols and communication data, execute software programs, and process the data of the software programs. The memory is mainly used to store software programs and data. The control circuit is mainly used for power supply and the transmission of various electrical signals. The input / output devices are mainly used to receive user-input data and output data to the user.

[0124] When the data processing device is a machine, the control circuit can be a motherboard, the memory includes storage media such as hard disks, RAM, and ROM, the processor can include a baseband processor and a central processing unit (CPU). The baseband processor is mainly used to process communication protocols and communication data, while the CPU is mainly used to control the entire data processing device, execute software programs, and process data frames of the software programs. Input / output devices include displays, keyboards, and mice. The control circuit can further include or be connected to transceiver circuits or transceivers, such as network cable interfaces, for sending or receiving data or signals, such as for data transmission and communication with other devices. Furthermore, it can also include an antenna for sending and receiving messages and for data transmission with other devices.

[0125] According to the method provided in the embodiments of this application, this application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to execute any of the methods described in the embodiments of this application.

[0126] This application also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be executed by a computer or a device with data processing capabilities, using computer programs or instructions to control related hardware. The computer program or set of instructions can be stored in the aforementioned computer-readable storage medium. When executed, the computer program or set of instructions can include the processes described in the above method embodiments. The computer-readable storage medium can be an internal storage unit of the device in any of the foregoing embodiments, such as the device's hard disk or memory. The aforementioned computer-readable storage medium can also be an external storage device of the device, such as the device's plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Further, the aforementioned computer-readable storage medium can include both internal storage units and external storage devices of the device. The aforementioned computer-readable storage medium is used to store the aforementioned computer program or instructions, as well as other programs and data required by the device. The aforementioned computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0127] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0128] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0129] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0130] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0131] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0132] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0133] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A processor, characterized in that, The processor is used to perform training or inference of an artificial intelligence model, and the processor includes an acquisition module, a dot product module, and an accumulation module. The acquisition module is used to acquire K pairs of floating-point numbers, which are obtained by combining the components at the same position in two K-dimensional floating-point vectors to be multiplied. The K pairs of floating-point numbers are divided into K1 first floating-point arrays, and the first floating-point array i includes Pi pairs of floating-point numbers, 1≤i≤K1. The dot product module is used to perform dot product of Pi and floating-point numbers in the first floating-point array i to obtain K1 first floating-point numbers. The accumulation module is used to accumulate the K1 first floating-point numbers to obtain a second floating-point number.

2. The processor according to claim 1, characterized in that, Each of the first floating-point arrays includes K0 pairs of floating-point numbers, where K = K0 * K1.

3. The processor according to claim 2, characterized in that, K0≥K1.

4. The processor according to any one of claims 1 to 3, characterized in that, The dot product module is specifically used to: multiply Pi floating-point numbers in the first floating-point array i to obtain Pi floating-point products; perform alignment, shifting, and rounding on the Pi floating-point products to obtain Pi first significant bits with a decimal place width equal to the width of the first bit; and accumulate the Pi first significant bits to obtain the K1 first floating-point numbers. The accumulation module is specifically used to: perform alignment, shifting, and rounding on the K1 first floating-point numbers to obtain K1 second significant bits with a decimal width of the second bit width, and accumulate the K1 second significant bits to obtain the second floating-point number; Wherein, the first bit width is less than or equal to the second bit width.

5. The processor according to any one of claims 1 to 3, characterized in that, The accumulation module includes L levels of accumulation units, and the input floating-point numbers of the j-th level accumulation unit are divided into N levels. j A second floating-point array, 1≤j<L; The j-th level accumulation unit is used to respectively add the N j All floating-point numbers in the second floating-point array are summed into a third floating-point number, and N is output to the (j+1)th level accumulation unit. j A third floating-point number; The Lth-level accumulation unit is used to accumulate all the input third floating-point numbers into a single floating-point number to obtain the second floating-point number.

6. The processor according to claim 5, characterized in that, The dot product module is specifically used to: multiply Pi floating-point numbers in the first floating-point array i to obtain Pi floating-point products; perform alignment, shifting, and rounding on the Pi floating-point products to obtain Pi third significant bits with a decimal width of the third bit width; and accumulate the Pi third significant bits to obtain the K1 first floating-point numbers. The j-th level accumulation unit is specifically used for: accumulating the N... j The floating-point numbers in each of the second floating-point arrays are aligned, shifted, and rounded to obtain a decimal place width of B. j The multiple fourth significant bits are accumulated to obtain the N. j A third floating-point number; The Lth-level accumulation unit is specifically used to: perform alignment, shifting, and rounding on all input third-level floating-point numbers to obtain a fractional part width of B. L The second floating-point number is obtained by summing the multiple fourth significant bits. Wherein, the third bit width is less than or equal to B. j The B j Less than or equal to B j+1 .

7. The processor according to any one of claims 1 to 6, characterized in that, The accumulation module is also used to add the second floating-point number to the source floating-point scalar to obtain the target floating-point scalar.

8. A data processing method, characterized in that, Applied to a processor for performing training or inference of an artificial intelligence model, the processor includes an acquisition module, a dot product module, and an accumulation module; the method includes: The acquisition module acquires K pairs of floating-point numbers, which are obtained by combining the components at the same position in two K-dimensional floating-point vectors to be multiplied. The K pairs of floating-point numbers are divided into K1 first floating-point arrays, and the first floating-point array i includes Pi pairs of floating-point numbers, 1≤i≤K1. The dot product module performs a dot product of Pi and floating-point numbers within the first floating-point array i to obtain K1 first floating-point numbers. The accumulation module accumulates the K1 first floating-point numbers to obtain the second floating-point number.

9. The method according to claim 8, characterized in that, Each of the first floating-point arrays includes K0 pairs of floating-point numbers, where K = K0 * K1.

10. The method according to claim 9, characterized in that, K0≥K1.

11. The method according to any one of claims 8 to 10, characterized in that, The dot product module performs dot product on the floating-point numbers Pi within the first floating-point array i to obtain K1 first floating-point numbers, including: Multiply the Pi floating-point numbers in the first floating-point array i by Pi floating-point numbers to obtain Pi floating-point products; The Pi floating-point products are aligned, shifted, and rounded to obtain Pi first significant bits with a decimal width equal to the width of the first bit. The first Pi bits are accumulated to obtain the K1 first floating-point numbers; The accumulation module accumulates the K1 first floating-point numbers to obtain a second floating-point number, including: The K1 first floating-point numbers are aligned, shifted, and rounded to obtain K1 second significant bits with a decimal width of the second bit width. The second floating-point number is obtained by accumulating the K1 second significant bits; Wherein, the first bit width is less than or equal to the second bit width.

12. The method according to any one of claims 8 to 10, characterized in that, The accumulation module includes L levels of accumulation units. The input floating-point number of the j-th level accumulation unit is divided into Nj second floating-point arrays, where 1 ≤ j < L. The accumulation module accumulates the K1 first floating-point numbers to obtain a second floating-point number, including: The j-th level accumulator unit respectively processes the N... j All floating-point numbers in the second floating-point array are summed into a third floating-point number, and N is output to the (j+1)th level accumulation unit. j A third floating-point number; The L-level accumulator unit accumulates all the input third floating-point numbers into a single floating-point number, thus obtaining the second floating-point number.

13. The method according to claim 12, characterized in that, The dot product module performs dot product on the floating-point numbers Pi within the first floating-point array i to obtain K1 first floating-point numbers, including: Multiply the Pi floating-point numbers in the first floating-point array i by Pi floating-point numbers to obtain Pi floating-point products; The Pi floating-point products are aligned, shifted, and rounded to obtain Pi third significant bits with a decimal width of the third bit width. The K1 first floating-point numbers are obtained by accumulating the Pi third significant bits. The j-th level accumulator unit respectively processes the N... j All floating-point numbers in the second floating-point array are summed into a third floating-point number, and N is output to the (j+1)th level accumulation unit. j The third floating-point number includes: The N j The floating-point numbers in each of the second floating-point arrays are aligned, shifted, and rounded to obtain a decimal place width of B. j Multiple fourth valid bits; The N is obtained by accumulating the multiple fourth effective bits. j A third floating-point number; The Lth-level accumulation unit accumulates all the input third floating-point numbers into a single floating-point number to obtain the second floating-point number, including: Perform alignment, shifting, and rounding on all input third-order floating-point numbers to obtain a fractional digit width of B. L Multiple fourth valid bits; The second floating-point number is obtained by accumulating the multiple fourth significant bits; Wherein, the third bit width is less than or equal to B. j The B j Less than or equal to B j+1 .

14. The method according to any one of claims 8 to 13, characterized in that, The method further includes: The accumulation module adds the second floating-point number to the source floating-point scalar to obtain the target floating-point scalar.

15. A data processing device, characterized in that, The device includes: One or more processors; Memory, used to store one or more computer programs or instructions; When the one or more computer programs or instructions are executed by the one or more processors, the one or more processors perform the method as described in any one of claims 8 to 14.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code that, when executed by a computer or processor, causes the computer or processor to perform the method as described in any one of claims 8 to 14.

Images