Data processing method, multiply accumulator and chip

By performing mantissa pre-splitting and exponential alignment of operands, the problems of large area occupation and low efficiency of multiplication accumulator when processing data with different precision are solved, and high-efficiency multi-precision calculation is achieved.

CN119002862APending Publication Date: 2024-11-22TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202310552334.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-16
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

When existing multiplication accumulators process data with different precisions, multiplier multiplexing leads to large chip area occupancy and low computing efficiency, making it difficult to support high-efficiency multi-precision calculations at the same time.

Method used

By performing mantissa pre-splitting and exponential alignment of operands, unify the operand format, reduce the logical selection steps in multiplication and addition operations, and adopt a parallel multiplexing structure to improve computational efficiency.

Benefits of technology

The multiplication and accumulation operation of data with different precisions is realized on a smaller chip area, and the calculation efficiency and area utilization of the multiplication and accumulator are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119002862A_ABST
    Figure CN119002862A_ABST
Patent Text Reader

Abstract

The invention discloses a data processing method, a multiply accumulator and a chip, and relates to the technical field of computers, in particular to the technical field of software testing. The method comprises the following steps: acquiring a plurality of operands according to the parallel multiplexing number of computing units of a multiply-accumulator and a target precision type; changing mantissas of the plurality of operands into a plurality of target mantissas according to the target precision type and a preset mantissa data width parameter; performing multiplication operation on the multiple target mantissas to obtain a first mantissa multiplication result of the multiple target mantissas; performing alignment operation on the first mantissa multiplication result according to the indexes of the multiple operands to generate a second mantissa multiplication result; and performing addition operation on the second mantissa multiplication result to generate a mantissa addition result, matching the mantissa addition result with indexes corresponding to the multiple operands, and outputting a multiply-accumulate operation result of the multiple operands.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application generally relates to the field of computer technology, specifically to the field of artificial intelligence technology, and in particular to a data processing method, a multiplier-accumulator and a chip. Background Art

[0002] With the rapid development and progress of computer technology, the technology of artificial intelligence (AI) is also constantly developing and progressing.

[0003] In the related art, a large number of operations are performed through the multiply & accumulate (MAC) in the chip to complete AI training. Generally, AI training can be performed through MAC using single-precision floating point (FP32) data with higher precision. Training with FP32 data can make the accuracy of AI training higher; AI training can also be performed through MAC using half-precision floating point (FP16) data. Training with FP16 data can make the throughput of AI training larger per unit time, and the efficiency of AI training higher. In order to combine the advantages of FP16 data training and FP32 data training, in the related art, the multipliers used for FP16 data training and FP32 data training will be reused in MAC, and the multiplexer MUX will be used to select the result of the multiplexed output, so that the same multiplier accumulator can perform operations on data of different precisions.

[0004] However, in the above operation process, in order to reuse the multipliers of MAC, generally multiple multiplexers MUX are required to select among the multipliers, which will occupy more chip area. At the same time, the reuse of multipliers will also greatly reduce the operation efficiency. Summary of the invention

[0005] In view of the above-mentioned defects or deficiencies in the prior art, it is desired to provide a data processing method, a multiplier-accumulator and a chip, through which a high-computing-power multiplier-accumulator capable of calculating data of different precision types can continuously maintain high-efficiency calculation, while reducing the area occupied by the circuit structure of the multiplier-accumulator on the chip.

[0006] In a first aspect, the present application provides a data processing method, the method comprising:

[0007] According to the number of parallel multiplexing of the computing units of the multiplication and accumulation device and the target precision type, a plurality of operands are obtained, wherein the operands include a mantissa part and an exponent part;

[0008] According to the target precision type and the preset mantissa data width parameter, the mantissas of the plurality of operands are changed into a plurality of target mantissas, the preset mantissa data width parameter being related to the computing power of the computing unit, and the width of the target mantissa being the preset mantissa data width parameter;

[0009] Performing a multiplication operation on the plurality of target mantissas to obtain a first mantissa multiplication result of the plurality of target mantissas;

[0010] performing an alignment operation on the first mantissa multiplication result according to the exponents of the plurality of operands to generate a second mantissa multiplication result; the state of the second mantissa multiplication result is an alignment state that can be used to perform an operand addition operation;

[0011] An addition operation is performed on the second mantissa multiplication result to generate a mantissa addition result, the mantissa addition result is matched with the exponents corresponding to the plurality of operands, and the multiplication and accumulation operation results of the plurality of operands are output.

[0012] In a second aspect, the present application provides a multiplier-accumulator, the multiplier-accumulator comprising:

[0013] A mantissa pre-splitting unit, used to obtain a plurality of operands according to the number of parallel multiplexing of the computing units of the multiplier-accumulator and the target precision type, wherein the operands include a mantissa part and an exponent part;

[0014] The mantissa pre-splitting unit is further used to change the mantissas of the plurality of operands into a plurality of target mantissas according to the target precision type and a preset mantissa data width parameter, the preset mantissa data width parameter being related to the computing power of the computing unit, and the width of the target mantissa being the preset mantissa data width parameter;

[0015] A multiplication operation unit, configured to perform a multiplication operation on the plurality of target mantissas changed by the mantissa pre-splitting unit, and obtain a first mantissa multiplication result of the plurality of target mantissas;

[0016] an alignment operation unit, configured to perform an alignment operation on the first mantissa multiplication result obtained by the multiplication operation unit according to the exponents of the plurality of operands, to generate a second mantissa multiplication result; the state of the second mantissa multiplication result is an alignment state that can be used to perform an operand addition operation;

[0017] The addition operation unit is used to perform an addition operation on the second mantissa multiplication result generated by the alignment operation unit to generate a mantissa addition result, match the mantissa addition result with the exponents corresponding to the multiple operands, and output the multiplication and accumulation operation results of the multiple operands.

[0018] In one embodiment, the mantissa pre-splitting unit is specifically used for:

[0019] According to the above target precision type, determine the mantissa data width parameter of the above operand;

[0020] The mantissa data width parameter of the above-mentioned operand is changed according to the above-mentioned preset mantissa data width parameter to obtain the above-mentioned multiple target mantissas.

[0021] In one embodiment, the mantissa pre-splitting unit is specifically used for:

[0022] In the case where the mantissa data width parameters of the plurality of operands are smaller than the preset mantissa data width parameters, the mantissa data width parameters of the plurality of operands are expanded to the preset mantissa data width parameters to generate the plurality of target mantissas;

[0023] In the case where the mantissa data width parameters of the plurality of operands are greater than the preset mantissa data width parameter, splitting the plurality of operands into a high-order part and a low-order part to generate the plurality of target mantissas;

[0024] Among them, the above-mentioned high-order part includes a high-order mantissa, the above-mentioned low-order part includes a low-order mantissa, and the mantissa data width parameter of the above-mentioned high-order mantissa and the mantissa data width parameter of the above-mentioned low-order mantissa are equal to the above-mentioned preset mantissa data width parameter.

[0025] In one embodiment, the multiplication operation unit comprises:

[0026] A partial product generating unit, used for performing a product generating operation on the above-mentioned multiple target mantissas to generate multiple product values;

[0027] The partial product compression unit is used to perform a product compression operation on the multiple product values ​​generated by the partial product generation unit to generate the first mantissa multiplication result.

[0028] In one embodiment, the multiplier-accumulator further comprises:

[0029] An exponential processing unit, used for comparing the exponential data width parameters of the plurality of operands with a preset exponential data width parameter;

[0030] The exponential processing unit is further configured to change the exponential data width parameters of the plurality of operands to the preset exponential data width parameter when the exponential data width parameter of the operand is different from the preset exponential data width parameter.

[0031] In one embodiment, the alignment operation unit includes:

[0032] The rank shift operation unit is used to perform a rank shift operation on the first mantissa multiplication result to generate the second mantissa multiplication result.

[0033] In one embodiment, when the mantissa data width parameter of the operand is less than the preset mantissa data width parameter, the multiplier accumulator includes:

[0034] an exponential processing unit, configured to perform an exponential addition operation on the exponent of each of the plurality of operands to generate a plurality of exponential addition results;

[0035] The exponential processing unit is further used to compare the multiple exponential addition results generated by the exponential processing unit and select the maximum value of the multiple exponential addition results;

[0036] The exponential processing unit is further used to convert the exponential values ​​of the exponents of the plurality of operands into the maximum value screened by the exponential processing unit;

[0037] The exponential processing unit is further used to obtain a plurality of differences between the maximum value of the plurality of exponential addition results screened by the comparator and the plurality of exponential addition results;

[0038] an order shift operation unit, configured to perform an order shift operation on the first mantissa multiplication results corresponding to the plurality of difference values ​​respectively according to the plurality of difference values ​​obtained by the exponent processing unit, so as to generate the second mantissa multiplication result;

[0039] Among them, the above-mentioned first mantissa multiplication result includes multiple mantissa multiplication results, the above-mentioned multiple differences correspond one-to-one to the multiple first mantissa multiplication results, and the above-mentioned second mantissa multiplication result includes multiple mantissa multiplication results.

[0040] In one embodiment, when the mantissa data width parameter of the operand is greater than a preset mantissa data width parameter, the multiplier accumulator comprises:

[0041] an exponent processing unit, configured to select a shift value corresponding to each of the first mantissa multiplication results based on an exponent of a multiplication result of high-order parts of the plurality of operands, wherein the first mantissa multiplication result includes a plurality of mantissa multiplication results;

[0042] The order shift operation unit is used to perform an order shift operation on each target mantissa multiplication result in the first mantissa multiplication result according to the shift value selected by the exponent processing unit to generate the second mantissa multiplication result, wherein the second mantissa multiplication result includes multiple mantissa multiplication results.

[0043] In one embodiment, when the second mantissa multiplication result includes a plurality of carry results and a plurality of sum results, the multiplication accumulator includes:

[0044] A carry-retaining adder, used for performing a first addition operation on the plurality of carry results and the plurality of sum results to generate a target carry result and a target sum result;

[0045] The mantissa addition unit is used to perform a second addition operation on the target carry result and the target sum result generated by the carry-preserving adder to generate a mantissa addition result.

[0046] In one embodiment, the multiplication accumulator includes a leading zero prediction unit, a normalization shift unit, and an exponent adjustment unit:

[0047] Using the mantissa addition unit to perform the second addition operation on the target carry result and the target sum result, and using the leading zero prediction unit to perform leading zero prediction on the target carry result and the target sum result;

[0048] generating the third mantissa addition result by using the mantissa addition unit and generating the leading zero prediction result by using the leading zero prediction unit;

[0049] According to the leading zero prediction result, using the normalization shift unit to perform normalization shift on the third mantissa addition result to generate the mantissa addition result;

[0050] According to the shift value of the normalized shift, the exponent corresponding to the third mantissa addition result is adjusted by using the exponent adjustment unit to generate a target exponent.

[0051] In a third aspect, an embodiment of the present application provides a chip, including the multiplier-accumulator of the second aspect, wherein the multiplier-accumulator is used to implement the method described in the embodiment of the present application.

[0052] The data processing method, multiplier-accumulator and chip proposed in the present application take into account that in the existing technical solutions, when using the multiplier-accumulator to perform multiplication and accumulation operations on multiple operands, the multiplication operation of multiple operands is first completed through the multiplication unit to obtain multiple multiplication results; then, the multiplication results of the operands are selected using multiple path selectors MUX; after that, the multiplication results are input into the instantiation addition tree that conforms to the precision type of the operands for calculation; finally, a MUX is required to select the final multiplication and accumulation result. The above-mentioned multiplication and accumulation operation mainly has the following two problems: First, due to the large number of operands, a large number of multiplication results will be obtained. Second, if you choose to fully configure the service, you will not be able to obtain the indicator data of the control group that is not affected by the service release after the full configuration of the service, so you cannot complete the long-term testing of the service.

[0053] Therefore, in the present application, the multiplier-accumulator obtains multiple operands including a mantissa part and an exponent part according to the number of parallel multiplexing of its computing units and the target precision type, and then, according to the target precision type and a preset mantissa data width parameter (the preset mantissa data width parameter is related to the computing power of the above-mentioned computing unit), changes the mantissas of the multiple operands to multiple target mantissas (the width of the target mantissa is the preset mantissa data width parameter); then, performs a multiplication operation on the multiple target mantissas to obtain a first mantissa multiplication result of multiple target bits; then, according to the exponents of the above-mentioned multiple operands, performs an alignment operation on the first mantissa multiplication result to generate a second mantissa multiplication result (the state of the second mantissa multiplication result is an alignment state that can be used to perform operand addition operations), and then, performs an addition operation on the second mantissa multiplication result to generate a mantissa addition result, matches the mantissa addition result with the exponents corresponding to the multiple operands, and outputs the multiplication-accumulation operation results of the multiple operands. In this way, the obtained multiple operands are pre-processed by normalization first (for example, the mantissa digits of operands of different precision types are normalized), and then the mantissa multiplication results of the multiple operands are aligned according to the exponents of the multiple operands by the multiplication accumulator before performing the addition operation, so that the multiplication accumulator can perform the addition operation later, and finally the mantissa multiplication result and the exponent are matched, and the multiplication and addition operation result is output. In actual operation, the number of operands to be processed is very large, and the steps of the above data processing method are completed by a multiplication and addition array composed of multiple multiplication and accumulators. On the one hand, according to the multiplication and addition operation logic of operands of different precision types, the mantissa placeholder format of operands of different precision types is modified to a mantissa placeholder format consistent with the calculation unit of the multiplication and accumulator, and the operands are pre-processed by normalization; on the other hand, according to the addition operation logic of operands of different precision types, different types of alignment operations are performed on the mantissa multiplication results of operands of different precision types, so as to accurately perform the addition operation. Therefore, there is no need to set up a large number of multiplexers to screen the multiplication results corresponding to operands of different precision types, nor is there a need to perform addition operations on operands of different precision types through addition tree instantiation. This greatly reduces the occupied area of ​​the multiplication and accumulator, and at the same time, can greatly improve the computational efficiency of the multiplication and accumulator with high computing power.

[0054] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0056] Figure 1 A schematic diagram of a structure of a multiplication and accumulation device capable of calculating different types of precision in the related technology provided in an embodiment of the present application;

[0057] Figure 2 A hardware architecture diagram of a multiplier-accumulator provided in an embodiment of the present application;

[0058] Figure 3 A schematic diagram of a multiply-accumulate array composed of multiply-accumulate devices provided in an embodiment of the present application;

[0059] Figure 4 A flowchart of a data processing method provided in an embodiment of the present application;

[0060] Figure 5 A schematic diagram of operands of different precision types provided in an embodiment of the present application;

[0061] Figure 6 A schematic diagram of a multiplication operation of a group of operands with a precision type of FP32 provided in an embodiment of the present application;

[0062] Figure 7 A schematic diagram of a comparison process of exponential addition results for four groups of operands with a precision type of FP16 or BF16 provided in an embodiment of the present application;

[0063] Figure 8 A schematic diagram of a calculation process of the mantissa of four groups of operands of FP16 or BF16 for the order shift value provided in an embodiment of the present application;

[0064] Fig. 9 A schematic diagram of a calculation process of calculating a mantissa and an order shift value for a multiplication-accumulation-addition device that can be used to perform multiplication-accumulation operations on FP32, FP16 or BF16 operands provided in an embodiment of the present application;

[0065] Fig.10 A schematic diagram of a calculation process of a mantissa of an FP32 operand versus an order shift value provided in an embodiment of the present application;

[0066] Fig.11 A schematic diagram of the structure of a multiplier-accumulator provided in an embodiment of the present application;

[0067] Fig.12 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] The present application is optionally described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the invention are shown in the accompanying drawings.

[0069] It will be understood that the term "plurality" used in this application means "two" and "more than two".

[0070] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0071] The following explains the terms used in the embodiments of the present application:

[0072] 1) Multiply and Accumulate (MAC)

[0073] The multiplier-accumulator is mainly used to complete multiplication and accumulation operations such as vector multiplication, matrix multiplication, and vector-matrix multiplication. It is an extremely important operation subsystem in processors such as digital signal processors (DSP) and central processing units (CPU). As the architectural carrier of vector instructions, the multiplier-accumulator architecture can be applied in many fields to complete important functions, such as vector or matrix processing of encoding and decoding of long-term evolution (LTE) uplink and downlink multiple input multiple output communication technology (MIMO), antenna data weighting processing in downlink beam forming technology (BF), data processing in the signal processing field, and data processing in the graphics and image field.

[0074] 2) D flip-flop (data-flipflop, dff)

[0075] The D flip-flop is an information storage device with memory function and two stable states. It is the most basic logic unit that constitutes a variety of sequential circuits and is also an important unit circuit in digital logic circuits.

[0076] Therefore, D flip-flops are widely used in digital systems and computers. The flip-flop has two stable states, 0 and 1, and can flip from one stable state to another under certain external signals.

[0077] 3) Floating point numbers

[0078] Floating point numbers are digital representations of numbers belonging to a specific subset of rational numbers, and are used in computers to approximate any real number. Specifically, this real number is obtained by multiplying an integer or fixed-point number (i.e., the mantissa) by an integer power of a base (usually 2 in computers), which is similar to scientific notation with a base of 10.

[0079] A floating point number a is represented by two numbers m and e: a = m × b^e. In any such system, we choose a base b (the base of the number system) and a precision p (that is, how many bits are used to store it). m (the mantissa) is p digits of the form ±d.ddd...ddd (each digit is an integer between 0 and b-1, including 0 and b-1). If the first digit of m is a non-zero integer, m is called normalized. Some descriptions use a single sign bit (s for + or -) to indicate positive or negative, so that m must be positive and e is the exponent.

[0080] Generally, floating point numbers can include single-precision floating point numbers and half-precision floating point numbers.

[0081] The biggest difference between half-precision floating-point numbers and single-precision floating-point numbers is the amount of memory space they occupy.

[0082] 4) Partial product

[0083] In multiplication, if the multiplier is a two-digit number or more, when multiplying, each digit of the multiplier must be multiplied by the multiplicand. The product obtained each time is called a partial product, or incomplete product.

[0084] The following is an introduction to the related technologies before this application:

[0085] Different AI application scenarios require different types of calculations with different precisions. For example, in AI training, FP32 with relatively high precision is required for training so that the parameters obtained through training will be more accurate. Usually, high-precision data types such as FP32 are used. In AI reasoning, a relatively high throughput is required, that is, the number of inputs that can be processed per unit time. In this case, excessive precision will not have a significant impact on the reasoning effect. Usually, data types such as FP16 / BF16 are used. Compared with FP32, FP16 has more powerful computing power.

[0086] From the perspective of hardware implementation, the area of ​​the FP32 multiplier-accumulator is much larger than that of the FP16 multiplier-accumulator. Some AI processors only support operations of one data type, which can minimize the area and reduce the difficulty of implementation. If you want to support multiplier-accumulators in multiple modes, you need to focus on the area.

[0087] like Figure 1 As shown, Figure 1It is a multiplication-accumulation unit that can be used to perform multiplication-accumulation operations on operands of various precision types. The multiplication-accumulation unit is a multiplication-accumulation unit set on any chip. Specifically, the multiplication-accumulation unit can support multiplication-accumulation operations on operands of two precision types, FP32 or FP16. It can be understood that, in general, the working mode of the multiplication-accumulation unit is to first perform a multiplication operation on the operand, and then perform an addition operation, and finally output the multiplication-accumulation result. Generally, the multiplication operation is performed first using the multiplication unit, and then the addition operation is performed using the addition unit (such as an addition tree).

[0088] In the related art, a multiplication-accumulation unit capable of supporting the calculation of operands of the above two different precision types is composed of a multiplication unit, an addition unit (such as an addition tree) and a multiplexer. Specifically:

[0089] 1) Multiplication unit (multiply, mul) 101: The multiplication unit includes a digital logic gate combined with a transistor for performing multiplication operations and a plurality of multiplexers (multiplexers, mux). Specifically, the mul 101 can be used to perform multiplication operations for operands of two precision types, FP32 or FP16, and obtain multiplication results. That is, in the related art, for operands of different precision types, some multiplication resources of mul101 can be reused. Furthermore, the way in which the mul101 implements multiplexing is to set a plurality of multiplexers in mul101 to select the logic of the multiplication operation corresponding to the operands of the two precision types of FP32 or FP16, that is, using the same multiplier, different multiplication operation logics can be executed according to the selection of mux. Therefore, the mul 101 includes a plurality of multiplexers (multiplexers, mux). It can be understood that since the logic of the multiplication operation is not exactly the same for operands of different precision types, it is necessary to select the multiplication operation corresponding to the precision type of the operand through mux. Meanwhile, generally, the number of operands for performing the multiplication operation is relatively large, and accordingly, the number of multiplication results generated is relatively large, and therefore, the number of muxes for the multiplication operation is selected to be multiple muxes.

[0090] 2) Addition tree (add_tree) 102: The multiplication unit includes a digital logic gate combined with a transistor for performing addition operations. The add_tree 102 is mainly used to receive the multiplication results of the output of operands of two precision types, FP32 or FP16, and perform addition operations. It should be noted that in the process of performing addition operations, the multiplication results of operands of different precision types output by the aforementioned multiplication unit need to be input into different add_trees 102 to complete the addition operation. That is, the multiplication results of the output of operands of the precision type FP32 need to be input into the add_tree of FP32 to complete the addition operation, and the multiplication results of the output of operands of the precision type FP16 need to be input into the add_tree of FP16 to complete the addition operation. It can be understood that the above add_tree is an add_tree instantiated for multiplication results of different precisions. (The dotted box is only used for identification and does not exist in the actual entity). The above instantiation means that for operands of a specific precision type, the adder of the specific precision type needs to be used to perform the addition operation.

[0091] 3) Multiplexer (mux) 103: The mux 103 is mainly used to select the addition result. It should be noted that the mux 103 here is different from the multiple muxes in the mul 101 mentioned above. The multiplexer here can be a separate multiplexer specifically used to select the final result.

[0092] The specific operation steps are as follows: SA1: Input the FP32 operand or the FP16 operand into mul101, use multiple muxes to select the multiplication operation logic corresponding to the operand precision type to perform the multiplication operation, and obtain the multiplication result; SA2: Input the multiplication results of operands of different precision types into the corresponding add_tree102, and further perform addition operations on the multiplication results; SA3: Use mux103 to select the addition result, and finally output the multiplication and accumulation result.

[0093] It can be seen from the above process that the technical solution of the multiplication-accumulation operation performed by the multiplication-accumulation unit in the related art has the following problems: 1) Since mul101 is reused when performing multiplication operations on operands of different precision types, and during the reuse process, a large number of multiplication results are generated, multiple muxes are required to select the multiplication operation, which will occupy a large chip area; 2) In the process of multiplexing mul101, since multiple muxes are required to select the multiplication results, the logic delay of the multiplication operation will be caused, which will cause the duration of the multiplication operation to become longer and the calculation efficiency to decrease; 3) Since the amount of data input to add_tree of different precision types is different, add_tree is difficult to reuse, and it is necessary to instantiate the addition tree, that is, set different add_trees for addition operations of different precision types, which also increases the chip area occupied, that is, causes chip area consumption.

[0094] Based on this, an embodiment of the present application provides a data processing method, a multiplier-accumulator and a chip. According to the number of parallel multiplexing of the computing units of the multiplier-accumulator, the operands are preprocessed in advance (for example, the mantissa data width is changed), that is, the operands of different precision types are normalized before the multiplication operation is performed to obtain the mantissa multiplication result, and before the subsequent addition operation is performed, the multiplication result is aligned according to the exponent of the operand, so that the mantissa multiplication result can be subjected to mantissa addition operation in an aligned state, and the mantissa addition result is obtained. Finally, the mantissa addition result is matched with the exponents of multiple operands, and the multiplication-accumulation operation results of multiple operands are output.

[0095] Therefore, since the technical solution in the present application performs normalization preprocessing on the operands before the multiplication operation, and then performs the multiplication operation and obtains the mantissa multiplication result, and before the addition operation, the multiplication result is again normalized and aligned, so that the mantissa multiplication result can perform mantissa addition operation in an aligned state and obtain the mantissa addition result. Therefore, the technical solution of the present application greatly reduces the steps of logic selection and logic processing in the multiplication and addition operations, and accordingly, it will reduce the occupied area of ​​the multiplication accumulator on the chip. At the same time, since there is no need for a large number of logic selection and logic processing in the multiplication and addition operations, the computing efficiency will be correspondingly improved.

[0096] The specific application scenarios of this application are introduced as follows:

[0097] This application is mainly used in various AI application scenarios. It can be understood that for different AI application scenarios, multiplication and accumulation operations of operands of different precision types are used. For example, if the AI ​​application scenario is AI training, FP32 operands with relatively high precision are required for AI training, so that the parameters obtained by training will be more accurate, and high-precision operands such as FP32 are usually used. When the AI ​​application scenario is AI reasoning, relatively high throughput and computing efficiency are required, that is, a large number of operands that can be processed per unit time. In this case, excessive precision will not have a significant impact on the reasoning effect. Usually, lower-precision operands such as FP16 / BF16 are used. Compared with FP32 operands, when calculating FP16 operands, the computing power of the multiplier and accumulator is more powerful.

[0098] Figure 2 A schematic diagram of the structure of a multiplier-accumulator provided in an embodiment of the present application is shown in FIG. Figure 3 FIG. 1 is a schematic diagram of the structure of a multiplication-accumulation array formed by a multiplication-accumulation unit. It can be understood that the above Figure 2 The structure of a multiplier-accumulator shown is the structure of the multiplier-accumulator on a chip. Figure 3 The multiply-accumulate array shown is a schematic diagram of a structure composed of a single multiply-accumulate unit, wherein a processing engine (PE) represents a computing unit of a multiply-accumulate unit (the mantissa pre-split (man_pre_split) unit is outside the PE and is part of the multiply-accumulate unit. Generally, the processing engine of the multiply-accumulate unit can be reused in parallel, and the number of parallel reuses of the processing engine can be determined according to the number of types of precision types corresponding to the operands that can be calculated by the multiply-accumulate array. For example, when Figure 3 The types of precision that the multiplier-accumulator in can calculate are: FP32, FP16 and BF16, so the number of parallel multiplexing is 4, that is, Figure 3 The four processing engines indicated by the dotted box 301 are a group of parallel multiplexing. For FP16 operands or BF16 operands, the above four processing engines can process eight FP16 operands or eight BF16 operands in parallel; for FP16 operands or BF16 operands, the above four processing engines can process two FP32 operands in parallel.

[0099] It should be noted that the dashed box 301 is only used to illustrate the parallel multiplexing of the processing engine. Figure 3 The other processing engines of the multiplication-accumulation array in can also be multiplexed in parallel like the four processing engines in the dashed box 301.

[0100] Further, such as Figure 2 As shown, the multiplier-accumulator includes the following hardware components:

[0101] 1) Mantissa pre-splitting unit: used to perform format normalization preprocessing operations on the mantissa part. It can be understood that when the precision types of the operands include FP16, BF16 and FP32, the normalization preprocessing here can be: before entering the actual multiplication and accumulation operation, the mantissa placeholders of the operands are unified at the operation level, that is, all are unified to the same mantissa placeholders. It should be noted that the normalization operation here is only for the purpose of operational unification, which is convenient for the subsequent execution of multiplication and addition operations. It does not actually change the operands themselves (for example, the mantissa value size has not changed), and will not affect the final operation results. Specifically, for the operands of the above three types of precision, the mantissa digits of the FP16 operand and the mantissa digits of the BF16 operand can be supplemented with placeholders, and the mantissa of the FP32 operand can be split, so as to achieve the unification of the mantissa digits of the above three types of precision operands.

[0102] 2) Partial product generation (boothencoder) unit 202: Generally, the partial product generation unit may include a serially connected encoder and a selector, and is used to perform a partial product generation operation on the mantissa part of the operand.

[0103] 3) Partial product compression (pp_compress) unit 203: Generally, the partial product compression unit may include a compressor for performing a partial product compression operation on the mantissa of the operand. Further, the purpose of the partial product compression unit is to sum the partial products generated by the aforementioned partial product generation unit.

[0104] 4) Subtractor (sub): used to obtain the shift value of the order shift operation when the mantissa data width parameter of the operand is less than the preset mantissa data width parameter (in Figure 2 Not shown in Figure 8 and Fig. 9 Specifically, the maximum exponent value among all exponent values ​​of the multiple target mantissas can be compared by a comparator, and then the difference between all exponent values ​​and the maximum exponent value can be obtained by using sub, and the difference is the shift value of the order shift operation, and the details are described later.

[0105] 5) Multiplexer (mux): Multiplexer is another name for data selector. In the process of multi-channel data transmission, the circuit that can select any one of them according to the need is called a data selector, also known as a multiplexer or a multi-way switch. It is used to select the shift value of the shift operation when the mantissa data width parameter of the operand is greater than the preset mantissa data width parameter (in Figure 2 Not shown in Fig. 9 and Fig.10Detailed description is given in the following section.

[0106] 6) Alignment shifter unit: used to perform alignment shift operation on the mantissa of the operand according to the exponent. It is understandable that after the multiplication operation is performed on the mantissa, the partial products generated are not uniformly aligned in the exponent size and / or position, and the subsequent addition operation can only be performed if the position is unified (for example, the product of exponent 6 and exponent 3 cannot be added directly, and the exponent needs to be unified before they can be added; the position of the partial products may also be different, and cannot be added directly, and the position needs to be unified before they can be added).

[0107] 7) Carry Save Adder (CSA): CSA can use a partial product array compression unit, which is commonly known as a partial accumulation tree (PPRT). It is an adder used to sum a large number of operands. It is used to perform carry-save addition operations on the mantissa part of the operand. The CSA can unify multiple groups of different mantissa addition results into one group of mantissa addition results.

[0108] 8) Mantissa addition unit (adder): used to perform addition operation on the carry-preserved addition result of the mantissa part of the operand. The adder can unify a group of mantissa addition results into one mantissa addition result.

[0109] 9) Leading-Zero Anticipator (LZA) unit: It can be composed of a leading zero detection circuit and can work in parallel with the adder. It is a prediction unit that calculates the shift value of the addition result of the mantissa part. It is used to perform a leading zero prediction operation on the addition result of the mantissa part of the operand. The LZA unit can calculate the shift value corresponding to the current mantissa addition result according to the pre-set format of the final result to be obtained, that is, after the subsequent normalization shift unit shifts the shift value calculated by the leading zero prediction operation, the format of the mantissa addition result can be processed into the format of the final result to be obtained.

[0110] 10) Normalize shifter unit: may be a unit provided with normalize shift logic, and is used to perform a normalize shift operation on the mantissa part of the operand according to the leading zero prediction result of the leading zero prediction operation.

[0111] 11) Exponent processing (exponentprocess, exp_proc) unit: used to perform exponential addition operations.

[0112] 12) Exponent adjustment (exponentadjust, exp_adj) unit: used to perform an exponent adjustment operation on the exponent value of the exponent according to the shift value of the normalized shift of the mantissa.

[0113] 13) d-flipflop (data-flipflop, dff): It is used to perform beat processing operations after multiplication operations, addition operations, etc.

[0114] 14) Processing engine: used to perform multiplication and accumulation operations of operands. Multiple processing engines can be reused in parallel to jointly complete the multiplication and accumulation operations of operands. Several processing engines can form a multiplication and accumulation array (refer to Figure 3 It should be noted that the processing engine is used to perform the multiplication and accumulation operations of the operands. The preprocessing of the operands (i.e., the aforementioned format normalization preprocessing operation on the mantissa part) does not belong to the scope of the processing engine. That is, the mantissa pre-splitting unit is a unit independent of the processing engine. At the same time, in the multiplication array, the number of mantissa pre-splitting units is different depending on the number of parallel multiplexing. Figure 3 As shown, when the number of parallel multiplexing is 4, every 4 processing engines are equipped with 1 mantissa pre-splitting unit, and the mantissa pre-processing unit is used to perform normalized pre-processing operations on the mantissa parts of the operands executed by the 4 processing engines corresponding thereto.

[0115] It should be noted that the fundamental reason for using the subtractor sub when the mantissa data width parameter of the operand is smaller than the preset mantissa data width parameter, and using the multiplexer mux when the mantissa data width parameter of the operand is greater than the preset mantissa data width parameter, is that: when the mantissa data width parameter of the operand is smaller than the preset mantissa data width parameter, the normalization operation method of the mantissa is to supplement the mantissa placeholder to facilitate the subsequent calculation of the multiplication and accumulation results of multiple groups of operands (one group of operands is two operands); and when the mantissa data width parameter of the operand is greater than the preset mantissa data width parameter, the normalization operation method of the mantissa is generally to split the mantissa of a group of operands to facilitate the subsequent calculation of the multiplication and accumulation results of this group of operands (one group of operands is two operands). It is understandable that for the aforementioned multiple groups of operands, the exponents of the multiple groups of operands are most likely different. In order to perform addition operations later, it is necessary to unify the exponent values, which is specifically to compare the exponent sizes of the multiple groups of operands, and to unify the exponents of all operands based on the maximum exponent, and to calculate the shift value accordingly according to the change in the exponent, and to shift the mantissa according to the shift value before the addition operation can be performed. For a group of operands, the exponents of the group of operands are the same, and there is no need to compare the exponent sizes, but in order to perform addition operations later, it is also necessary to align the multiplication results. The alignment method of the multiplication results can directly use a multiplexer to select the mantissa shift value, and align according to the shift value before the addition operation can be performed. The specific operation steps here can refer to the subsequent description.

[0116] The following will be combined Figure 1 and Figure 2 The technical solution of the present application and how the technical solution of the present application solves the above technical problems are described in detail with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0117] like Figure 4 As shown, an embodiment of the present application provides a data processing method, which is specifically applied to the field of computer technology. The method can be applied to a multiplier-accumulator, and the method specifically includes the following steps:

[0118] S101. Acquire multiple operands according to the number of parallel multiplexing of computing units of the multiply-accumulate unit and the target precision type.

[0119] In the embodiment of the present application, the above operand includes a mantissa part and an exponent part.

[0120] In the embodiment of the present application, the operand is a floating point number. Among them, the floating point number may include a half-precision floating point number and a single-precision floating point number. The introduction of the half-precision floating point number and the single-precision floating point number can refer to the above description, which will not be repeated here.

[0121] In the embodiment of the present application, the plurality of operands may be an even number of operands, for example, 2 operands, or 8 operands.

[0122] Furthermore, in the case where the multiple operands are an even number of operands, the multiple operands may appear in pairs. For example, when the operands are 2 operands, it is 1 group of operands; when the operands are 8 operands, it is 4 groups of operands.

[0123] In the embodiment of the present application, the above-mentioned multiplication and accumulation device can be used to normalize the operands as described above, and can also be used to perform multiplication and accumulation operations on the operands.

[0124] It should be noted that:

[0125] 1) The multiplier-accumulator in the embodiment of the present application is different from the multiplier-accumulator in the previous technical solution. It can pre-normalize the operands and then can be used for operations on operands of different precision types, thereby greatly reducing the dependence on the selection of mux, so as to achieve the purpose of reducing the chip area occupied by the multiplier-accumulator. Please refer to the subsequent description for details.

[0126] 2) If Figure 2 In the multiplication and accumulation device of the embodiment of the present application shown in the figure, during the process of performing a single multiplication and accumulation operation, the operation object is an operand of the same precision type, and cannot be an operand of mixed precision types. Figure 3 The multiplication-accumulation array composed of multiple multiplication-accumulation units shown in the figure also has operands of the same precision type in the process of performing a single multiplication-accumulation operation. For example, if the operand is a half-precision floating-point number of FP16, the multiplication-accumulation unit and the entire multiplication-accumulation array composed of multiple multiplication-accumulation units perform multiplication-accumulation operations on multiple half-precision floating-point numbers of FP16 in the process of a single multiplication-accumulation operation, but cannot perform multiplication-accumulation operations on half-precision floating-point numbers of FP16 and single-precision floating-point numbers of FP32 at the same time, or cannot perform multiplication-accumulation operations on half-precision floating-point numbers of FP16 and half-precision floating-point numbers of BF16 at the same time.

[0127] In the embodiment of the present application, the above-mentioned target precision type refers to the precision type of the operand.

[0128] Exemplarily, as described above, the precision type of the operand may include the precision type of a half-precision floating point number, or may include the precision type of a single-precision floating point number, wherein the precision type is also referred to as a precision format.

[0129] In one example, when the operand is a half-precision floating-point number, the precision type of the operand may include any one of the following: BF16, FP16; when the operand is a single-precision operand, the precision type of the operand may be FP32.

[0130] Furthermore, all operands include the following three parts: a sign bit S (sign), an exponent bit E (exponent), and a mantissa bit m (mantissa). However, operands of different precision types have different mantissa data width parameters and / or different exponent data width parameters. The mantissa data width parameter is the number of mantissa bits, and correspondingly, the exponent data width parameter is the number of exponent bits.

[0131] Specifically, Figure 5 As shown in (a) in FIG. 1 , when the precision type of the operand is FP32, the number of bits of an operand includes 32 bits, of which the data width parameter of the sign bit (S) is 1 bit, the data width parameter of the exponent bit (E) is 8 bits, and the data width parameter of the mantissa bit (M) is 23 bits; as shown in FIG. Figure 5 As shown in (b) in FIG. 1 , when the precision type of the operand is BF16, the data width parameter of an operand includes 16 bits, of which the data width parameter of the sign bit (S) is 1 bit, the data width parameter of the exponent bit (E) is 8 bits, and the data width parameter of the mantissa bit (M) is 7 bits; Figure 5 As shown in (c), when the precision type of the operand is FP16, the data width parameter of an operand includes 16 bits, of which the data width parameter of the sign bit (S) is 1 bit, the data width parameter of the exponent bit (E) is 5 bits, and the data width parameter of the mantissa bit (M) is 10 bits.

[0132] In the embodiment of the present application, the above-mentioned calculation unit refers to: a unit in a multiplication and accumulation device for performing multiplication and accumulation operations. In one embodiment, the above-mentioned calculation unit may include a multiplication unit and an addition unit.

[0133] In the embodiment of the present application, the above-mentioned calculation unit is mainly used to perform multiplication and accumulation operations of the mantissa.

[0134] In the embodiment of the present application, the number of parallel multiplexing of computing units refers to the number of computing units that need to be started simultaneously when the multiplication and accumulation unit performs a multiplication and accumulation operation. The number of parallel multiplexing of computing units is determined by referring to the number of mantissa bits of the operand before performing the multiplication and accumulation operation on the operand.

[0135] It is understandable that:

[0136] The exponent data width parameters and mantissa data width parameters of operands of different precision types are different. Therefore, in the process of multiplying and adding different operands, the multiplying and adding operation logic (including the logic of multiplication operation and the logic of addition operation) used is also completely different. Therefore, if the same multiplying and accumulating unit is required to calculate operands of different precision types, the multiplying and accumulating unit needs to switch the operation logic when calculating operands of different precision types.

[0137] It can be seen from the foregoing that in the related technologies before the embodiments of the present application, in order to enable the same multiplier-accumulator to calculate operands of different precision types, a large number of multiplexers mux were used to complete the switching of operation logic when calculating operands of different precision types. That is, in the related technologies, the multiplier-accumulator needs to select the operation logic of the multiplication operation and the logic of the addition operation that conform to the current precision type of operands through the assistance of mux. The problems brought about by this are operation delay and the multiplier-accumulator occupies too large a chip area. At the same time, the computing power requirements for the operands are often high, that is, a large number of operands need to be calculated at the same time. At this time, the problems of operation delay and the multiplier-accumulator occupies too large a chip area will be more prominent.

[0138] In order to enable the multiplier-accumulator in the embodiment of the present application to complete the multiplication-accumulation operation on operands of different precision types with the minimum occupied area and maximum efficiency, the embodiment of the present application first needs to unify the formats of operands of different precision types (that is, normalization operation). In this way, in the subsequent process of the multiplier-accumulator performing multiplication-accumulation operations, the operation logic of the multiplication operation and the logic of the addition operation can well complete the multiplication-accumulation operation even if the same operation logic is used, thereby reducing the use of a large number of muxes, and well avoiding the operation delay caused by a large number of muxes and the problem of excessive chip area occupied by the multiplier-accumulator.

[0139] Specifically, since the mantissa data width parameters of operands of different precision types are likely to be different (the exponent data width parameters of operands of certain precision types are also different from the exponent data width parameters of operands of other precision types), the normalization operation refers to the need to unify the mantissa data width parameters of operands of different precision types, or the mantissa data width parameters and exponent data width parameters of operands of different precision types. On this basis, after the format of the operands is unified, in order to maximize the efficiency of the multiplication and accumulation unit during the calculation process, according to the data width parameters of the operands, the number of operands that can be calculated at a single time corresponding to operands of different precision types will change accordingly, and the number of parallel reuse of the computing units of the multiplication and accumulation unit will also change adaptively.

[0140] In one embodiment, since the mantissa data width parameters of the FP32 operands are different from the mantissa data width parameters of the FP16 operands and the BF16 operands, and the mantissa data width parameters of the FP32 are quite different from those of the FP16 and BF16, the middle number of the mantissa data width parameters of operands of different precision types is selected as a reference value for determining the number of parallel multiplexing.

[0141] Example 1: For operands of three precision types, namely, FP32 operands, FP16 operands, and BF16 operands, the number of parallel multiplexing of computing units is 4, that is, for the multiplication and accumulation unit, 4 multiplication units and 4 addition units need to be enabled at the same time to perform multiplication and accumulation operations.

[0142] It should be noted that, since the exponent data width parameters of some precision types (such as FP32 and BF16) are consistent, in the embodiments of the present application, after determining the target precision type of the operand, there is also a situation where only the mantissa data width parameter can be changed without changing the exponent data width parameter. The specific mantissa change can refer to the subsequent description.

[0143] S102. According to the target precision type and a preset mantissa data width parameter, the mantissas of the plurality of operands are changed into a plurality of target mantissas.

[0144] In the embodiment of the present application, the above-mentioned preset mantissa data width parameter is related to the computing power of the above-mentioned computing unit, and the width of the above-mentioned target mantissa is the above-mentioned preset mantissa data width parameter.

[0145] In the embodiment of the present application, the above-mentioned target accuracy type can refer to the above description and will not be repeated here.

[0146] In the embodiment of the present application, the above-mentioned preset mantissa data width parameter can be preset by the multiplier-accumulator, or can be customized by the user, and the embodiment of the present application does not limit this.

[0147] In the embodiment of the present application, changing the mantissas of the above-mentioned multiple operands into multiple target mantissas means: changing the mantissa data width parameters of the above-mentioned multiple operands into preset mantissa data width parameters.

[0148] In the embodiment of the present application, the computing power of the above-mentioned computing unit refers to the computing power of the unit in the multiplier-accumulator for performing multiplication and addition operations, that is, the mantissa data width parameter corresponding to the highest computing efficiency when the computing unit performs multiplication and addition operations.

[0149] Example 2: For the three types of precision operands, namely, FP32 operands, FP16 operands, and BF16 operands, their respective mantissa data width parameters are: 24 bits (23 bits of original mantissa bits + 1 bit of implicit bit), 11 bits (10 bits of original mantissa bits + 1 bit of implicit bit), and 8 bits (7 bits of original mantissa bits + 1 bit of implicit bit). Since the number of parallel multiplexing of the three types of precision operands, namely, FP32 operands, FP16 operands, and BF16 operands, is 4, the mantissas of the above-mentioned operands of different precisions can be unified to 12 bits, that is, the preset mantissa data width parameter is 12 bits. That is, the mantissa data width parameter of the above-mentioned FP32 operands, FP16 operands, or BF16 operands can be changed to 12 bits to generate the target mantissa.

[0150] S103, performing a multiplication operation on the above-mentioned multiple target mantissas to obtain a first mantissa multiplication result of the above-mentioned multiple target mantissas.

[0151] It should be noted that, in the process of the multiplier-accumulator performing multiplication-accumulation operations on multiple operands, the multiplication-accumulation operations are mainly performed on the mantissas, and some addition operations are performed on the exponents (see the subsequent description for details). That is, the mantissas and exponents of multiple operands are calculated separately in the multiplier-accumulator. At the end of the calculation, the mantissas are matched and combined with the corresponding exponents to output the final multiplication-accumulation operation results.

[0152] The mantissas and exponents of multiple operands are calculated separately, and the complete process of the multiplication and accumulation operation is mainly performed on the mantissas.

[0153] In the embodiment of the present application, the multiplication operation may be a multiplication operation for obtaining a partial product.

[0154] In the embodiment of the present application, when the multiplication operation is performed on the above-mentioned multiple target mantissas, the multiplication operation is performed on the multiple target mantissas in pairs.

[0155] In an example, the first mantissa multiplication result may include a carry result (carry) and a sum result (sum).

[0156] In an embodiment of the present application, the number of the first mantissa multiplication results of the above-mentioned multiple target mantissas is multiple, and the number corresponds to the target mantissa.

[0157] In one example, when the above-mentioned first mantissa multiplication result can include a carry result and a sum result, if the above-mentioned multiple target mantissas are 8 target mantissas, the multiple target bits are multiplied in pairs, and the multiplication results of every two target mantissas include a carry result and a sum result, then the above-mentioned first mantissa multiplication result includes a total of 8 first mantissa multiplication results, which are 4 carry results and 4 sum results respectively.

[0158] S104. Perform an alignment operation on the first mantissa multiplication result according to the exponents of the plurality of operands to generate a second mantissa multiplication result.

[0159] In the embodiment of the present application, the state of the second mantissa multiplication result is an alignment state that can be used to perform operand addition operation.

[0160] It can be understood that, from the foregoing, the mantissas and exponents of multiple operands are calculated separately in the multiplication accumulator, and after obtaining the first mantissa multiplication result, the first mantissa multiplication result is multiple. There may be a situation where the mantissa placeholders are not aligned between the multiple first mantissa multiplication results, in which case the subsequent addition operation cannot be performed. Therefore, it is necessary to align the mantissa placeholders through an alignment operation to generate a second mantissa multiplication result, and the mantissa placeholders of the second mantissa multiplication result are in an aligned state, so that the conditions for performing the operand addition operation are met.

[0161] In the embodiment of the present application, the number of the second mantissa multiplication results is the same as the number of the first mantissa multiplication results. For example, when the first mantissa multiplication results include 8 first mantissa multiplication results in total, the number of the second mantissa multiplication results is also 8.

[0162] S105, performing an addition operation on the second mantissa multiplication result to generate a mantissa addition result, matching the mantissa addition result with the exponents corresponding to the plurality of operands, and outputting the multiplication and accumulation operation results of the plurality of operands.

[0163] In one embodiment, the above addition operation may include carry-preserved addition and final mantissa result addition.

[0164] It can be understood from the foregoing that since the multiplication and accumulation unit operates the mantissa and exponent of the operands separately, when obtaining the multiplication and accumulation results, it is necessary to match and combine the mantissa addition results and the exponent obtained by the separate operations in order to output the multiplication and accumulation results of multiple operands.

[0165] The data processing method proposed in the embodiment of the present application takes into account that in the existing technical solutions, when using a multiplication and accumulation unit to perform multiplication and accumulation operations on multiple operands, the multiplication operation of multiple operands is first completed through the multiplication unit to obtain multiple multiplication results; then, multiple path selectors MUX are used to select the multiplication results of the operands; after that, the multiplication results are input into the instantiation addition tree that conforms to the precision type of the operands for calculation; finally, a MUX is required to select the final multiplication and accumulation result. The above-mentioned multiplication and accumulation operation mainly has the following two problems: First, due to the large number of operands, a large number of multiplication results will be obtained. Second, if you choose to fully configure the service, you cannot obtain the indicator data of the control group that is not affected by the service release after the full configuration of the service, so you cannot complete the long-term testing of the service.

[0166] Therefore, in the data processing method proposed in the embodiment of the present application, the multiplier-accumulator obtains multiple operands including a mantissa part and an exponent part according to the number of parallel multiplexing of its computing units and the target precision type, and then, according to the target precision type and a preset mantissa data width parameter (the preset mantissa data width parameter is related to the computing power of the computing unit), the mantissas of the multiple operands are changed to multiple target mantissas (the width of the target mantissa is the preset mantissa data width parameter); thereafter, a multiplication operation is performed on the multiple target mantissas to obtain a first mantissa multiplication result of multiple target bits; then, according to the exponents of the above-mentioned multiple operands, an alignment operation is performed on the first mantissa multiplication result to generate a second mantissa multiplication result (the state of the second mantissa multiplication result is an alignment state that can be used to perform operand addition operations), and then, an addition operation is performed on the second mantissa multiplication result to generate a mantissa addition result, the mantissa addition result is matched with the exponents corresponding to the multiple operands, and the multiplication-accumulation operation results of the multiple operands are output. In this way, the obtained multiple operands are pre-processed by normalization first (for example, the mantissa digits of operands of different precision types are normalized), and then the mantissa multiplication results of the multiple operands are aligned according to the exponents of the multiple operands by the multiplication accumulator before performing the addition operation, so that the multiplication accumulator can perform the addition operation later, and finally the mantissa multiplication result and the exponent are matched, and the multiplication and addition operation result is output. In actual operation, the number of operands to be processed is very large, and the steps of the above data processing method are completed by a multiplication and addition array composed of multiple multiplication and accumulators. On the one hand, according to the multiplication and addition operation logic of operands of different precision types, the mantissa placeholder format of operands of different precision types is modified to a mantissa placeholder format consistent with the calculation unit of the multiplication and accumulator, and the operands are pre-processed by normalization; on the other hand, according to the addition operation logic of operands of different precision types, different types of alignment operations are performed on the mantissa multiplication results of operands of different precision types, so as to accurately perform the addition operation. Therefore, there is no need to set up a large number of multiplexers to screen the multiplication results corresponding to operands of different precision types, nor is there a need to perform addition operations on operands of different precision types through addition tree instantiation. This greatly reduces the occupied area of ​​the multiplication and accumulator, and at the same time, can greatly improve the computational efficiency of the multiplication and accumulator with high computing power.

[0167] In one embodiment of the present application, before changing the mantissa data width parameter of the operand, it is necessary to first confirm the actual precision type of the operand, that is, the target precision type of the operand, so as to determine the change method of the mantissas of multiple operands in the subsequent process. Therefore, in one implementation, the mantissas of the multiple operands are changed to multiple target mantissas according to the target precision type and the preset mantissa data width parameter, including: determining the mantissa data width parameter of the operand according to the target precision type; changing the mantissa data width parameter of the operand according to the preset mantissa data width parameter to obtain the multiple target mantissas.

[0168] It can be understood that, as described above, the mantissa data width parameters of operands of different precision types are different. Therefore, it is necessary to confirm the mantissa data width parameter according to the precision type corresponding to the operand, and then change the mantissa of each operand according to the preset mantissa data width parameter.

[0169] In one embodiment of the present application, the mantissas of multiple operands of different precision types are changed in different ways, and the mantissa change ways are related to the mantissa data width parameters of the multiple operands. Therefore, in one implementation, the mantissa data width parameters of the above-mentioned operands are changed according to the preset mantissa data width parameters to obtain the above-mentioned multiple target mantissas, including: when the mantissa data width parameters of the above-mentioned multiple operands are less than the above-mentioned preset mantissa data width parameters, the mantissa data width parameters of the above-mentioned multiple operands are expanded to the preset mantissa data width parameters to generate the above-mentioned multiple target mantissas; when the mantissa data width parameters of the above-mentioned multiple operands are greater than the above-mentioned preset mantissa data width parameters, the above-mentioned multiple operands are split into high-order parts and low-order parts to generate the above-mentioned multiple target mantissas.

[0170] Exemplarily, the high-order part includes a high-order mantissa, the low-order part includes a low-order mantissa, and the mantissa data width parameter of the high-order mantissa and the mantissa data width parameter of the low-order mantissa are equal to the preset mantissa data width parameter.

[0171] Exemplarily, the above-mentioned expanding the mantissa data width parameters of the above-mentioned multiple operands to the preset mantissa data width parameters means: expanding the mantissa data width parameters of each operand in the multiple operands to the preset mantissa data width parameters. Correspondingly, the above-mentioned splitting the above-mentioned multiple operands into high-order parts and low-order parts means: splitting each operand in the above-mentioned multiple operands into high-order parts and low-order parts.

[0172] It is understandable that in the subsequent process of executing the multiplication operation, each of the multiple operands needs to participate, so the object of the above-mentioned "expansion" or "splitting" is also for each of the multiple operands.

[0173] Furthermore, when the mantissa data width parameters of multiple operands are smaller than the above-mentioned preset mantissa data width parameters, the mantissa data widths of the multiple operands need to be padded to be the same as the preset mantissa data width parameters; when the mantissa data width parameters of the above-mentioned multiple operands are larger than the above-mentioned preset mantissa data width parameters, the multiple operands need to be split, and the mantissa data width parameters of the multiple operands after the splitting are the same as the preset mantissa data width parameters.

[0174] In a possible embodiment, when the mantissa data width parameters of the above-mentioned multiple operands are greater than the above-mentioned preset mantissa data width parameters, if direct splitting cannot make the mantissa data width parameters of the multiple operands after splitting the same as the preset mantissa data width parameters, then a padding operation can be performed on the multiple operands first, and the mantissa data width parameter of each operand in the multiple operands can be padded to an integer multiple of the preset mantissa data width parameter, and then the multiple operands can be split.

[0175] Example 4: In combination with the above Example 2, the following specifically introduces the specific process of changing the mantissa of three types of operands, namely, FP32 operands, FP16 operands, and BF16 operands, to the target number of bits.

[0176] As known from the above content, for the three types of precision operands, FP32 operands, FP16 operands, and BF16 operands, their respective mantissa data width parameters are: 24bit (23bit original mantissa bits + 1bit implicit bit), 11bit (10bit original mantissa bits + 1bit implicit bit), 8bit (7bit original mantissa bits + 1bit implicit bit), and at the same time, the preset mantissa data width parameter is 12bit, and the number of parallel multiplexing of computing units is 4. The above-mentioned number of parallel multiplexing of computing units is 4, which means that a single multiplication accumulator has 4 multiplication units that can be used to perform multiplication operations and addition units for addition operations.

[0177] 1) For FP16 operands, the mantissa change process is as follows: Since the mantissa data width parameter of the FP16 operand is 11 bits, the mantissa of the FP16 operand is expanded to 12 bits (the preset mantissa data width parameter).

[0178] 2) For the operand of BF16, the mantissa changing process is: since its mantissa data width parameter is 8 bits, the mantissa of BF16 is expanded to 12 bits (the preset mantissa data width parameter).

[0179] 3) For FP32 operands, the mantissa change process is as follows: Since the mantissa data width parameter is 24 bits, each of the two operands with a mantissa data width parameter of 24 bits can be split into an operand with a mantissa width parameter of 12 bits (preset mantissa data width parameter). Each split operand includes a high-order part and a low-order part.

[0180] Furthermore, since the number of computing units reused in parallel is 4, when the precision type of the operand is BF16 or FP16, 8 operands need to be input into a multiplication accumulator, and then they are paired up for multiplication, that is, each of the 4 groups of operands is multiplied separately, so that 4 computing units can be reused in parallel to perform multiplication operations; when the precision type of the operand is FP32, only 2 operands need to be input into a multiplication accumulator, that is, one group of operands, and each of the two operands is split into 2 operands, and then they are paired up for multiplication, and finally 4 groups of operands are multiplied.

[0181] Specifically, for a multiplication operation with an operand precision type of FP32, Figure 6 As shown:

[0182] Example 5: Assume that a set of FP32 operands are operands A and B, and according to the preset mantissa data width parameter of 12 bits, they are split into operands A_H and A_L (H represents the high-order part, L represents the low-order part), and operands B_H and B_L, where the mantissa width parameters of operands A_H, A_L, B_H and B_L are all 12 bits. When performing subsequent multiplication operations, operands A_H, A_L, B_H and B_L need to be cross-paired in pairs, forming a total of 4 groups of multiplications, namely the first multiplication result of the multiplication operation of operands B_L and A_L, the second multiplication result of the multiplication operation of operands A_H and B_L, the third multiplication result of the multiplication operation of operands B_H and A_L, and the fourth multiplication result of the multiplication operation of operands A_H and B_H.

[0183] In this way, by changing the mantissa of operands of different precision types as described above, the mantissa data width parameters of the operands are made the same, that is, a normalization change operation is performed. Then, when multiplication and accumulation operations are performed on the operands subsequently, a unified operation logic can be used to avoid using mux selection operation logic, thereby greatly shortening the operation time and reducing the area ratio of the chip occupied by the multiplication and accumulation unit.

[0184] In one embodiment of the present application, a specific process of how to perform a multiplication operation on the above-mentioned multiple target mantissas is described. The above-mentioned multiplication operation is performed on the above-mentioned multiple target mantissas to obtain the first mantissa multiplication result of the above-mentioned multiple target mantissas, including: performing a product generation operation on the above-mentioned multiple target mantissas to generate multiple product values; performing a product compression operation on the above-mentioned multiple product values ​​to generate the above-mentioned first mantissa multiplication result.

[0185] It can be understood that the above multiplication operation includes two steps, the first step is to perform a product generation operation, and the second step is to perform a product compression operation. Accordingly, the multiplication unit that performs the product generation operation and the multiplication unit that performs the product compression operation both belong to the computing unit of the aforementioned multiplication accumulator, so they can be reused in parallel. For example, in the above examples 1 to 5, there are 4 computing units of the multiplication accumulator, and then there are 4 multiplication units that perform the product generation operation and 4 multiplication units that perform the product compression operation.

[0186] In one embodiment, the above-mentioned product generation operation includes: a partial product generation operation of multiple target mantissas.

[0187] Further, in the case where the above-mentioned product generation operation is a partial product generation operation, the above-mentioned multiple product values ​​are multiple partial product generation results.

[0188] In one embodiment, the product compression operation includes: a partial product compression operation.

[0189] Further, in the case where the above-mentioned multiple product values ​​are multiple partial product generation results, the above-mentioned partial product compression operation is multiple partial product compression results, that is, the first mantissa multiplication result.

[0190] Example 6: The following is an example of a specific description of the process of generating the above-mentioned first mantissa multiplication result: Based on the above-mentioned Examples 1 to 4, it can be seen that the number of parallel multiplexing of computing units is 4, that is, there are 4 multiplication units, and each multiplication unit includes a partial product generation unit and a partial product compression unit. Correspondingly, for a multiplication accumulator, the multiplication unit needs to receive 4 groups of target mantissas to perform multiplication operations. Among them, for operands with a precision type of FP16 or BF16, the above-mentioned 4 groups of target mantissas are the target mantissas of 8 operands, and for operands with a precision type of FP32, the above-mentioned 4 groups of target mantissas are 4 groups of target mantissas corresponding to the multiplication of two operands after splitting 2 operands into 4 operands (refer to the above-mentioned Example 5 for details).

[0191] After performing the partial product generation operation on the four groups of target mantissas, each group of target mantissas will generate 7 partial products, generating 28 partial products. Then, the 28 partial products are divided into groups according to the previous grouping, and the partial product compression operation is continued for every 7 partial products as a group. Finally, the partial product compression operation will compress the 7 partial products into a first mantissa multiplication result, and finally generate 4 first mantissa multiplication results. Among them, each first mantissa multiplication result includes a carry result (carry) and a sum result (sum), that is, 4 first mantissa multiplication results are finally generated.

[0192] In one embodiment of the present application, it can be seen from the above content that the alignment operation is performed according to the exponents of multiple operands, but before performing the alignment operation, if the data width parameters of the exponents are not uniform, the subsequent alignment operation cannot be performed. Therefore, before the alignment operation, it is necessary to normalize the data width parameters of the exponents. Specifically, before performing the alignment operation on the first mantissa multiplication result according to the exponents of the above multiple operands, the above method also includes: comparing the exponent data width parameters of the above multiple operands with the preset exponent data width parameters; when the exponent data width parameters of the above operands are different from the preset exponent data width parameters, changing the exponent data width parameters of the above multiple operands to the preset exponent data width parameters.

[0193] It should be noted that the logical relationship between the above-mentioned "comparing the exponential data width parameters of the above-mentioned multiple operands with the preset exponential data width parameters" and step S103 can be before S103, can also be after S103, and can also be in parallel with S103, and the embodiment of the present application does not limit this.

[0194] Exemplarily, the above-mentioned preset exponential data width parameter may be preset by the multiplier-accumulator, or may be user-defined, which is not limited in the embodiments of the present application.

[0195] Exemplarily, the method of changing the exponential data width parameters of the plurality of operands to the preset exponential data width parameters may be to expand the exponential data width parameters. For example, when the exponential data width parameters of the plurality of operands are smaller than the preset exponential data width parameters, the exponential data width parameters of the plurality of operands are expanded to the preset exponential data width parameters.

[0196] Example 7, combined with the above content, it can be seen that when the operand is an FP32 operand, an FP16 operand or a BF16 operand, the exponent data width parameters of these three operands are 8 bits, 5 bits or 8 bits respectively, that is, the exponent data width parameter of the FP16 operand is different from the exponent data width parameters of the other two precision types of operands. Therefore, the preset exponent data width parameter can be set to 8 bits, and before performing the alignment operation, the exponent data width parameters of multiple operands are compared to see if they are 8 bits. If they are not 8 bits, the exponent data width parameters of multiple operands are expanded (i.e., the above-mentioned change) to 8 bits.

[0197] In this way, before the alignment operation, the actual exponential data width parameter of the operand is compared with the preset exponential data width parameter, and when the exponential data width parameters of the two are different, they are unified, that is, a normalization operation is performed, so that in the subsequent exponential addition operation, there is no need to select the operation logic through mux or other selection hardware, thereby greatly improving the operation efficiency and reducing the chip area occupied by the selection hardware.

[0198] In one embodiment of the present application, the alignment operation may be a shift operation, and performing the alignment operation on the first mantissa multiplication result to generate the second mantissa multiplication result includes: performing a shift operation on the first mantissa multiplication result to generate the second mantissa multiplication result.

[0199] Exemplarily, the above-mentioned order shift operation refers to: aligning the placeholder of the first mantissa multiplication result.

[0200] It can be understood that the alignment and shift operation actually includes two steps, alignment and shift, wherein alignment is used to calculate the difference in the number of bits between the first mantissa multiplication results in different positions, and shift is used to align the positions of the first mantissa multiplication results in different positions. Based on this, when the positions of multiple first mantissa multiplication results are different, it is necessary to first select an alignment standard, and then calculate the alignment mantissa according to the standard and complete the shift.

[0201] Furthermore, for operands with different mantissa data width parameters, after performing the multiplication operation to obtain the first multiplication result, the ordering method is different. The reason is:

[0202] 1) For operands whose mantissa data width parameter is smaller than a preset mantissa data width parameter (for example, operands with a precision type of FP16 or a precision type of BF16), since the first mantissa multiplication result obtained after performing the multiplication operation is obtained by multiplying several different operands by two each, and the differences in the operands themselves will result in different exponent values ​​of the exponents, the order shift operation process is as follows: select the exponent of a first mantissa multiplication result among multiple first mantissa multiplication results as a reference, change the exponent values ​​of the exponents of other first mantissa multiplication results accordingly, and shift the mantissa right.

[0203] 2) For operands whose mantissa data width parameter is greater than the preset mantissa data width parameter (for example, operands with precision type FP32), since the first mantissa multiplication result obtained after performing the multiplication operation can be obtained by splitting two operands and then multiplying them, that is, the operands are the same group of operands, and the exponent value of the exponent is always the exponent value of the same group of operands, that is, the exponent value is the same. Therefore, the order shift operation process is: select the number of digits occupied by the exponents of the two high-order part multiplication results among multiple first mantissa multiplication results as the basis, and change the number of digits occupied by the mantissas of other first mantissa multiplication results accordingly.

[0204] The two different order shifting methods are described below.

[0205] In one embodiment of the present application, the first shifting mode is described. When the mantissa data width parameter of the operand is less than the preset mantissa data width parameter, the alignment operation is performed on the first mantissa multiplication result to generate the second mantissa multiplication result, including: performing an exponential addition operation on the exponent of each operand in the multiple operands to generate multiple exponential addition results; comparing the multiple exponential addition results to select the maximum value of the multiple exponential addition results; converting the exponential values ​​of the exponents of the multiple operands to the maximum value; obtaining multiple differences between the maximum value of the multiple exponential addition results and the multiple exponential addition results; and performing a shifting operation on the first mantissa multiplication results corresponding to the multiple differences according to the multiple differences to generate the second mantissa multiplication result.

[0206] In one example, the mantissa data width parameter of the above-mentioned multiple operands is less than the preset mantissa data width parameter means that the mantissa data width parameter of each operand in the multiple operands is less than the preset mantissa data width parameter. For example, when the multiple operands are operands of precision type FP16 or BF16, and the preset mantissa data width parameter is 12 bits, the mantissa data width parameters of the FP16 operand or the BF16 operand are 8 bits (including 1 bit implicit bit) and 11 bits (including 1 bit implicit bit), respectively.

[0207] Exemplarily, the first mantissa multiplication result includes multiple mantissa multiplication results, the multiple differences correspond one-to-one to the multiple first mantissa multiplication results, and the second mantissa multiplication result includes multiple mantissa multiplication results.

[0208] Illustratively, the multiple difference values ​​mentioned above may be alignment values ​​required to perform an alignment operation.

[0209] In an example, the alignment value may be a shift value.

[0210] Exemplarily, the above exponent addition operation refers to: corresponding to the mantissa multiplication operation and the addition operation.

[0211] Example 8, such as Figure 7 As shown, in the above examples 1 to 4, there are 4 groups of target mantissas corresponding to 4 groups of operands, and correspondingly there are 4 groups of exponents corresponding to the 4 groups of operands, the first group is the exponent a0exp of the a0 operand, the exponent b0exp of the b0 operand, the second group is the exponent a1exp of the a1 operand, the exponent b1exp of the b1 operand, the third group is the exponent a2exp of the a2 operand, the exponent b2exp of the b2 operand, the fourth group is the exponent a3exp of the a3 operand, the exponent b3exp of the b3 operand, the 4 groups of exponents are respectively added using 4 exponent addition units to obtain 4 addition results, namely 1 group of exponential addition results add0, 2 groups of exponential addition results add1, 3 groups of exponential addition results add2, and 4 groups of exponential addition results add3.

[0212] For example, from the above description of the rank shift operation, it can be seen that before the shift, a rank standard needs to be selected first.

[0213] In one embodiment, the criterion for the order is the exponent corresponding to the largest addition result among the multiple addition results.

[0214] Furthermore, the above screening method of screening out the maximum value of multiple exponential addition results from multiple exponential addition results can be screened using a comparator.

[0215] Example 9: In the case where four exponential addition results are obtained in the above example 8, two-stage comparators can be used to perform two screenings to select the largest exponential addition result among the four exponential addition results. Figure 7 As shown, the first-stage comparator includes comparator 1 and comparator 2, and the second-stage comparator includes comparator 3.

[0216] First, use comparator 1 to compare add0 and add1, and use comparator 2 to compare add2 and add3; it is concluded that add0 is the maximum exponent addition result of add0 and add1, and add3 is the maximum exponent addition result of add2 and add3. Then, use comparator 3 to compare add0 and add3, and it is concluded that add3 is the maximum exponent addition result addmax of add0 and add3, which is also the maximum exponent addition result among all exponent addition results.

[0217] Exemplarily, the method for obtaining the multiple differences between the maximum value among the multiple exponential addition results and the multiple exponential addition results is: using a subtractor, respectively subtracting the multiple exponential addition results from the maximum exponential addition result, and the difference is the number of bits when the first mantissa result is subsequently shifted, that is, the data width parameter when the first mantissa result is subsequently shifted.

[0218] The following example uses an operand as a floating-point number to describe the order shift operation before the addition operation: Assuming that two floating-point numbers 1.0000*2^5 and 1.0000*2^3 are to be added, the maximum value of the exponents of the two floating-point numbers, that is, 5, needs to be selected through a comparator; then, based on 2^5, 2^3 is converted to 2^5, that is, the exponent value is converted from 3 to 5; then, the mantissa of 1.0000*2^3 is shifted, that is, 1.0000 is converted to 0.0100; finally, 1.0000*2^5 and 0.0100*2^5 are added, and the final sum is 1.0100*2^5.

[0219] Example 10: According to the above examples 6 to 9, if Figure 8 As shown, when the multiplier accumulator can only be used to calculate operands with a precision type of FP16 or BF16, the four groups of exponential addition results are used as a reference, and four subtractors are used to calculate the differences between all the exponential addition results and the four groups of exponential addition results, which are delta0, delta1, delta2, and delta3. When the difference is other data other than 0, the exponents of other operands are first changed to the four groups of exponential addition results. Then, the right shift operation unit is used to perform a right shift operation on the four first mantissa multiplication results with a difference other than 0 according to the difference.

[0220] It should be noted that if the multiplier-accumulator can calculate both FP32 operands and FP16 and BF16 operands, then Fig. 9 As shown, in the above-mentioned hardware component for determining the order shift value, both a subtractor and a multiplexer will be included, but the multiplexer can receive the precision type parameter of the operand. When it is confirmed that the precision type of the operand is an FP16 or BF16 operand, the multiplexer is not enabled, and the order shift value is obtained by calculating the difference between the exponential addition result of each group of operands and the maximum exponent addition result through the subtractor; when it is confirmed that the precision type of the operand is an FP32 operand, the subtractor is not enabled, and the multiplexer is directly used to select the shift value corresponding to each first mantissa multiplication result in the first mantissa multiplication result based on the exponent of the multiplication result of the high-order parts of multiple operands.

[0221] In this way, by performing the order shift operation, when performing the subsequent addition operation on operands smaller than the preset mantissa data width parameter, the addition operation can be performed according to the preset addition operation logic, thereby improving the operation accuracy.

[0222] In one embodiment of the present application, the second shifting mode is described. When the mantissa data width parameter of the above-mentioned multiple operands is greater than the preset mantissa data width parameter, the first mantissa multiplication result is subjected to the shifting operation to generate the second mantissa multiplication result, including: taking the exponent of the multiplication result of the high-order parts of the above-mentioned multiple operands as a reference, selecting the shift value corresponding to each first mantissa multiplication result in the above-mentioned first mantissa multiplication result; according to the above-mentioned shift value, performing the shifting operation on each target mantissa multiplication result in the above-mentioned first mantissa multiplication result to generate the second mantissa multiplication result.

[0223] Exemplarily, the first mantissa multiplication result includes multiple mantissa multiplication results, and the second mantissa multiplication result includes multiple mantissa multiplication results.

[0224] In one example, the mantissa data width parameter of the above-mentioned multiple operands is greater than the preset mantissa data width parameter means that the mantissa data width parameter of each operand in the multiple operands is greater than the preset mantissa data width parameter. For example, when the multiple operands are operands of precision type FP32 and the preset mantissa data width parameter is 12 bits, the mantissa data width parameter of the FP32 operand is 24 bits (including 1 bit implicit bit).

[0225] Illustratively, the high-order parts of the above-mentioned multiple operands can refer to the above description and will not be repeated here.

[0226] Exemplarily, the shift value may be an alignment value required to perform an alignment operation.

[0227] In an example, the alignment value may be a shift value.

[0228] For example, from the above description of the rank shift operation, it can be seen that before the shift, a rank standard needs to be selected first.

[0229] Furthermore, the exponent criterion may be an exponent of a multiplication result of high-order parts of a plurality of operands.

[0230] It is understandable that, since the mantissa data width parameter of multiple operands is greater than the preset mantissa data width parameter, the process of performing the multiplication operation is actually to obtain the first mantissa multiplication result by splitting each operand in the multiple operands and then cross-multiplying them. In this way, the first mantissa multiplication result obtained by cross-multiplication is actually likely to be obtained by splitting the same operand and multiplying it, so that the exponential value of the exponent corresponding to the first mantissa multiplication result is the same. On this basis, for multiple operands whose mantissa data width parameter is greater than the preset mantissa data width parameter, it is necessary to use the exponent of the multiplication result of the high-order part as the reference, and then select the alignment value corresponding to the alignment operation according to the exponent of the multiplication result.

[0231] It should be noted that the above-mentioned alignment operation method, which takes the exponent of the multiplication result of the high-order part as the basis and then selects the alignment value corresponding to the alignment operation according to the exponent of the multiplication result, is different from the alignment operation method of the first mantissa multiplication result of multiple operands whose mantissa data width parameter is smaller than the preset mantissa data width parameter, and needs to be distinguished.

[0232] Exemplarily, the shift value corresponding to each of the first mantissa multiplication results may be selected by a selector. In an example, the selector may be a multiplexer mux.

[0233] Example 11: Figure 6 As shown, combined with the above example 5, for a multiplication accumulator, assuming that the multiple operands are a group of FP32 operands, that is, 2 operands, namely operand A and operand B. It can be seen from the above example 5 that the multiplication results of operand A and operand B are the first multiplication results of the multiplication operations of operand B_L and operand A_L, the second multiplication results of the multiplication operations of operand A_H and operand B_L, the third multiplication results of the multiplication operations of operand B_H and operand A_L, and the fourth multiplication results of the multiplication operations of operand A_H and operand B_H, that is, 4 multiplication results are obtained.

[0234] Afterwards, combined Figure 6 and Fig.10, the exponent of the multiplication result of the multiplication operation of the operand A_H and the operand B_H (i.e., the exponent of the multiplication result of the high-order parts of the above-mentioned multiple operands) is used as the basis for the above-mentioned multiple mantissa multiplication results to shift the above-mentioned 4 multiplication results, and the precision type is selected by mux, and the rank shift value of the 4 multiplication results is adaptively selected, wherein the rank shift value of the first multiplication result is the shift value a, i.e., 24 bits; the rank shift value of the second multiplication result is the shift value b, i.e., 12 bits; the rank shift value of the third multiplication result is the shift value b, i.e., 12 bits; the rank shift value of the fourth multiplication result is 0 bits, and the rank shift values ​​corresponding to each multiplication result are output, delta0, delta1, delta2, delta3. Then, the rank shift operation unit is used to shift the above-mentioned 4 multiplication results in the subsequent steps to obtain the first final multiplication result, the second final multiplication result, the third final multiplication result, and the fourth final multiplication result (i.e., the second mantissa multiplication result).

[0235] It should be noted that if the multiplier-accumulator can calculate both FP32 operands and FP16 and BF16 operands, then Fig. 9 As shown, in the above-mentioned hardware component for determining the order shift value, both a subtractor and a multiplexer will be included, but the multiplexer can receive the precision type parameter of the operand. When it is confirmed that the precision type of the operand is an FP16 or BF16 operand, the multiplexer is not enabled, and the order shift value is obtained by calculating the difference between the exponential addition result of each group of operands and the maximum exponent addition result through the subtractor; when it is confirmed that the precision type of the operand is an FP32 operand, the subtractor is not enabled, and the multiplexer is directly used to select the shift value corresponding to each first mantissa multiplication result in the first mantissa multiplication result based on the exponent of the multiplication result of the high-order parts of multiple operands.

[0236] In this way, by performing the order shift operation, when the subsequent operands greater than the preset mantissa data width parameter are added, the addition operation can be performed according to the preset addition operation logic, thereby improving the operation accuracy.

[0237] In one embodiment of the present application, a specific addition operation process will be introduced. Specifically, when the second mantissa multiplication result includes multiple carry results and multiple sum results, the above-mentioned addition operation is performed on the second mantissa multiplication result to generate a mantissa addition result, including: performing a first addition operation on the multiple carry results and the multiple sum results to generate a target carry result and a target sum result; performing a second addition operation on the target carry result and the target sum result to generate a mantissa addition result.

[0238] It is understandable that:

[0239] 1) Since the first mantissa multiplication result is the multiplication result corresponding to multiple operands, the first mantissa multiplication result is also a plurality of mantissa multiplication results, and the second mantissa multiplication result is obtained after the first mantissa multiplication result is aligned, therefore, the number of the second mantissa multiplication results is the same as the number of the first mantissa multiplication results.

[0240] 2) The first mantissa multiplication result may be a plurality of carry results (carry) and a plurality of sum results (sum), and correspondingly, the second mantissa multiplication result may also be a plurality of carry results (carry) after performing an alignment operation and a plurality of sum results (sum) after performing an alignment operation.

[0241] Exemplarily, the first addition operation is used to operate on all the second mantissa addition results obtained above to obtain a target carry result and a target sum result. For example, assuming that all the second mantissa addition results obtained above are 4 carry results and 4 sum results, after the first addition operation, 1 target mantissa result and 1 target sum result will be obtained.

[0242] It can be understood that the multiple carry results, the multiple sum results, the target carry result and the target sum result are all addition results with respect to the mantissa.

[0243] In an example, the first addition operation may be performed by a carry save adder (CSA), and the first addition operation may be a carry save addition.

[0244] Exemplarily, the second addition operation is used to perform an addition operation on the target carry result and the target sum result to generate a final mantissa addition result. For example, assuming that the aforementioned steps obtain 1 target carry result and 1 target sum result, after the second addition operation, 1 mantissa addition result will be obtained.

[0245] In an example, the second addition operation may be performed by a mantissa adding unit (adder), and the first addition operation may be a final mantissa addition operation.

[0246] Example 12: In combination with the above Example 11, assuming that the precision type of the current multi-operand is FP16 or BF16, for a multiplication accumulator, when obtaining the 4 second mantissa multiplication results corresponding to the 4 first mantissa multiplication results, each second mantissa multiplication result also includes a carry result and a sum result. That is, the 4 second mantissa multiplication results include: carry result 1 and sum result 1, carry result 2 and sum result 2, carry result 3 and sum result 3, carry result 4 and sum result 4, that is, a total of 8 addition result data. In this case, the above 8 addition result data are input into the CSA, and the CSA is used to perform a carry-retaining adder to perform an addition operation on the above 8 addition result data to obtain 1 target carry result and 1 target sum result.

[0247] Afterwards, the above-mentioned 1 target carry result and 1 target sum result are input into adder, and the addition operation is performed to finally generate the mantissa addition result.

[0248] Example 13: In combination with the above-mentioned Example 11, assuming that the precision type of the current multi-operand is FP32, for a multiplication accumulator, after obtaining the first final multiplication result, the second final multiplication result, the third final multiplication result and the fourth final multiplication result (i.e., the second mantissa multiplication result mentioned above), each final multiplication result also includes a carry result and a sum result. That is, the above-mentioned 4 final multiplication results include: carry result 5 and sum result 5, carry result 6 and sum result 6, carry result 7 and sum result 7, carry result 8 and sum result 8, that is, a total of 8 addition result data. In this case, the above-mentioned 8 addition result data are input into the CSA, and the CSA is used to perform a carry-preserving adder to perform an addition operation on the above-mentioned 8 addition result data to obtain 1 target carry result and 1 target sum result.

[0249] Afterwards, the above-mentioned 1 target carry result and 1 target sum result are input into adder, and the addition operation is performed to finally generate the mantissa addition result.

[0250] Thus, it can be seen that after executing the aforementioned preprocessing operation, in the above-mentioned addition operation, there is essentially no need to select the addition operation logic according to the different types of operand precision, which greatly reduces the use of multiplexers and does not require instantiation of adders. This greatly increases the computational efficiency of the multiplier-accumulator and also reduces the area occupied by the multiplier-accumulator on the chip.

[0251] In one embodiment of the present application, after performing the addition operation, it is also necessary to perform a normalized shift on the result of the addition operation, so as to change the format of the addition result into an operand that can be actually used. Therefore, the above-mentioned second addition operation is performed on the above-mentioned target carry result and the above-mentioned target sum result to generate the mantissa addition result, including: performing the above-mentioned second addition operation on the above-mentioned target carry result and the above-mentioned target sum result, and performing a leading zero prediction on the above-mentioned target carry result and the above-mentioned target sum result; generating the above-mentioned third mantissa addition result and the leading zero prediction result; performing a normalized shift on the above-mentioned third mantissa addition result according to the above-mentioned leading zero prediction result to generate the above-mentioned mantissa addition result; adjusting the exponent corresponding to the above-mentioned third mantissa addition result according to the shift value of the above-mentioned normalized shift to generate the target exponent.

[0252] It is understandable that after obtaining the target carry result and the target sum result, if it is necessary to finally obtain operands that can directly participate in subsequent operations (for example, operands in floating-point output format), normalization shifting is also required.

[0253] Exemplarily, the shift amount of the normalized shift may be preset in advance by the multiplier-accumulator, or may be customized by a user.

[0254] In one example, when the shift amount of the normalized shift is preset in advance for the multiplier accumulator, the leading zero prediction result can be used to assist in confirming the shift amount of the normalized shift of the target carry result and the target sum result.

[0255] Furthermore, the leading-zero prediction may be performed by a leading-zero anticipator (LZA).

[0256] In an example, the normalizing shift performed on the third mantissa addition result according to the leading zero prediction result may be performed by a normalizing shifter.

[0257] In an example, the third mantissa addition result may be obtained by a mantissa adding unit (ie, adder) performing an addition operation.

[0258] In an example, the adjustment of the exponent corresponding to the third mantissa addition result may be performed by an exponent adjustment unit (exponentadjust, exp_adj).

[0259] Exemplarily, the target index is an index adjusted according to the shift value.

[0260] Example 14: In combination with the above-mentioned Example 12 or Example 13, the above-mentioned 1 target carry result and 1 target sum result are input into adder, and while the addition operation is performed, the above-mentioned 1 target carry result and 1 target sum result are input into LZA. Afterwards, after the third mantissa addition result and the leading zero prediction result are output from adder, the normalization shift unit performs a shift operation on the third mantissa addition result according to the leading zero prediction result, and outputs the mantissa addition result in a standard floating-point output format. At the same time, according to the shift value of the above-mentioned shift operation, the exponent of the third mantissa addition result also needs to be adjusted. For example, if the shift value of the third mantissa addition result is 2 bits shifted to the left, the exponent needs to be subtracted by 2, and if the mantissa is 2 bits shifted to the right, the exponent needs to be added by 2.

[0261] In one embodiment of the present application, during the execution of the above multiplication and accumulation operation, a D-type flip-flop can be used for beat processing to reduce the delay time. Specifically, the above-mentioned D-type flip-flop can be used in many processes in which the multiplication and accumulation unit performs multiplication and addition operations.

[0262] In one example, the D trigger may be used after the aforementioned “performing a multiplication operation on the multiple target mantissas to obtain a first mantissa multiplication result of the multiple target mantissas”.

[0263] In an example, the D-type flip-flop may be used after the aforementioned “performing a first addition operation on the multiple carry results and the multiple sum results to generate a target carry result and a target sum result”.

[0264] In an example, the above-mentioned D flip-flop can be used after "performing a normalization shift on the third mantissa addition result according to the above-mentioned leading zero prediction result to generate the mantissa addition result".

[0265] In one example, the D flip-flop may be used to “adjust the exponent corresponding to the third mantissa addition result according to the shift value of the normalized shift to generate a target exponent.”

[0266] In summary, the method provided in the embodiments of the present application can specifically achieve the following beneficial effects:

[0267] 1) For the multiplication and accumulation device in the present application, according to the difference of the multiplication and accumulation operation of the operands of different precision types, the operands of different precision types are normalized in advance (for example, the normalization of the mantissa data width parameter and the normalization of the exponent data width parameter), so that in the process of performing the multiplication and accumulation operation of the operands, in the multiplication and accumulation operation process, the scenario of using a large number of multiplexers to continuously select the multiplication and accumulation operation results of the operands of different precision types is reduced. Specifically, in the process of processing the mantissa of the operands, there is no need to introduce a multiplexer, but when the mantissa data width parameter of multiple operands is greater than the preset mantissa data width parameter, it is necessary to use a selector on the shift value of the alignment operation (select the shift value of each first mantissa multiplication result according to the exponent of the multiplication result of the high-order parts of the multiple operands), so that the area of ​​the chip occupied by the multiplexer is greatly reduced.

[0268] 2) Since the use of multiplexers is reduced, the overall computing time is greatly shortened and the computing efficiency is greatly improved.

[0269] 3) Since operands of different precision types are normalized in advance, that is, the mantissa data width parameter and the exponent data width parameter are unified, no additional processing logic is required to process operands of different precision types in subsequent multiplication and accumulation operations.

[0270] 4) Since the operands of different precision types are normalized in advance, that is, the mantissa data width parameter and the exponent data width parameter are unified, it only takes a small amount of resources to support multiple precisions, that is, it realizes the operation of operands of multiple precision types on the same multiplication and accumulation device at a very small area cost, and the chip surface efficiency is greatly improved. Specifically, in the actual experiment, the area of ​​the prior art is 1900um^2 and the area of ​​the present invention is 1300um^2 through the synthesis tool designcompiler. It can be seen that the area of ​​the present invention is about 68% of the prior art.

[0271] The following will be based on the above scheme. Figure 2 , describes the complete execution process of the data processing method for multiple operands of three different precision types.

[0272] It should be noted that the following complete execution process is based on a parallel multiplexing number of 4, that is, 4 multiplication accumulators (i.e., 4 processing engines (PE)) in the multiplication array are grouped as a group, the preset mantissa data width parameter is 12 bits, and the preset mantissa data width parameter is 8 bits, which describes the multiplication and accumulation operations on multiple operands.

[0273] 1) The first precision type: BF16

[0274] When the precision type is BF16 and the number of parallel multiplexing of the computing units of the multiplication and accumulation device is 4, 4 computing units can perform multiplication and accumulation operations on 8 operands of the precision type BF16, that is, 4 groups of operands.

[0275] Step A1: According to the above-mentioned preset mantissa data width parameter, 12 bits, the mantissas of multiple operands are normalized, the mantissa data width parameter of the BF16 operand is expanded from 8 bits to 12 bits, and 8 target mantissas (i.e., 4 groups of target mantissas) are obtained, and the mantissa data width parameter of the target mantissas is 12 bits.

[0276] Step A2: Perform a multiplication operation on each of the four groups of target mantissas. Specifically, first perform a partial product generation operation to obtain four groups of partial products, each group of partial products including seven partial products; then, perform a partial product compression operation on the four groups of partial products, compress the sum of each of the seven partial products into two compression results, which are a carry result and a sum result, respectively, and generate a total of eight compression results, which are four first carry results and four second sum results.

[0277] Step A3: Use the 4 carry results and the 4 sum results to trigger the D flip-flop (dff) to perform beat processing.

[0278] Step A4: perform exponential addition operation on the exponents of the above four groups of operands to obtain four exponential addition results.

[0279] Step A5: Compare the four exponential addition results and select the largest exponential addition result among the four exponential addition results.

[0280] Step A6: Using the maximum exponent addition result, convert the exponent values ​​of the exponents of the other three groups of operands into the maximum exponent addition result.

[0281] Step A7: Obtain four differences between the maximum exponent addition result and the exponent addition results of the four groups of operands.

[0282] Step A8: According to the above 4 differences, perform a shift operation on the above 4 first carry results and the 4 first sum results to obtain 4 second carry results and 4 second sum results.

[0283] Step A9: Perform a carry-preserving addition operation on the above 4 second carry results and the 4 second sum results to obtain 1 third carry result and 1 third sum result.

[0284] Step A10: Use the 4 carry results and the 4 sum results to trigger the D flip-flop (dff) to perform beat processing.

[0285] Step A11: perform an addition operation on the above-mentioned 1 third carry result and 1 third sum result, and simultaneously perform a leading zero prediction on the 1 third carry result and the 1 third sum result to obtain an addition result and a leading zero prediction result.

[0286] Step A12: Use the leading zero prediction result to perform normalization shift on the addition result.

[0287] Step A13: According to the shift value of the normalized shift, the exponents of the four groups of operands corresponding to the addition result are adjusted by corresponding exponent shift, so as to obtain the addition result (including the mantissa addition result and the exponent addition result) in a standard floating-point output format.

[0288] Step A14: Use the result of the mantissa addition to trigger the D flip-flop to perform beat processing.

[0289] Step A15: Use the exponential addition result to trigger the D flip-flop to perform beat processing.

[0290] Step A16: Match the mantissa addition result with the exponent addition result, and combine and output the multiplication and accumulation operation results of the above multiple operands.

[0291] 2) The second precision type: FP16

[0292] When the precision type is FP16 and the number of parallel multiplexing of the computing units of the multiplication and accumulation device is 4, the 4 computing units can perform multiplication and accumulation operations on 8 operands of the precision type FP16, that is, 4 groups of operands.

[0293] Step B1: According to the above-mentioned preset mantissa data width parameter, 12 bits, the mantissas of multiple operands are normalized, the mantissa data width parameter of the FP16 operand is expanded from 11 bits to 12 bits, and 8 target mantissas (i.e., 4 groups of target mantissas) are obtained, and the mantissa data width parameter of the target mantissas is 12 bits.

[0294] Step B2: Perform a multiplication operation on each of the four groups of target mantissas. Specifically, first perform a partial product generation operation to obtain four groups of partial products, each group of partial products including seven partial products; then, perform a partial product compression operation on the four groups of partial products, compress the sum of each of the seven partial products into two compression results, which are a carry result and a sum result, respectively, and generate a total of eight compression results, which are four first carry results and four second sum results.

[0295] Step B3: Use the 4 carry results and the 4 sum results to trigger the D flip-flop (dff) to perform beat processing.

[0296] Step B4: Compare the exponent data width parameters of the exponents of the above four groups of operands with the preset exponent data width parameter 8 bits.

[0297] Step B5: Expand the exponent data width parameters of the exponents of the above four groups of operands to 8 bits.

[0298] Step B6: Perform exponential addition operation on the exponents of the above four groups of operands to obtain four exponential addition results.

[0299] Step B7: Compare the four exponential addition results and select the largest exponential addition result among the four exponential addition results.

[0300] Step B8: Using the maximum exponent addition result, convert the exponent values ​​of the exponents of the other three groups of operands into the maximum exponent addition result.

[0301] Step B9: Obtain four differences between the maximum exponent addition result and the exponent addition results of the four groups of operands.

[0302] Step B10: According to the above 4 differences, perform a shift operation on the above 4 first carry results and the 4 first sum results to obtain 4 second carry results and 4 second sum results.

[0303] Step B11: perform a carry-preserving addition operation on the above 4 second carry results and 4 second sum results to obtain 1 third carry result and 1 third sum result.

[0304] Step B12: Use the 4 carry results and the 4 sum results to trigger the D flip-flop (dff) to perform beat processing.

[0305] Step B13: performing an addition operation on the above-mentioned 1 third carry result and 1 third sum result, and simultaneously performing a leading zero prediction on the 1 third carry result and the 1 third sum result to obtain an addition result and a leading zero prediction result.

[0306] Step B14: Use the leading zero prediction result to perform normalization shift on the addition result.

[0307] Step B15: According to the shift value of the normalized shift, the exponents of the four groups of operands corresponding to the addition result are adjusted by corresponding exponent shift, so as to obtain the addition result (including the mantissa addition result and the exponent addition result) in a standard floating-point output format.

[0308] Step B16: Use the result of the mantissa addition to trigger the D flip-flop to perform beat processing.

[0309] Step B17: Use the exponential addition result to trigger the D flip-flop to perform beat processing.

[0310] Step B18: Match the mantissa addition result with the exponent addition result, and combine and output the multiplication and accumulation operation results of the above multiple operands.

[0311] 3) The third precision type: FP32

[0312] When the precision type is FP32 and the number of parallel multiplexing of the computing units of the multiplication and accumulation device is 4, the 4 computing units can perform multiplication and accumulation operations on 2 operands of the precision type FP32, that is, 1 group of operands.

[0313] Step C1: According to the above-mentioned preset mantissa data width parameter, 12 bits, the mantissas of multiple operands are normalized, and each operand in a group of FP32 operands is split into a high-order part and a low-order part, and the mantissa data width parameter of the high-order part and the mantissa data width parameter of the low-order part are both 12 bits. The split mantissa parts are cross-combined to obtain a total of 4 groups of target mantissas, and the mantissa data width parameter of the target mantissa is 12 bits.

[0314] Step C2: Perform a multiplication operation on each of the four groups of target mantissas. Specifically, first perform a partial product generation operation to obtain four groups of partial products, each group of partial products including seven partial products; then, perform a partial product compression operation on the four groups of partial products, compress the sum of each of the seven partial products into two compression results, which are a carry result and a sum result, respectively, and generate a total of eight compression results, which are four first carry results and four second sum results.

[0315] Step C3: Use the 4 carry results and the 4 sum results to trigger the D flip-flop (dff) to perform beat processing.

[0316] Step C4: Based on the exponent of the multiplication result of the high-order parts of the two operands, select the shift values ​​corresponding to the four first carry results and the shift values ​​corresponding to the four second sum results.

[0317] Step C5: according to the shift values ​​corresponding to the above 4 first carry results and the shift values ​​corresponding to the 4 second sum results, perform a shift operation on the above 4 first carry results and the 4 first sum results to obtain 4 second carry results and 4 second sum results.

[0318] Step C6: Perform a carry-preserving addition operation on the above 4 second carry results and the 4 second sum results to obtain 1 third carry result and 1 third sum result.

[0319] Step C7: Use the 4 carry results and the 4 sum results to trigger the D flip-flop (dff) to perform beat processing.

[0320] Step C8: performing an addition operation on the above-mentioned 1 third carry result and 1 third sum result, and simultaneously performing a leading zero prediction on the 1 third carry result and the 1 third sum result to obtain an addition result and a leading zero prediction result.

[0321] Step C9: Use the leading zero prediction result to perform normalization shift on the addition result.

[0322] Step C10: According to the shift value of the normalized shift, the exponents of the four groups of operands corresponding to the addition result are adjusted by corresponding exponent shift, so as to obtain the addition result (including the mantissa addition result and the exponent addition result) in a standard floating-point output format.

[0323] Step C11: Use the result of the mantissa addition to trigger the D flip-flop to perform the beat processing.

[0324] Step C12: Use the exponential addition result to trigger the D trigger to perform beat processing.

[0325] Step C13: Match the mantissa addition result with the exponent addition result, and combine and output the multiplication and accumulation operation results of the above multiple operands.

[0326] Fig.11 A schematic diagram of the structure of a multiplier-accumulator according to an embodiment of the present application.

[0327] like Fig.11 As shown, the multiplier accumulator 1100 includes: a mantissa pre-splitting unit 1101, a multiplication operation unit 1102, an alignment operation unit 1103 and an addition operation unit 1104.

[0328] A mantissa pre-splitting unit 1101 is used to obtain a plurality of operands according to the number of parallel multiplexing of the computing units of the multiplier-accumulator and the target precision type, wherein the operands include a mantissa part and an exponent part;

[0329] The mantissa pre-splitting unit 1101 is further used to change the mantissas of the plurality of operands into a plurality of target mantissas according to the target precision type and a preset mantissa data width parameter, wherein the preset mantissa data width parameter is related to the computing power of the computing unit, and the width of the target mantissa is the preset mantissa data width parameter;

[0330] A multiplication operation unit 1102 is used to perform a multiplication operation on the multiple target mantissas changed by the mantissa pre-splitting unit 1101 to obtain a first mantissa multiplication result of the multiple target mantissas;

[0331] An alignment operation unit 1103 is used to perform an alignment operation on the first mantissa multiplication result obtained by the multiplication operation unit 1102 according to the exponents of the multiple operands to generate a second mantissa multiplication result; the state of the second mantissa multiplication result is an alignment state that can be used to perform operand addition operation;

[0332] The addition operation unit 1104 is used to perform an addition operation on the second mantissa multiplication result generated by the alignment operation unit 1103 to generate a mantissa addition result, match the mantissa addition result with the exponents corresponding to the multiple operands, and output the multiplication and accumulation operation results of the multiple operands.

[0333] In one embodiment, the mantissa pre-splitting unit 1101 is specifically used for:

[0334] According to the above target precision type, determine the mantissa data width parameter of the above operand;

[0335] The mantissa data width parameter of the above-mentioned operand is changed according to the above-mentioned preset mantissa data width parameter to obtain the above-mentioned multiple target mantissas.

[0336] In one embodiment, the mantissa pre-splitting unit 1101 is specifically used for:

[0337] In the case where the mantissa data width parameters of the plurality of operands are smaller than the preset mantissa data width parameters, the mantissa data width parameters of the plurality of operands are expanded to the preset mantissa data width parameters to generate the plurality of target mantissas;

[0338] In the case where the mantissa data width parameters of the plurality of operands are greater than the preset mantissa data width parameter, splitting the plurality of operands into a high-order part and a low-order part to generate the plurality of target mantissas;

[0339] Among them, the above-mentioned high-order part includes a high-order mantissa, the above-mentioned low-order part includes a low-order mantissa, and the mantissa data width parameter of the above-mentioned high-order mantissa and the mantissa data width parameter of the above-mentioned low-order mantissa are equal to the above-mentioned preset mantissa data width parameter.

[0340] In one embodiment, the multiplication unit 1102 includes:

[0341] A partial product generating unit, used for performing a product generating operation on the above-mentioned multiple target mantissas to generate multiple product values;

[0342] The partial product compression unit is used to perform a product compression operation on the multiple product values ​​generated by the partial product generation unit to generate the first mantissa multiplication result.

[0343] In one embodiment, the multiplier-accumulator 1100 further includes:

[0344] An exponential processing unit, used for comparing the exponential data width parameters of the plurality of operands with a preset exponential data width parameter;

[0345] The exponential processing unit is further configured to change the exponential data width parameters of the plurality of operands to the preset exponential data width parameter when the exponential data width parameter of the operand is different from the preset exponential data width parameter.

[0346] In one embodiment, the alignment operation unit 1103 includes:

[0347] The rank shift operation unit is used to perform a rank shift operation on the first mantissa multiplication result to generate the second mantissa multiplication result.

[0348] In one embodiment, when the mantissa data width parameter of the operand is less than the preset mantissa data width parameter, the multiplier accumulator 1100 includes:

[0349] The exponential processing unit is further used to compare the multiple exponential addition results generated by the exponential processing unit and select the maximum value of the multiple exponential addition results;

[0350] The exponential processing unit is further used to convert the exponential values ​​of the exponents of the plurality of operands into the maximum value screened by the exponential processing unit;

[0351] The exponential processing unit is further used to obtain a plurality of differences between the maximum value of the plurality of exponential addition results screened by the comparator and the plurality of exponential addition results;

[0352] an order shift operation unit, configured to perform an order shift operation on the first mantissa multiplication results corresponding to the plurality of difference values ​​respectively according to the plurality of difference values ​​obtained by the exponent processing unit, so as to generate the second mantissa multiplication result;

[0353] Among them, the above-mentioned first mantissa multiplication result includes multiple mantissa multiplication results, the above-mentioned multiple differences correspond one-to-one to the multiple first mantissa multiplication results, and the above-mentioned second mantissa multiplication result includes multiple mantissa multiplication results.

[0354] In one embodiment, when the mantissa data width parameter of the operand is greater than a preset mantissa data width parameter, the multiplier accumulator 1100 includes:

[0355] an exponent processing unit, configured to select a shift value corresponding to each of the first mantissa multiplication results based on an exponent of a multiplication result of high-order parts of the plurality of operands, wherein the first mantissa multiplication result includes a plurality of mantissa multiplication results;

[0356] The order shift operation unit is used to perform an order shift operation on each target mantissa multiplication result in the first mantissa multiplication result according to the shift value selected by the exponent processing unit to generate the second mantissa multiplication result, wherein the second mantissa multiplication result includes multiple mantissa multiplication results.

[0357] In one embodiment, when the second mantissa multiplication result includes multiple carry results and multiple sum results, the multiplication accumulator 1100 includes:

[0358] A carry-retaining adder, used for performing a first addition operation on the plurality of carry results and the plurality of sum results to generate a target carry result and a target sum result;

[0359] The mantissa addition unit is used to perform a second addition operation on the target carry result and the target sum result generated by the carry-preserving adder to generate a mantissa addition result.

[0360] In one embodiment, the multiplication accumulator includes a leading zero prediction unit, a normalization shift unit, and an exponent adjustment unit:

[0361] Using the mantissa addition unit to perform the second addition operation on the target carry result and the target sum result, and using the leading zero prediction unit to perform leading zero prediction on the target carry result and the target sum result;

[0362] generating the third mantissa addition result by using the mantissa addition unit and generating the leading zero prediction result by using the leading zero prediction unit;

[0363] According to the leading zero prediction result, using the normalization shift unit to perform normalization shift on the third mantissa addition result to generate the mantissa addition result;

[0364] According to the shift value of the normalized shift, the exponent corresponding to the third mantissa addition result is adjusted by using the exponent adjustment unit to generate a target exponent.

[0365] The multiplication and accumulation device proposed in the embodiment of the present application takes into account that in the existing technical solutions, when using the multiplication and accumulation device to perform multiplication and accumulation operations on multiple operands, the multiplication operations of multiple operands are first completed through the multiplication unit to obtain multiple multiplication results; then, the multiplication results of the operands are selected by using multiple path selectors MUX; after that, the multiplication results are input into the instantiation addition tree that conforms to the precision type of the operands for calculation; finally, a MUX is required to select the final multiplication and accumulation result. The above-mentioned multiplication and accumulation operation mainly has the following two problems: First, due to the large number of operands, a large number of multiplication results will be obtained. Second, if you choose to fully configure the service, you cannot obtain the indicator data of the control group that is not affected by the service release after the full configuration of the service, so you cannot complete the long-term testing of the service.

[0366] Therefore, in the multiplier-accumulator proposed in the embodiment of the present application, the multiplier-accumulator will obtain multiple operands including a mantissa part and an exponent part according to the number of parallel multiplexing of its computing units and the target precision type, and then, according to the target precision type and a preset mantissa data width parameter (the preset mantissa data width parameter is related to the computing power of the computing unit), the mantissas of the multiple operands are changed to multiple target mantissas (the width of the target mantissa is the preset mantissa data width parameter); thereafter, a multiplication operation is performed on the multiple target mantissas to obtain a first mantissa multiplication result of multiple target bits; then, according to the exponents of the above-mentioned multiple operands, an alignment operation is performed on the first mantissa multiplication result to generate a second mantissa multiplication result (the state of the second mantissa multiplication result is an alignment state that can be used to perform operand addition operations), and then, an addition operation is performed on the second mantissa multiplication result to generate a mantissa addition result, the mantissa addition result is matched with the exponents corresponding to the multiple operands, and the multiplication-accumulation operation results of the multiple operands are output. In this way, the obtained multiple operands are pre-processed by normalization first (for example, the mantissa digits of operands of different precision types are normalized), and then the mantissa multiplication results of the multiple operands are aligned according to the exponents of the multiple operands by the multiplication accumulator before performing the addition operation, so that the multiplication accumulator can perform the addition operation later, and finally the mantissa multiplication result and the exponent are matched, and the multiplication and addition operation result is output. In actual operation, the number of operands to be processed is very large, and the steps of the above data processing method are completed by a multiplication and addition array composed of multiple multiplication and accumulators. On the one hand, according to the multiplication and addition operation logic of operands of different precision types, the mantissa placeholder format of operands of different precision types is modified to a mantissa placeholder format consistent with the calculation unit of the multiplication and accumulator, and the operands are pre-processed by normalization; on the other hand, according to the addition operation logic of operands of different precision types, different types of alignment operations are performed on the mantissa multiplication results of operands of different precision types, so as to accurately perform the addition operation. Therefore, there is no need to set up a large number of multiplexers to screen the multiplication results corresponding to operands of different precision types, nor is there a need to perform addition operations on operands of different precision types through addition tree instantiation. This greatly reduces the occupied area of ​​the multiplication and accumulator, and at the same time, can greatly improve the computational efficiency of the multiplication and accumulator with high computing power.

[0367] It should be understood that the units described in the multiplier-accumulator are similar to those described in the reference Figure 4 The various steps in the described method correspond to each other. Therefore, the operations and features described above for the method are also applicable to the multiplier accumulator and the units contained therein, and will not be repeated here. The multiplier accumulator can be pre-implemented in a browser or other security application of a computer device, or can be loaded into a browser or its security application of a computer device by downloading or the like. The corresponding units in the multiplier accumulator can cooperate with the units in the computer device to implement the solution of the embodiment of the present application.

[0368] The several modules or units mentioned in the above detailed description are not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0369] It should be noted that for details not disclosed in the multiplier-accumulator of the embodiment of the present application, please refer to the details disclosed in the above embodiments of the present application, which will not be repeated here.

[0370] Reference below Fig.12 , Fig.12 A schematic diagram of the structure of a computer device suitable for implementing the embodiments of the present application is shown. Fig.12 As shown, the computer system 1200 includes a central processing unit (CPU) 1201, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1202 or the program loaded from the storage part 1208 to the random access memory (RAM) 1203. In the RAM 1203, various programs and data required for the operation instructions of the system are also stored. The CPU 1201, the ROM 1202, and the RAM 1203 are connected to each other through a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0371] The following components are connected to the I / O interface 1205: an input section 1206 including a keyboard, a mouse, etc.; an output section 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN card, a modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the I / O interface 1205 as needed. A removable medium 1211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1210 as needed, so that a computer program read therefrom is installed into the storage section 1208 as needed.

[0372] In particular, according to an embodiment of the present application, the above reference flow chart Figure 4The described process can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flow chart. In such an embodiment, the computer program includes a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1209, and / or installed from the removable medium 1211. When the computer program is executed by the central processing unit (CPU) 1201, the above-mentioned functions defined in the system of the present application are executed.

[0373] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium such as a computer-readable storage medium that can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0374] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operating instructions of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the aforementioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, the boxes represented by two connections can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operating instruction, or can be implemented with a combination of dedicated hardware and computer instructions.

[0375] The units or modules involved in the embodiments described in the present application may be implemented by software or hardware. The units or modules described may also be configured in a processor, for example, may be described as: a processor includes an illegal person detection unit, a multimodal detection unit, and a recognition unit. The names of these units or modules do not, in some cases, constitute limitations on the units or modules themselves.

[0376] As another aspect, the present application further provides a computer-readable storage medium, which may be included in the computer device described in the above embodiment, or may exist independently without being installed in the computer device. The above computer-readable storage medium stores one or more programs, and when the above programs are used by one or more processors to execute the data processing method described in the present application. For example, it can be executed Figure 4 The various steps of the data processing method shown.

[0377] The present application embodiment provides a computer program product, which includes instructions. When the instructions are executed, the method described in the present application embodiment is executed. For example, Figure 3 The various steps of the data processing method shown.

[0378] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the aforementioned disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A data processing method, characterized in that: Applied to a multiplier-accumulator, the method comprises: According to the number of parallel multiplexing of computing units of the multiplier-accumulator and the target precision type, a plurality of operands are obtained, wherein the operands include a mantissa part and an exponent part; According to the target precision type and a preset mantissa data width parameter, the mantissas of the plurality of operands are changed into a plurality of target mantissas, the preset mantissa data width parameter being related to the computing power of the computing unit, and the width of the target mantissa being the preset mantissa data width parameter; Performing a multiplication operation on the multiple target mantissas to obtain a first mantissa multiplication result of the multiple target mantissas; performing an alignment operation on the first mantissa multiplication result according to the exponents of the plurality of operands to generate a second mantissa multiplication result; the state of the second mantissa multiplication result is an alignment state that can be used to perform an operand addition operation; An addition operation is performed on the second mantissa multiplication result to generate a mantissa addition result, the mantissa addition result is matched with the exponents corresponding to the multiple operands, and the multiplication and accumulation operation results of the multiple operands are output.

2. The method according to claim 1, characterized in that The step of changing the mantissas of the plurality of operands to a plurality of target mantissas according to the target precision type and a preset mantissa data width parameter comprises: Determining a mantissa data width parameter of the operand according to the target precision type; The mantissa data width parameter of the operand is changed according to the preset mantissa data width parameter to obtain the multiple target mantissas.

3. The method according to claim 2, characterized in that The mantissa data width parameter of the operand is changed according to a preset mantissa data width parameter to obtain the plurality of target mantissas, including: In the case where the mantissa data width parameters of the plurality of operands are smaller than the preset mantissa data width parameter, the mantissa data width parameters of the plurality of operands are expanded to the preset mantissa data width parameter to generate the plurality of target mantissas; In a case where the mantissa data width parameters of the plurality of operands are greater than the preset mantissa data width parameter, splitting the plurality of operands into a high-order part and a low-order part to generate the plurality of target mantissas; The high-order part includes a high-order mantissa, the low-order part includes a low-order mantissa, and the mantissa data width parameter of the high-order mantissa and the mantissa data width parameter of the low-order mantissa are equal to the preset mantissa data width parameter.

4. The method according to claim 1, characterized in that: The performing a multiplication operation on the multiple target mantissas to obtain a first mantissa multiplication result of the multiple target mantissas includes: Performing a product generation operation on the plurality of target mantissas to generate a plurality of product values; A product compression operation is performed on the plurality of product values ​​to generate the first mantissa multiplication result.

5. The method according to claim 1, characterized in that Before performing the alignment operation on the first mantissa multiplication result according to the exponents of the plurality of operands, the method further comprises: comparing the exponential data width parameters of the plurality of operands with a preset exponential data width parameter; When the exponential data width parameter of the operand is different from the preset exponential data width parameter, the exponential data width parameters of the plurality of operands are changed to the preset exponential data width parameter.

6. The method according to claim 1, characterized in that The performing an alignment operation on the first mantissa multiplication result to generate a second mantissa multiplication result includes: Performing an order shift operation on the first mantissa multiplication result to generate the second mantissa multiplication result.

7. The method according to any one of claims 1 to 6, characterized in that: In a case where the mantissa data width parameter of the operand is smaller than a preset mantissa data width parameter, performing an alignment operation on the first mantissa multiplication result to generate a second mantissa multiplication result includes: performing an exponential addition operation on the exponent of each operand in the plurality of operands to generate a plurality of exponential addition results; Comparing the multiple exponential addition results, and selecting the maximum value of the multiple exponential addition results; converting exponent values ​​of the exponents of the plurality of operands to the maximum value; Obtaining multiple differences between a maximum value among the multiple exponential addition results and the multiple exponential addition results; According to the multiple difference values, respectively perform an order shift operation on the first mantissa multiplication results corresponding to the multiple difference values ​​to generate the second mantissa multiplication result; The first mantissa multiplication result includes multiple mantissa multiplication results, the multiple differences correspond one-to-one to the multiple first mantissa multiplication results, and the second mantissa multiplication result includes multiple mantissa multiplication results.

8. The method according to any one of claims 1 to 6, characterized in that: In a case where the mantissa data width parameters of the plurality of operands are greater than a preset mantissa data width parameter, performing an alignment operation on the first mantissa multiplication result to generate the second mantissa multiplication result, comprising: Based on the exponent of the multiplication result of the high-order parts of the multiple operands, select a shift value corresponding to each first mantissa multiplication result in the first mantissa multiplication results, wherein the first mantissa multiplication result includes multiple mantissa multiplication results; According to the shift value, a shift operation is performed on each target mantissa multiplication result in the first mantissa multiplication result to generate the second mantissa multiplication result, where the second mantissa multiplication result includes a plurality of mantissa multiplication results.

9. The method according to claim 1, characterized in that: In a case where the second mantissa multiplication result includes a plurality of carry results and a plurality of sum results, performing an addition operation on the second mantissa multiplication result to generate a mantissa addition result includes: performing a first addition operation on the plurality of carry results and the plurality of sum results to generate a target carry result and a target sum result; A second addition operation is performed on the target carry result and the target sum result to generate a mantissa addition result.

10. The method according to claim 9, characterized in that The step of performing a second addition operation on the target carry result and the target sum result to generate a mantissa addition result comprises: performing the second addition operation on the target carry result and the target sum result, and performing leading zero prediction on the target carry result and the target sum result; generating a third mantissa addition result and a leading zero prediction result; According to the leading zero prediction result, performing a normalization shift on the third mantissa addition result to generate the mantissa addition result; According to the shift value of the normalized shift, the exponent corresponding to the third mantissa addition result is adjusted to generate a target exponent.

11. A multiplier-accumulator, characterized in that: The multiply-accumulate device comprises: A mantissa pre-splitting unit, used to obtain a plurality of operands according to the number of parallel multiplexing of the computing units of the multiplier-accumulator and the target precision type, wherein the operands include a mantissa part and an exponent part; The mantissa pre-splitting unit is further used to change the mantissas of the multiple operands into multiple target mantissas according to the target precision type and a preset mantissa data width parameter, wherein the preset mantissa data width parameter is related to the computing power of the computing unit, and the width of the target mantissa is the preset mantissa data width parameter; A multiplication operation unit, configured to perform a multiplication operation on the multiple target mantissas changed by the mantissa pre-splitting unit, and obtain a first mantissa multiplication result of the multiple target mantissas; an alignment operation unit, configured to perform an alignment operation on the first mantissa multiplication result obtained by the multiplication operation unit according to the exponents of the plurality of operands, to generate a second mantissa multiplication result; the state of the second mantissa multiplication result is an alignment state that can be used to perform an operand addition operation; An addition operation unit is used to perform an addition operation on the second mantissa multiplication result generated by the alignment operation unit to generate a mantissa addition result, match the mantissa addition result with the exponents corresponding to the multiple operands, and output the multiplication and accumulation operation results of the multiple operands.

12. The multiplier-accumulator according to claim 11, characterized in that: The mantissa pre-splitting unit is specifically used for: Determining a mantissa data width parameter of the operand according to the target precision type; The mantissa data width parameter of the operand is changed according to the preset mantissa data width parameter to obtain the multiple target mantissas.

13. The multiplier-accumulator according to claim 12, characterized in that: The mantissa pre-splitting unit is specifically used for: In the case where the mantissa data width parameters of the plurality of operands are smaller than the preset mantissa data width parameter, the mantissa data width parameters of the plurality of operands are expanded to the preset mantissa data width parameter to generate the plurality of target mantissas; In a case where the mantissa data width parameters of the plurality of operands are greater than the preset mantissa data width parameter, splitting the plurality of operands into a high-order part and a low-order part to generate the plurality of target mantissas; The high-order part includes a high-order mantissa, the low-order part includes a low-order mantissa, and the mantissa data width parameter of the high-order mantissa and the mantissa data width parameter of the low-order mantissa are equal to the preset mantissa data width parameter.

14. The multiplier-accumulator according to claim 11, characterized in that: The multiplication operation unit comprises: A partial product generating unit, configured to perform a product generating operation on the plurality of target mantissas to generate a plurality of product values; The partial product compression unit is used to perform a product compression operation on the multiple product values ​​generated by the partial product generation unit to generate the first mantissa multiplication result.

15. The multiplier-accumulator according to claim 11, characterized in that: The multiply-accumulate device further comprises: an exponent processing unit, configured to compare the exponent data width parameters of the plurality of operands with a preset exponent data width parameter; The exponent processing unit is further configured to change the exponent data width parameters of the plurality of operands to the preset exponent data width parameter when the exponent data width parameter of the operand is different from the preset exponent data width parameter.

16. The multiplier-accumulator according to claim 11, characterized in that: The alignment operation unit comprises: The rank shift operation unit is used to perform a rank shift operation on the first mantissa multiplication result to generate the second mantissa multiplication result.

17. The multiplier-accumulator according to any one of claims 11 to 16, characterized in that: In the case where the mantissa data width parameter of the operand is less than a preset mantissa data width parameter, the multiplier accumulator comprises: an exponential processing unit, configured to perform an exponential addition operation on the exponent of each operand of the plurality of operands to generate a plurality of exponential addition results; The exponential processing unit is further used to compare the multiple exponential addition results generated by the exponential processing unit and filter out the maximum value of the multiple exponential addition results; The exponential processing unit is further used to convert the exponential values ​​of the exponents of the multiple operands into the maximum value screened by the exponential processing unit; The exponential processing unit is further used to obtain a plurality of differences between a maximum value among the plurality of exponential addition results screened by the comparator and the plurality of exponential addition results; an order shift operation unit, configured to perform an order shift operation on the first mantissa multiplication results corresponding to the plurality of difference values ​​respectively according to the plurality of difference values ​​obtained by the exponent processing unit, so as to generate the second mantissa multiplication result; The first mantissa multiplication result includes multiple mantissa multiplication results, the multiple differences correspond one-to-one to the multiple first mantissa multiplication results, and the second mantissa multiplication result includes multiple mantissa multiplication results.

18. The multiplier-accumulator according to any one of claims 11 to 16, characterized in that: In the case where the mantissa data width parameter of the operand is greater than a preset mantissa data width parameter, the multiplier accumulator comprises: an exponent processing unit, configured to select a shift value corresponding to each of the first mantissa multiplication results based on an exponent of a multiplication result of high-order parts of the plurality of operands, wherein the first mantissa multiplication result includes a plurality of mantissa multiplication results; The exponent shift operation unit is used to perform an exponent shift operation on each target mantissa multiplication result in the first mantissa multiplication result according to the shift value selected by the exponent processing unit to generate the second mantissa multiplication result, wherein the second mantissa multiplication result includes multiple mantissa multiplication results.

19. The multiplier-accumulator according to claim 11, characterized in that: In the case where the second mantissa multiplication result includes a plurality of carry results and a plurality of sum results, the multiplication accumulator comprises: a carry-retaining adder, configured to perform a first addition operation on the plurality of carry results and the plurality of sum results to generate a target carry result and a target sum result; The mantissa addition unit is used to perform a second addition operation on the target carry result and the target sum result generated by the carry-preserving adder to generate a mantissa addition result.

20. A chip, comprising the multiplier-accumulator according to any one of claims 11 to 19, wherein the multiplier-accumulator is used to implement the data processing method according to any one of claims 1 to 10.

Citation Information

Cited By

  • Signal processing method and device, equipment, storage medium and program product

    CN122268711A