Multi-data-type multiplier and method for realizing multiplication of multiple types of data
By combining the basic multiplication units and selecting the control unit of the multiplier, the problem of wasted area and power consumption in traditional multiplier design is solved, and efficient multiplication operations of various data types are realized.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-01
- Publication Date
- 2026-05-15
AI Technical Summary
Traditional multiplier designs require each multiplier to handle multiple data types separately, resulting in wasted integrated circuit area and power consumption.
A multi-data-type multiplier is employed, which combines basic multiplication units of different data types through a combination of basic multiplication units and a control unit to select combinations of basic multiplication units of different data types, decomposes the input matrix and performs operations, and combines shift and accumulation operations to realize multiplication operations of multiple data types.
It improves the reusability of multipliers, saves integrated circuit area and power consumption, and is suitable for data operations of various data types.
Smart Images
Figure CN122047150A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of digital integrated circuit technology. In particular, it relates to a multi-data type multiplier and a method for multiplying multiple types of data. Background Technology
[0002] In Matrix Multiply-Add (MMA) designs of integrated circuits such as GPGPUs (General-Purpose computing on Graphics Processing Units), supporting matrix multiplication for multiple data types is crucial. For example, deep learning algorithms running on GPUs require a large number of matrix multiplications and dot products during training and inference. These operations pose challenges to multiplier design because different data types require different multipliers. Traditional multiplier designs require separate designs for each data type, which means additional area on the integrated circuit (chip) is needed to accommodate these multipliers, resulting in wasted area and power consumption. Summary of the Invention
[0003] In view of this, embodiments of the present invention provide a multi-data type multiplier and a method for implementing multiplication of multiple types of data, which facilitates the implementation of a multiplier that supports data operations of multiple data types at a lower cost, thereby saving integrated circuit area and power consumption resources.
[0004] To achieve the above-mentioned objectives, the following technical solution is adopted: This application provides a multi-data type multiplier, including: At least two basic multiplication units of data types, the basic multiplication units being used for combined operations to multiply the mantissa of floating-point numbers or fixed-point numbers of predetermined data types; The control unit is used to select different combinations of basic multiplication units based on the data type of the input matrix. Furthermore, the input matrix is decomposed into multiple parts, which are then transmitted to the corresponding basic multiplication units for computation. In addition, the results of the operations in each part are shifted and accumulated.
[0005] According to a specific implementation of an embodiment of this application, the basic multiplication unit of at least two data types includes: two 8×4 and two 12×4 basic multiplication units; The predetermined data type and the data type of the input matrix respectively include at least one of tf32, bf16, fp16, int8 and int4.
[0006] According to a specific implementation of an embodiment of this application, the input matrix includes a first input matrix and a second input matrix, wherein the first input matrix and the second input matrix are tf32 or fp16 respectively, and their mantissa portions are required to be implemented using int12×int12 multiplication units; The control unit is specifically configured to decompose the mantissa of the first input matrix into high 4 bits and low 8 bits, and to decompose the mantissa of the second input matrix into high 4 bits, middle 4 bits and low 4 bits. By combining two 8×4 and 12×4 basic multiplication units, the decomposed parts are operated on to obtain four operation results. Any two operation results are shifted and accumulated to obtain two intermediate results. The two intermediate results are shifted and accumulated to obtain the final result.
[0007] According to a specific implementation of an embodiment of this application, the input matrix includes a first input matrix and a second input matrix, wherein the first input matrix and the second input matrix are bf16 or int8 respectively, and their mantissa portions are required to be implemented using int8×int8 multiplication units; The control unit is specifically configured to decompose the mantissa of the first input matrix or the first input matrix into high 8 bits and low 8 bits, and to decompose the mantissa of the second input matrix or the second input matrix into high 8 bits and low 8 bits; wherein, when the first input matrix and the second input matrix are both bf16, it corresponds to their mantissa; when the first input matrix and the second input matrix are both int8, it corresponds to their matrix itself. The two basic multiplication units, 8×4 and 12×4, are divided into two groups and combined. The two groups of data are then processed in parallel, with each group of data yielding two results. Shift and accumulate the two calculation results obtained from each set of data to obtain two final results.
[0008] According to a specific implementation of an embodiment of this application, the input matrix includes a first input matrix and a second input matrix, wherein the first input matrix and the second input matrix are fp8 or int4 respectively, and their mantissa portions are required to be implemented using int4×int4 multiplication units; The control unit is specifically used to combine two 8×4 and 12×4 basic multiplication units into four groups, and perform parallel operations on the four groups of decomposed data respectively. Each group of data is used to calculate two operation results. The two calculation results from each set of data are shifted and accumulated to obtain four final results.
[0009] Secondly, this application also provides a method for multiplying multiple types of data, including: Different combinations of basic multiplication units of different data types are selected according to the data type of the input matrix; wherein, the basic multiplication unit includes basic multiplication units of at least two data types, and the data type of the input matrix is floating-point or fixed-point; The input matrix is decomposed into multiple parts, which are then sent to the corresponding basic multiplication units for operation. The results of the operations on each part are shifted and accumulated to obtain the result of multiplying the mantissa parts of the input matrix of floating-point type or the result of multiplying the input matrix of fixed-point type.
[0010] According to a specific implementation of an embodiment of this application, the basic multiplication unit of at least two data types includes: two 8×4 and two 12×4 basic multiplication units; The data type of the input matrix includes at least one of tf32, bf16, fp16, int8, and int4.
[0011] According to a specific implementation of an embodiment of this application, the input matrix includes a first input matrix and a second input matrix, wherein the first input matrix and the second input matrix are tf32 or fp16 respectively, and their mantissa portions are required to be implemented using int12×int12 multiplication units; The step of decomposing the input matrix into multiple parts and transmitting them to the corresponding basic multiplication units for operation; performing shift and accumulation operations on the operation results of each part, including: decomposing the mantissa of the first input matrix into high 4 bits and low 8 bits, and decomposing the mantissa of the second input matrix into high 4 bits, middle 4 bits and low 4 bits. By combining two 8×4 and 12×4 basic multiplication units, the decomposed parts are operated on to obtain four operation results. Any two operation results are shifted and accumulated to obtain two intermediate results. The two intermediate results are shifted and accumulated to obtain the final result.
[0012] According to a specific implementation of an embodiment of this application, the input matrix includes a first input matrix and a second input matrix, wherein the first input matrix and the second input matrix are bf16 or int8 respectively, and their mantissa portions are required to be implemented using int8×int8 multiplication units; The step of decomposing the input matrix into multiple parts and transmitting them to the corresponding basic multiplication units for operation; performing shift and accumulation operations on the operation results of each part, including: decomposing the mantissa of the first input matrix or the first input matrix into high 8 bits and low 8 bits, and decomposing the mantissa of the second input matrix or the second input matrix into high 8 bits and low 8 bits; wherein, when the first input matrix and the second input matrix are both bf16, it corresponds to their mantissa; when the first input matrix and the second input matrix are both int8, it corresponds to their matrices themselves. The two basic multiplication units, 8×4 and 12×4, are divided into two groups and combined. The two groups of data are then processed in parallel, with each group of data yielding two results. The two calculation results obtained from each set of data are shifted and accumulated to obtain two final results.
[0013] According to a specific implementation of an embodiment of this application, the input matrix includes a first input matrix and a second input matrix, wherein the first input matrix and the second input matrix are fp8 or int4 respectively, and their mantissa portions are required to be implemented using int4×int4 multiplication units; The step of decomposing the input matrix into multiple parts and transmitting them to the corresponding basic multiplication units for operation; performing shift and accumulation operations on the operation results of each part, including: using two 8×4 and 12×4 basic multiplication units to divide the data into four groups and combine them, and performing parallel operations on the four groups of data after decomposition, wherein each group of data calculates two operation results; The two calculation results from each set of data are shifted and accumulated to obtain four final results.
[0014] The multi-data type multiplier and method for multiplying multiple data types provided in this application include at least two basic multiplication units of different data types. These basic multiplication units are used to combine operations to multiply the mantissa of floating-point numbers or fixed-point numbers of predetermined data types. A control unit is used to select different combinations of basic multiplication units based on the data type of the input matrix. The input matrix is also decomposed into multiple parts, each of which is sent to a corresponding basic multiplication unit for operation. By utilizing at least two basic multiplication units to create a multiplication unit supporting multiple data formats, operations can be performed without designing separate multipliers for different data types. This improves the reusability of the multiplier, facilitating the implementation of a multiplier supporting multiple data types at a lower cost, thereby saving integrated circuit area and power consumption resources. Attached Figure Description
[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0016] Figure 1 A schematic block diagram of an embodiment of the multi-data type multiplier provided in this application.
[0017] Figure 2 A schematic block diagram illustrating an embodiment of the multi-data type multiplier provided in this application for implementing int12×int12 matrix multiplication required by tf32 or fp16.
[0018] Figure 3 A schematic block diagram illustrating an embodiment of the multi-data-type multiplier provided in this application for implementing int8×int8 matrix multiplication required by bf16 or int8.
[0019] Figure 4 This is a schematic flowchart of an embodiment of the method for multiplying multiple types of data provided in this application.
[0020] Figure 5 This is a schematic diagram of the structure of one embodiment of the electronic device of this application. Detailed Implementation
[0021] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.
[0022] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.
[0023] To help understand the innovative essence of the technical solution in this application, some related technologies are briefly introduced below: The data types supported by multipliers vary depending on their design and application requirements. Fixed-point numbers (such as INT8 / INT16) are typically used to reduce computational complexity and storage requirements, while floating-point numbers (such as FP16) offer higher precision. In floating-point multiplication, the exponents of the multiplier and multiplicand are added, and the mantissas are multiplied; the computational load is mainly in the mantissa multiplication. In fixed-point multiplication, the mantissas can be multiplied directly. Since different data formats have different mantissa lengths, designing a separate multiplier for each data format would waste area and power consumption to accommodate multiple data formats.
[0024] Therefore, see Figure 1 This invention provides a multi-data type multiplier, which is suitable for integrated circuit design scenarios, especially for GPGPUs involved in deep learning training or inference. It utilizes basic multiplication units to combine multiplication modules that support multiple data formats, thereby improving the reusability of the multiplier and avoiding the increase in chip area and power consumption caused by designing multipliers for each data format separately.
[0025] The multi-data type multiplier includes: The basic multiplication unit comprises at least two data types, wherein the basic multiplication unit is used for combination operations to multiply the mantissa of floating-point numbers or fixed-point numbers of a predetermined data type.
[0026] In this embodiment, the basic multiplication unit combination can implement multiplication operations of various data types, such as int12×int12 / int8×int8 / uint8×int8 / int8×int4 / int4×int4, and generate corresponding product results.
[0027] The control unit is configured to select different data type combinations of basic multiplication units based on the data type of the input matrix; decompose the input matrix into multiple parts and transmit them to the corresponding basic multiplication units for operation; and perform shift and accumulation operations on the operation results of each part.
[0028] like Figure 1 The dashed line in the diagram indicates the data control flow executed by the control unit.
[0029] Specifically, the control unit is used to determine the required basic multiplication unit based on the data type identifier of each element in the input matrix. For example, if an input matrix is int8, the control unit will select a basic multiplication unit that supports int8 multiplication. The data is stored in memory for the control unit to read.
[0030] The multi-data type provided in this invention includes at least two basic multiplication units for different data types. These basic multiplication units are used to combine operations to multiply the mantissa of floating-point numbers or fixed-point numbers of predetermined data types. A control unit is configured to select different combinations of basic multiplication units based on the data type of the input matrix; and to decompose the input matrix into multiple parts, each of which is then sent to a corresponding basic multiplication unit for operation. By utilizing at least two basic multiplication units to create multiplication units supporting multiple data formats, operations can be performed without designing separate multipliers for different data types. This improves the reusability of the multiplier, facilitating the implementation of multipliers supporting multiple data types at a lower cost, thereby saving integrated circuit area and power consumption resources.
[0031] In some embodiments, the basic multiplication units of the at least two data types include: two 8×4 and two 12×4 basic multiplication units; the data types of the predetermined data type and the input matrix respectively include at least one of tf32, bf16, fp16, int8 and int4.
[0032] In some embodiments, the input matrix includes a first input matrix and a second input matrix, wherein the first input matrix and the second input matrix are tf32 or fp16 respectively, and their mantissa portions are required to be implemented using int12×int12 multiplication units; the control unit is specifically used to decompose the mantissa portion of the first input matrix into high 4 bits and low 8 bits, and to decompose the mantissa portion of the second input matrix into high 4 bits, middle 4 bits and low 4 bits; using a combination of two 8×4 and 12×4 basic multiplication units, operations are performed on each of the decomposed parts to obtain four operation results; any two operation results are shifted and accumulated to obtain two intermediate results; the two intermediate results are shifted and accumulated to obtain the final result.
[0033] See Figure 2 As shown, for example, for the implementation of an int12×int12 multiplier, assuming A[11:0]={H[3:0],L[7:0]} and B[11:0]={W2[3:0],W1[3:0],W0[3:0]}, the steps using the above basic multiplication unit are as follows: Step 1: Decompose the multiplier and multiplicand and assign them to the basic multiplication unit, then calculate the result of each sub-part.
[0034] As can be seen from the composition of A and B, in this embodiment, the multiplicand A and multiplier B are first decomposed into different parts. The multiplicand A consists of the high-order bits H and the low-order bits L, and the multiplier B consists of three parts W2, W1, and W0. Specifically, different sized basic multiplication units are used to process the multiplication operations of each part as follows: Temp0 = {H[3:0],L[7:0]} × W0[3:0]} is implemented using a 12×4 basic multiplication unit; Temp1 = {L[7:0]} × W1[3:0], implemented using an 8×4 basic multiplication unit; Temp2 = {H[3:0],L[7:0]} × W2[3:0]} is also implemented using a 12×4 basic multiplication unit; Temp3 = {H[3],H[3],H[3],H[3],H[3:0]} × W1[3:0], which is also implemented using an 8×4 basic multiplication unit.
[0035] Step 2: Combine the calculation results from Step 1 to obtain intermediate results. Shift Temp0 and Temp1, calculated in the first step, left by 4 bits and then add them together to get Result0. Specifically: Result0 = (Temp1 << 4) + Temp0.
[0036] Similarly, shift Temp2 and Temp3 left by 4 bits, then add them together to get Result1, specifically: Result1 = (Temp3 << 4) + Temp2.
[0037] Step 3: Calculate the final result Shift Result0 left by 8 bits and then add it to Result1 to get the final multiplication result: Result = (Result0 << 8) + Result1.
[0038] In this embodiment of the invention, by decomposing the int12×int12 multiplication into smaller parts to accommodate the use of different sizes of basic multiplication units for calculation, and then merging the intermediate results to obtain the final multiplication result, it can be applied to scenarios that require int12×int12 multiplication operations.
[0039] The multi-data type multiplier provided in this embodiment of the invention, through the combination of basic multiplication units, can also be applied to scenarios requiring other types of multiplication operations. See [link / reference]. Figure 3 As shown, in some embodiments, the input matrix includes a first input matrix and a second input matrix, wherein the first input matrix and the second input matrix are bf16 or int8 respectively, and their mantissa portions are required to be implemented using int8×int8 multiplication units; The control unit is specifically configured to decompose the mantissa of the first input matrix or the first input matrix into high 8 bits and low 8 bits, and to decompose the mantissa of the second input matrix or the second input matrix into high 8 bits and low 8 bits; wherein, when the first input matrix and the second input matrix are both bf16, it corresponds to their mantissa; when the first input matrix and the second input matrix are both int8, it corresponds to their matrices themselves; using two 8×4 and 12×4 basic multiplication units to divide the data into two groups and combine them, the two groups of data after decomposition are operated on in parallel, wherein each group of data calculates two operation results; the two operation results calculated from each group of data are subjected to shift and accumulation operations to obtain two final results.
[0040] In this embodiment, the multiplication of two sets of data can be calculated simultaneously. For example, H / L represent A1 and A2 respectively, W3 / W2 represents the high / low 4 bits of B1, and W1 / W0 represents the high / low 4 bits of B2. The specific process of implementing two sets of int8×int8 multiplication using basic multiplication units of different sizes to process the multiplication operations of each part is as follows: A1[7:0]={L[7:0]}, A2[7:0]={H[7:0]}, B1[7:0]={W1[3:0],W0[3:0]}, B2[7:0]={W3[3:0],W2[3:0]}: Step 1: Decompose the multiplier and multiplicand and assign them to the basic multiplication units, then calculate the result of each sub-part. Temp0 = {L[7],L[7],L[7],L[7],L[7],L[7:0]} × W0[3:0]}, implemented by a 12×4 basic multiplication unit; Temp1 = {L[7:0]} × W1[3:0], implemented by an 8×4 basic multiplication unit. Temp2 = {H[7],H[7],H[7],H[7],H[7],H[7:0]} × W2[3:0]}, implemented by a 12×4 basic multiplication unit. Temp3 = {H[7:0]} × W3[3:0], implemented by an 8×4 basic multiplication unit.
[0041] Step 2: Combine the calculation results from Step 1 to obtain intermediate results. Shift Temp0 and Temp1, calculated in the first step, left-shift by 4 bits and then add them together to get Result0. Similarly, shift Temp2 and Temp3 left-shift by 4 bits and then add them together to get Result1. The specific calculation is as follows: Result0 = (Temp1 << 4) + Temp0 Result1 = (Temp3 << 4) + Temp2. In this embodiment of the invention, by decomposing multiplications of different data formats into smaller parts and using a common basic multiplication unit for calculation, the reusability of int8×int8 multiplication can be achieved, avoiding the need to design separate multipliers for specific data formats, thereby reducing chip area and power consumption. Furthermore, two sets of int8×int8 multiplication operations can be processed in parallel, improving the multiplier's computational efficiency.
[0042] In some embodiments, the input matrix includes a first input matrix and a second input matrix, wherein the first input matrix and the second input matrix are fp8 or int4 respectively, and the mantissa portion is required to be implemented using an int4×int4 multiplication unit.
[0043] In this embodiment, four sets of int4×int4 multiplication unit operations can be implemented.
[0044] The control unit is specifically used to combine two 8×4 and 12×4 basic multiplication units into four groups, and perform parallel operations on the four groups of decomposed data respectively. Each group of data is used to calculate two operation results. The two calculation results from each set of data are shifted and accumulated to obtain four final results.
[0045] In this embodiment, when the input A / B is fp8 or int4, the mantissa is implemented using an int4×int4 combined multiplication unit, and in this mode, four sets of matrix multiplications can be calculated simultaneously. The implementation process is similar to the two sets of int8×int8 processes mentioned above. The calculation is divided into two parts: the first part uses 8×4 and 12×4 basic multiplication units to implement four int4×int4 multiplications; the second part is used to fuse the results and derive the calculation result.
[0046] To implement the above multiplier, in some embodiments, it is necessary to first implement the 8×4 and 12×4 basic multiplication units. The specific implementation method is as follows: The basic multiplication unit implementation algorithm is a radix-4 Booth multiplier, and the algorithm steps are as follows: 1) Find the two's complement of the multiplier B and the multiplicand A 2) Perform booth encoding on the multiplier B (encoding algorithm as follows) B = -2 n-1 B n-1 +2 n-2 B n-2 +2 n-3 B n-3 + ....... + 2 2 B2 + 2 1 B1 + 2 0 B0 + B -1 = -2×2 n-2 B n-1 +2 n-2 B n-2 + 2×2 n-3 B n-3 - 2×2 n-4 B n-3 +2 n-4 B n-4+ ......- 2 1 B1+ 2 0 B0 + B -1 = 2 n-2 (-2B n-1 + B n-2 + B n-3 ) + 2 n-4 (-2B n-3 + B n-4 + B n-5 ) +......+ 2 0 (-2B1+B0 + B -1 ); 3) Based on the booth encoding result, generate a partial product with A. The partial product is shown in the table below:
[0047] In this embodiment, the basic multiplication unit constituting the above-mentioned multiplication units is implemented by a radix-4 Booth multiplier. Then, the basic multiplication units are combined to achieve multiplication with different input data formats.
[0048] As can be seen from the above embodiments, the multiplier provided by the present invention can perform four int4×int4 matrix multiplications, or two int8×int8 matrix multiplications, or one int12×int12 matrix multiplication in each clock cycle. It can adapt to data multiplication operations of various data types without the need to design separate multipliers for multiple data types, thereby improving the reusability of the multiplier, reducing the chip area occupied, reducing power consumption, and improving the energy efficiency ratio of the entire multiplier.
[0049] Please refer to Figure 2 As shown in the embodiment, this application also provides a method for multiplying multiple types of data, including: S210, selecting a combination of basic multiplication units of different data types according to the data type of the input matrix; wherein, the basic multiplication unit includes at least two basic multiplication units of data types, and the data type of the input matrix is floating-point number or fixed-point number; S220. Decompose the input matrix into multiple parts and transmit them to the corresponding basic multiplication units for operation; S230. Perform shift and accumulation operations on the results of each part to obtain the result of multiplying the mantissa parts of the input matrix of floating-point type or the result of multiplying the input matrix of fixed-point type.
[0050] The method for multiplying multiple types of data provided in this invention combines at least two basic multiplication units to form a multiplication unit that supports multiple data formats for operation. This eliminates the need to design separate multipliers for different types of data, thereby improving the reusability of the multiplier and facilitating the implementation of a multiplier that supports multiple data types at a lower cost.
[0051] In some embodiments, the basic multiplication units of the at least two data types include: two 8×4 and two 12×4 basic multiplication units; The data type of the input matrix includes at least one of tf32, bf16, fp16, int8, and int4.
[0052] In some embodiments, the input matrix includes a first input matrix and a second input matrix, wherein the first input matrix and the second input matrix are tf32 or fp16 respectively, and their mantissa portions are required to be implemented using int12×int12 multiplication units; The step of decomposing the input matrix into multiple parts and transmitting them to the corresponding basic multiplication units for operation; performing shift and accumulation operations on the operation results of each part, including: decomposing the mantissa of the first input matrix into high 4 bits and low 8 bits, and decomposing the mantissa of the second input matrix into high 4 bits, middle 4 bits and low 4 bits. By combining two 8×4 and 12×4 basic multiplication units, the decomposed parts are operated on to obtain four operation results. Any two operation results are shifted and accumulated to obtain two intermediate results. The two intermediate results are shifted and accumulated to obtain the final result.
[0053] In some embodiments, the input matrix includes a first input matrix and a second input matrix, wherein the first input matrix and the second input matrix are bf16 or int8 respectively, and their mantissa portions are required to be implemented using int8×int8 multiplication units; The step of decomposing the input matrix into multiple parts and transmitting them to the corresponding basic multiplication units for operation; performing shift and accumulation operations on the operation results of each part, including: decomposing the mantissa of the first input matrix or the first input matrix into high 8 bits and low 8 bits, and decomposing the mantissa of the second input matrix or the second input matrix into high 8 bits and low 8 bits; wherein, when the first input matrix and the second input matrix are both bf16, it corresponds to their mantissa; when the first input matrix and the second input matrix are both int8, it corresponds to their matrices themselves. The two basic multiplication units, 8×4 and 12×4, are divided into two groups and combined. The two groups of data are then processed in parallel, with each group of data yielding two results. The two calculation results obtained from each set of data are shifted and accumulated to obtain two final results.
[0054] In some embodiments, the input matrix includes a first input matrix and a second input matrix, wherein the first input matrix and the second input matrix are fp8 or int4 respectively, and their mantissa portions are required to be implemented using int4×int4 multiplication units; The process of decomposing the input matrix into multiple parts and transmitting them to the corresponding basic multiplication units for operation, and performing shift and accumulation operations on the operation results of each part, includes: using two 8×4 and 12×4 basic multiplication units to divide the data into four groups and combine them, and performing parallel operations on the four groups of data after decomposition, wherein each group of data calculates two operation results; and performing shift and accumulation operations on the two operation results calculated from each group of data to obtain four final results.
[0055] The method for multiplying multiple types of data provided in this application is similar in principle and technical effect to that in Embodiment 1, and will not be repeated here. They can be referred to each other.
[0056] Figure 5 This is a schematic block diagram illustrating the architecture of an embodiment of the electronic device of the present invention. Based on a technical concept substantially the same as that of Embodiment 1 described above, the electronic device provided in this embodiment of the present invention includes: one or more processors; a memory; the memory stores one or more executable program codes, and the one or more processors read the executable program codes stored in the memory to run a program corresponding to the executable program codes, for use in executing any of the methods described in the foregoing embodiments. Figure 5 As shown, the steps and flow of any of the methods described in Embodiment 1 of the present invention can be implemented.
[0057] The aforementioned electronic device may include: a housing 41, a processor 42, a memory 43, a circuit board 44, and a power supply circuit 45, wherein the circuit board 44 is disposed inside the space enclosed by the housing 41, and the processor 42 and the memory 43 are disposed on the circuit board 44; the power supply circuit 45 is used to supply power to the various circuits or devices of the aforementioned electronic device; the memory 43 is used to store executable program code; the processor 42 runs a program corresponding to the executable program code by reading the executable program code stored in the memory 43, for executing any of the methods described in the foregoing embodiments.
[0058] For details on the specific execution process of the above steps by the processor 42 and the steps further executed by the processor 42 by running executable program code, please refer to the description of Embodiment 1 of the present invention, which will not be repeated here.
[0059] The electronic device exists in various forms, including but not limited to: (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and are primarily designed to provide voice and data communication. These terminals include smartphones (such as iPhones), multimedia phones, feature phones, and low-end phones.
[0060] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access features. These terminals include PDAs, MIDs, and UMPCs, such as the iPad.
[0061] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes audio and video players (such as iPods), handheld game consoles, e-book readers, as well as smart toys and portable car navigation devices.
[0062] (4) Server: A device that provides computing services. The components of a server include a processor, hard disk, memory, system bus, etc. Servers are similar to general computer architectures, but because they need to provide highly reliable services, they have higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.
[0063] (5) Other electronic devices with data interaction functions.
[0064] In summary, the multi-data type multiplier and the method for multiplying multiple types of data provided in this application combine at least two basic multiplication units to form a multiplication unit that supports multiple data formats for operation. This eliminates the need to design separate multipliers for different data types, thereby improving the reusability of the multiplier. This facilitates the implementation of a multiplier that supports multiple data types at a lower cost, reduces the space occupied by the integrated circuit, and improves the overall energy efficiency of the multiplier.
[0065] It should be noted that while the various embodiments described herein have different focuses, they are interconnected. When understanding this application, reference can be made between the various embodiments. Furthermore, relational terms such as "first" and "second" are merely used to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or measurement control unit that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or measurement control unit. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or measurement control unit that includes said element.
[0066] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A multi-data type multiplier, characterized in that, include: At least two basic multiplication units of data types, the basic multiplication units being used for combined operations to multiply the mantissa of floating-point numbers or fixed-point numbers of predetermined data types; The control unit is used to select different combinations of basic multiplication units based on the data type of the input matrix. Furthermore, the input matrix is decomposed into multiple parts, which are then transmitted to the corresponding basic multiplication units for operation. In addition, the results of the operations in each part are shifted and accumulated.
2. The multi-data type multiplier according to claim 1, characterized in that, The basic multiplication units of the at least two data types include: two 8×4 and two 12×4 basic multiplication units; The predetermined data type and the data type of the input matrix respectively include at least one of tf32, bf16, fp16, int8 and int4.
3. The multi-data type multiplier according to claim 2, characterized in that, The input matrix includes a first input matrix and a second input matrix, wherein the first input matrix and the second input matrix are tf32 or fp16 respectively, and their mantissas are required to be implemented using int12×int12 multiplication units; The control unit is specifically configured to decompose the mantissa of the first input matrix into high 4 bits and low 8 bits, and to decompose the mantissa of the second input matrix into high 4 bits, middle 4 bits and low 4 bits. By combining two 8×4 and 12×4 basic multiplication units, the decomposed parts are operated on to obtain four operation results. Any two operation results are shifted and accumulated to obtain two intermediate results. The two intermediate results are shifted and accumulated to obtain the final result.
4. The multi-data type multiplier according to claim 2, characterized in that, The input matrix includes a first input matrix and a second input matrix, wherein the first input matrix and the second input matrix are bf16 or int8 respectively, and their mantissas are required to be implemented using int8×int8 multiplication units. The control unit is specifically configured to decompose the mantissa of the first input matrix or the first input matrix into high 8 bits and low 8 bits, and to decompose the mantissa of the second input matrix or the second input matrix into high 8 bits and low 8 bits; wherein, when the first input matrix and the second input matrix are both bf16, it corresponds to their mantissa; when the first input matrix and the second input matrix are both int8, it corresponds to their matrix itself. The two basic multiplication units, 8×4 and 12×4, are divided into two groups and combined. The two groups of data are then processed in parallel, with each group of data yielding two results. Shift and accumulate the two calculation results obtained from each set of data to obtain two final results.
5. The multi-data type multiplier according to claim 2, characterized in that, The input matrix includes a first input matrix and a second input matrix, wherein the first input matrix and the second input matrix are fp8 or int4 respectively, and their mantissas are required to be implemented using int4×int4 multiplication units; The control unit is specifically used to combine two 8×4 and 12×4 basic multiplication units into four groups, and perform parallel operations on the four groups of decomposed data respectively. Each group of data is used to calculate two operation results. The two calculation results from each set of data are shifted and accumulated to obtain four final results.
6. A method for multiplying multiple types of data, characterized in that, include: Different combinations of basic multiplication units of different data types are selected according to the data type of the input matrix; wherein, the basic multiplication unit includes basic multiplication units of at least two data types, and the data type of the input matrix is floating-point or fixed-point; The input matrix is decomposed into multiple parts, which are then sent to the corresponding basic multiplication units for operation. The results of the operations on each part are shifted and accumulated to obtain the result of multiplying the mantissa parts of the input matrix of floating-point type or the result of multiplying the input matrix of fixed-point type.
7. The method for multiplying multiple types of data according to claim 6, characterized in that, The basic multiplication units of the at least two data types include: two 8×4 and two 12×4 basic multiplication units; The data type of the input matrix includes at least one of tf32, bf16, fp16, int8, and int4.
8. The method for multiplying multiple types of data according to claim 7, characterized in that, The input matrix includes a first input matrix and a second input matrix, wherein the first input matrix and the second input matrix are tf32 or fp16 respectively, and their mantissas are required to be implemented using int12×int12 multiplication units; The input matrix is decomposed into multiple parts, which are then transmitted to the corresponding basic multiplication units for operation. The operation results of each part are shifted and accumulated, including: decomposing the mantissa of the first input matrix into high 4 bits and low 8 bits, and decomposing the mantissa of the second input matrix into high 4 bits, middle 4 bits and low 4 bits. By combining two 8×4 and 12×4 basic multiplication units, the decomposed parts are operated on to obtain four operation results. Any two operation results are shifted and accumulated to obtain two intermediate results. The two intermediate results are shifted and accumulated to obtain the final result.
9. The method for multiplying multiple types of data according to claim 7, characterized in that, The input matrix includes a first input matrix and a second input matrix, wherein the first input matrix and the second input matrix are bf16 or int8 respectively, and their mantissas are required to be implemented using int8×int8 multiplication units. The input matrix is decomposed into multiple parts, which are then transmitted to the corresponding basic multiplication units for operation. The operation results of each part are shifted and accumulated, including: decomposing the mantissa of the first input matrix or the first input matrix into high 8 bits and low 8 bits, and decomposing the mantissa of the second input matrix or the second input matrix into high 8 bits and low 8 bits; wherein, when the first input matrix and the second input matrix are both bf16, it corresponds to their mantissa; when the first input matrix and the second input matrix are both int8, it corresponds to their matrices themselves. The two basic multiplication units, 8×4 and 12×4, are divided into two groups and combined. The two groups of data are then processed in parallel, with each group of data yielding two results. The two calculation results obtained from each set of data are shifted and accumulated to obtain two final results.
10. The method for multiplying multiple types of data according to claim 7, characterized in that, The input matrix includes a first input matrix and a second input matrix, wherein the first input matrix and the second input matrix are fp8 or int4 respectively, and their mantissas are required to be implemented using int4×int4 multiplication units; The input matrix is decomposed into multiple parts, which are then transmitted to the corresponding basic multiplication units for operation. The results of each part of the operation are shifted and accumulated, including: using two 8×4 and 12×4 basic multiplication units to divide the data into four groups and combine them, and then performing operations on the four groups of data in parallel. Each group of data is used to calculate two operation results. The two calculation results from each set of data are shifted and accumulated to obtain four final results.