Floating-point number processing method, and computing device and computing chip

Abstract

The present application relates to the technical field of computers. Provided are a floating-point number processing method, and a computing device and a computing chip. The computing device acquires a first floating-point number, and obtains a second floating-point number on the basis of a first sign field, a first exponent field and a first mantissa field of the first floating-point number. The data format used by the second floating-point number comprises a shared data part and a variable data part, wherein the shared data part is stored in a shared area of a memory, and the shared data part is shared by a plurality of pieces of data within the data range to which the first floating-point number belongs. In the shared area of the memory, a common shared data part may be stored for a plurality of data, thereby facilitating the representation of floating point numbers by using limited bit widths. The shared data part comprises a second exponent field and a second mantissa field, wherein the second mantissa field of the shared data part may be used for compensating for the precision of values that can be represented by the variable data part, thereby improving the numerical precision of data that can be represented by the floating-point numbers, and thus satisfying the requirements of AI training and inference.

Description

A floating-point number processing method, computing device and computing chip

[0001] Cross-reference to related applications

[0002] This application claims priority to Chinese Patent Application No. 202411629510.5, filed on November 14, 2024, with the title “A Floating-Point Number Processing Method, Computing Device and Computing Chip”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of computer technology, and in particular to a floating-point number processing method, computing device, and computing chip. Background Technology

[0004] In computer systems, floating point (FP) is an approximate numerical representation of real numbers, also known as floating-point data representation. For example, floating-point data representations can include FP16 and FP32, where FP16 represents a 16-bit floating-point number and FP32 represents a 32-bit floating-point number.

[0005] Typically, floating-point numbers contain three fields: the sign field, the exponent field, and the mantissa field. The value of the exponent field represents the exponent of a given base. For example, if the base is 2, the value of the exponent field (i.e., the exponent of 2) is usually an integer, representing an integer power of 2. The value of the mantissa field, multiplied by the exponent of the base, yields a data value. The sign field indicates whether the data value is positive or negative. The bit width of the exponent field determines the range of values ​​that the floating-point number can represent, while the bit width of the mantissa field determines the precision of the values ​​that the floating-point number can represent.

[0006] With the rapid development of artificial intelligence (AI) applications and neural network model training and inference, floating-point numbers are increasingly being used in AI scenarios. As the scale of data used in AI scenarios (such as network parameters) increases dramatically, using floating-point numbers with shorter bit widths for AI training and inference can improve the computational efficiency of floating-point numbers. However, in AI scenarios, the numerical precision of the data is also crucial to the performance of model training and inference.

[0007] Improving the numerical precision of floating-point numbers is a problem that urgently needs to be solved. Summary of the Invention

[0008] This application provides a floating-point number processing method, a computing device, and a computing chip, which can improve the numerical accuracy of floating-point numbers.

[0009] In a first aspect, embodiments of this application provide a floating-point number processing method, which can be executed by a computing device, including an encoder and a memory. The floating-point number processing method includes: an encoder acquiring a first floating-point number, wherein the first floating-point number is a normalized floating-point number, and the first floating-point number includes a first sign field, a first exponent field, and a first mantissa field. The first sign field indicates the sign of the first floating-point number, the first exponent field represents the exponent of the first floating-point number, and the first mantissa field represents the mantissa of the first floating-point number. The encoder obtains a second floating-point number based on the first sign field, the first exponent field, and the first mantissa field of the first floating-point number. The second floating-point number uses a data format including a shared data portion and a variable data portion. The shared data portion is determined by the data range to which the data represented by the first floating-point number belongs, and the shared data portion is stored in a shared area of ​​the memory. The shared data portion is shared by multiple data within the data range to which the first floating-point number belongs; the shared data portion includes a second exponent field and a second mantissa field. The second floating-point number represents the same data as the first floating-point number.

[0010] In this embodiment, a single data point is represented by a shared data portion and a variable data portion. Multiple data points can use the same shared data portion. When storing data, a common shared data portion can be used for storing multiple data points in the shared area of ​​the memory, while the variable data portion of each data point is stored separately in the memory. This helps to express floating-point numbers using a limited bit width. Furthermore, the shared data portion includes a second exponent field and a second mantissa field. The second mantissa field of the shared data portion can be used to supplement the numerical precision that the variable data portion can represent, thereby improving the numerical precision of the data that floating-point numbers can represent and meeting the needs of AI training and inference.

[0011] In one alternative implementation, the second exponent field is used to characterize the exponent of the shared data portion, and the second mantissa field is used to characterize the mantissa of the shared data portion.

[0012] In another alternative implementation, the second exponent field is used to characterize the exponent of the shared data portion; when the value of the second exponent field is a set value, the second mantissa field is used to characterize the offset of the exponent of the shared data portion.

[0013] In the above implementation, when the value of the second exponent field is a set value, the second mantissa field can be used to represent the offset of the exponent in the shared data part; when the value of the second exponent field is not a set value, the second mantissa field can be used to represent the mantissa of the shared data part. By reusing the second mantissa field, it is possible to ensure that the numerical precision of the floating-point number is minimally affected, while increasing the encoding space of the exponent, thereby expanding the dynamic range of the numerical value represented by the floating-point number.

[0014] In one alternative implementation, the encoder can determine the data range to which the data represented by the first floating-point number belongs based on the first sign field, the first exponent field, and the first mantissa field of the first floating-point number, obtain the shared data part corresponding to the data range, and determine the variable data part based on the first floating-point number and the shared data part to obtain the second floating-point number.

[0015] In the above implementation, by first determining the data range to which the first floating-point number belongs, obtaining the shared data portion, and then determining the variable data portion, the first floating-point number can be quickly converted into a second floating-point number. The encoder can determine the shared data portion of the data range to which the first floating-point number belongs through calculation, or it can retrieve the pre-saved shared data portion from memory, thus saving computing resources and improving the computational efficiency of the data conversion process.

[0016] In one alternative implementation, the shared data portion corresponding to a data range can be determined as follows: the encoder determines a first ratio between the maximum absolute value of multiple data points within the data range and a predetermined maximum value; based on this first ratio, the shared data portion is obtained. The predetermined maximum value is the maximum value of data that can be represented by a second floating-point data format. Obtaining the shared data portion of a data range based on the ratio of the maximum absolute value of multiple data points to the predetermined maximum value ensures that the shared data portions corresponding to multiple data ranges conform to a certain pattern, avoiding disorder.

[0017] In one alternative implementation, the encoder may determine the second exponent field of the shared data portion based on the exponent of the first ratio, or based on the sum of the exponent of the first ratio and a first preset exponent. The encoder may also determine the second mantissa field of the shared data portion based on the mantissa of the first ratio.

[0018] For example, assuming the second exponent field of the shared data portion has a bit width of 5 bits, if the first floating-point number is an FP16 format floating-point number, the bit width of the exponent field of the resulting first ratio is 5 bits, and the exponent of the first ratio can be used as the value of the second exponent field of the shared data portion; if the first floating-point number is a BF16 format floating-point number, the bit width of the exponent field of the resulting first ratio is 8 bits, when converting the exponent field from 8 bits to 5 bits, to avoid the risk of overflow after data format conversion, the second exponent field can be determined based on the sum of the exponent of the first ratio and the first preset exponent.

[0019] In another alternative implementation, the encoder can determine the second exponent field of the shared data portion based on the exponent of the first ratio, and when the value of the second exponent field is a set value, determine the second mantissa field of the shared data portion based on the exponent of the first ratio.

[0020] In another alternative implementation, the encoder can determine the second exponent field of the shared data portion based on the sum of the exponent of the first ratio and the first preset exponent; when the second exponent field is a set value, the second mantissa field of the shared data portion is determined based on the sum of the exponent of the first ratio and the first preset exponent.

[0021] In the above implementation, when the value of the second exponent field is a set value, the second mantissa field of the shared data part is used to characterize the offset of the exponent of the shared data part. Therefore, the second mantissa field of the shared data part can be determined based on the exponent of the first ratio, or the second mantissa field of the shared data part can be determined based on the sum of the exponent of the first ratio and the first preset exponent.

[0022] In one alternative implementation, the encoder can determine a second ratio of the first floating-point number to the shared data portion, and based on the second ratio, obtain the variable data portion of the second floating-point number.

[0023] In one alternative implementation, the computing device may further include a computing unit. After the encoder obtains the second floating-point number, the computing unit may use the second floating-point number and the third floating-point number to perform a multiplication calculation task to obtain the calculation result.

[0024] The second floating-point number data format provided in this application embodiment can use a finite bit width to express floating-point numbers. Using the second floating-point number to participate in the calculation task can improve the calculation efficiency.

[0025] In one optional implementation, the calculation unit may use the sum of the exponent of the shared data portion of the second floating-point number and the exponent of the shared data portion of the third floating-point number as the first part of the exponent of the calculation result; use the product of the mantissa of the shared data portion of the second floating-point number and a preset mantissa as the first part of the mantissa of the calculation result; and obtain the calculation result based on the first part of the exponent, the first part of the mantissa, and the product of the variable data portion of the second floating-point number and the variable data portion of the third floating-point number.

[0026] The above implementation provides a method and computing architecture for performing multiplication operations between the second floating-point number and other floating-point numbers of different data formats using the second floating-point number in the embodiments of this application. This computing architecture has high computing efficiency.

[0027] In another optional implementation, the calculation unit can use the sum of the exponent of the shared data portion of the second floating-point number and the exponent of the shared data portion of the third floating-point number as the second part of the exponent of the calculation result; use the product of the mantissa of the shared data portion of the second floating-point number and the mantissa of the shared data portion of the third floating-point number as the second part of the mantissa of the calculation result; and obtain the calculation result based on the second part of the exponent, the second part of the mantissa, and the product of the variable data portion of the second floating-point number and the variable data portion of the third floating-point number.

[0028] The above implementation provides a method and computing architecture for multiplying two floating-point numbers using the data format proposed in the embodiments of this application. This computing architecture has high computing efficiency.

[0029] Secondly, embodiments of this application provide a floating-point number processing method, which can be executed by a computing device, including a decoder and a memory. The floating-point number processing method may include: the decoder acquiring a second floating-point number. The second floating-point number uses a data format including a shared data portion and a variable data portion. The shared data portion is shared by multiple data within the data range to which the second floating-point number belongs. The shared data portion can be stored in a shared area of ​​the memory. The shared data portion includes a second exponent field and a second mantissa field. Multiple data within the data range to which the second floating-point number belongs contain the same shared data portion. The decoder can decode the second floating-point number to obtain a first floating-point number. The first floating-point number is a normalized floating-point number, which may include a first sign field, a first exponent field, and a first mantissa field. The first floating-point number and the second floating-point number represent the same data.

[0030] In one alternative implementation, the second exponent field is used to characterize the exponent of the shared data portion, and the second mantissa field is used to characterize the mantissa of the shared data portion.

[0031] In another alternative implementation, the second exponent field is used to characterize the exponent of the shared data portion; when the value of the second exponent field is a set value, the second mantissa field is used to characterize the offset of the exponent of the shared data portion.

[0032] In one alternative implementation, the decoder may determine the first sign field of the first floating-point number based on the sign of the variable data portion of the second floating-point number; and determine the first exponent field and the first mantissa field of the first floating-point number based on the product of the variable data portion and the shared data portion of the second floating-point number.

[0033] Thirdly, embodiments of this application provide a computing device, which includes an encoder and a memory; the encoder is used to execute a floating-point number processing method executed by any of the encoders provided in the first aspect above, and the memory is used to store a second floating-point number.

[0034] In one alternative implementation, the computing device may further include a computing unit for executing the floating-point processing method performed by any of the computing units provided in the first aspect above.

[0035] Fourthly, embodiments of this application provide a computing device including a decoder and a memory; the decoder is used to execute any of the floating-point number processing methods provided in the second aspect above.

[0036] Fifthly, embodiments of this application provide a computing chip, which includes a computing unit and an interface;

[0037] The interface is used to obtain the second floating-point number and the third floating-point number. The data format of the second floating-point number includes a shared data part and a variable data part. The shared data part is shared by multiple data within the data range to which the second floating-point number belongs. The shared data part is stored in the shared area of ​​the memory and includes the second exponent field and the second mantissa field.

[0038] The computing unit is used to perform multiplication calculations with a third floating-point number using the shared data portion and variable data portion of the second floating-point number.

[0039] In one alternative implementation, the third floating-point number includes a shared data portion and a variable data portion; the shared data portion of the third floating-point number includes the exponent; the computation unit may include a shared data operation module and a multiplier;

[0040] The shared data operation module is used to sum the exponent of the shared data part of the second floating-point number and the exponent of the shared data part of the third floating-point number as the first part of the exponent of the calculation result;

[0041] The shared data processing module is also used to take the product of the mantissa of the shared data part of the second floating-point number and the preset mantissa as the first part of the mantissa of the calculation result;

[0042] The multiplier is used to calculate the result by multiplying the first part of the exponent, the first part of the mantissa, and the variable data part of the second floating-point number with the variable data part of the third floating-point number.

[0043] In another alternative implementation, the third floating-point number uses the same data format as the second floating-point number; the computation unit includes a shared data operation module and a multiplier.

[0044] The shared data operation module is used to sum the exponent of the shared data part of the second floating-point number and the exponent of the shared data part of the third floating-point number as the second part of the exponent of the calculation result;

[0045] The shared data operation module is also used to take the product of the mantissa of the shared data part of the second floating-point number and the mantissa of the shared data part of the third floating-point number as the second part of the mantissa of the calculation result;

[0046] The multiplier is used to calculate the result by multiplying the second part of the exponent, the second part of the mantissa, and the variable data part of the second floating-point number with the variable data part of the third floating-point number.

[0047] In a sixth aspect, embodiments of this application provide an encoding chip, including an interface and an encoder. The interface is used to acquire a first floating-point number, and the encoder is used to execute any of the floating-point number processing methods executed by the encoder in the first aspect described above.

[0048] In a seventh aspect, embodiments of this application provide a decoding chip, including an interface and a decoder. The interface is used to obtain a second floating-point number, and the decoder is used to execute any of the floating-point number processing methods provided in the second aspect above.

[0049] The technical effects that can be achieved by any of the second to seventh aspects mentioned above can be referred to the description of the beneficial effects in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0050] Figure 1 is a schematic diagram of an application scenario according to an embodiment of this application;

[0051] Figure 2 is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0052] Figure 3 is a schematic diagram of a floating-point number data format provided in an embodiment of this application;

[0053] Figure 4 is a flowchart of a floating-point number processing method provided in an embodiment of this application;

[0054] Figure 5 is a flowchart of another floating-point number processing method provided in an embodiment of this application;

[0055] Figure 6 is a flowchart of another floating-point number processing method provided in an embodiment of this application;

[0056] Figure 7 is a flowchart of another floating-point number processing method provided in an embodiment of this application;

[0057] Figure 8 is a flowchart of another floating-point number processing method provided in an embodiment of this application;

[0058] Figure 9 is a schematic diagram of a floating-point multiplication calculation process provided in an embodiment of this application;

[0059] Figure 10 is a schematic diagram of the processing of shared data in a multiplication calculation according to an embodiment of this application;

[0060] Figure 11 is a schematic diagram of another multiplication calculation process for the shared data portion provided in an embodiment of this application;

[0061] Figure 12 is a schematic diagram of another multiplication calculation process for the shared data portion provided in an embodiment of this application;

[0062] Figure 13 is a schematic diagram of another multiplication calculation process for the shared data portion provided in an embodiment of this application;

[0063] Figure 14 is a schematic diagram of a matrix multiplication calculation application scenario provided by an embodiment of this application;

[0064] Figure 15 is a flowchart of another floating-point number processing method provided in an embodiment of this application;

[0065] Figure 16 is a schematic diagram of the structure of a computing chip provided in an embodiment of this application;

[0066] Figure 17 is a schematic diagram of the structure of an encoding chip provided in an embodiment of this application. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the embodiments of this application will be described in detail below with reference to the accompanying drawings. The terminology used in the implementation section of this application is only for explaining specific embodiments of this application and is not intended to limit this application.

[0068] In this application embodiment, "multiple" refers to two or more. Therefore, in this application embodiment, "multiple" can also be understood as "at least two". "At least one" can be understood as one or more, such as one, two, or more. For example, "including at least one" means including one, two, or more, and it does not limit which ones are included. For example, including at least one of A, B, and C, then it could include A, B, C, A and B, A and C, B and C, or A and B and C. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / ", unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0069] Unless otherwise stated, the ordinal numbers such as "first" and "second" mentioned in the embodiments of this application are used to distinguish multiple objects, and are not used to limit the order, sequence, priority or importance of multiple objects.

[0070] Floating-point representation is a scientific computing method. The Institute of Electrical and Electronics Engineers (IEEE) 754 binary floating-point standard specifies a normalized floating-point number. A normalized floating-point number can contain three fields: a sign field, an exponent field, and a mantissa field. The value of the exponent field represents the integer exponent of a given base. The value of the mantissa field multiplied by the integer exponent of the given base yields a data value. The sign field indicates whether the data is positive or negative. The formula for representing data using normalized floating-point numbers can be written as: (-1) s ×2 E ×(1+M / 1024), where the exponent s of the base -1 is the value of the sign field in the floating-point number, used to indicate the positive or negative sign of the data; E is the value of the exponent field in the floating-point number, and M is the value of the mantissa field in the floating-point number. For example, taking the decimal data 4.625 as an example, this data is converted to binary data 100.101, and represented using binary floating-point numbers, this data can be represented as (-1). 0 ×2 2 ×1.00101, where the exponent 0 of the base -1 is the value of the sign field in the floating-point number, used to indicate the positive or negative sign of the data. The exponent 2 of the base 2 is the value of the exponent field in the floating-point number, representing the number of decimal point shifts. For example, from 100.101 to 1.00101, the decimal point is shifted by 2. 1.00101 is the mantissa. Since the value before the decimal point must be 1, the actual value stored in the mantissa field of the floating-point number can only include the data after the decimal point, i.e., 00101.

[0071] In commonly used floating-point data representations, the bit width of each field of a floating-point number is fixed. For example, as shown in Table 1, floating-point data representations can include FP32, FP16, and BF16 formats. FP32 format represents a 32-bit floating-point number, including a 1-bit sign field, an 8-bit exponent field, and a 23-bit mantissa field. FP16 format represents a 16-bit floating-point number, including a 1-bit sign field, a 5-bit exponent field, and a 10-bit mantissa field. BF16 format represents a 16-bit floating-point number, including a 1-bit sign field, an 8-bit exponent field, and a 7-bit mantissa field. The bit width of the exponent field determines the range of values ​​that the floating-point number can represent, and the bit width of the mantissa field determines the precision of the values ​​that the floating-point number can represent. The floating-point numbers in the above data formats can be considered to include a first sign field, a first exponent field, and a first mantissa field.

[0072] Table 1

[0073] With the rapid development of AI applications and neural network model training and inference, floating-point numbers are increasingly being used in AI scenarios. As the scale of data used in AI scenarios increases dramatically, using floating-point numbers with shorter bit widths for AI training and inference can improve computing power and efficiency. For example, the shift from FP32 to FP16 format, reducing the total bit width of floating-point numbers from 32 bits to 16 bits, can double the computing power. However, a smaller overall bit width for floating-point numbers also means smaller bit widths in the exponent and mantissa fields, resulting in a smaller range of values ​​that can be represented and reduced numerical precision. This cannot meet the needs of AI training and inference, thus impacting the performance of AI training and inference.

[0074] Based on this, this application provides a floating-point number processing method. The floating-point number provided in this application includes a shared data portion and a variable data portion. The shared data portion includes a second exponent field and a second mantissa field. The floating-point number in the above data format can be called a float microscaling (FMX) floating-point number. The shared data portion and the variable data portion jointly represent a single data point, and multiple data points within the same data range use the same shared data portion. This application uses the shared data portion and the variable data portion to jointly represent a single data point. Multiple data points can use the same shared data portion. When storing data, a common shared data portion can be used for multiple data points, and the variable data portion of each data point can be stored separately. This helps to express floating-point numbers using a limited bit width. Furthermore, the shared data portion includes a second exponent field and a second mantissa field. The second mantissa field of the shared data portion can be used to supplement the numerical precision that the variable data portion can represent, thereby improving the numerical precision of the data that the floating-point number can represent and meeting the needs of AI training and inference.

[0075] The floating-point number processing method provided in this application embodiment can be applied to computing devices and is widely applicable to various industries such as scientific research, engineering, finance, aerospace, and medicine. These industries have a large amount of data that needs to be stored, calculated, and transmitted every day. Figure 1 illustrates an application scenario provided by this application embodiment. As shown in Figure 1, in this application scenario, computing devices 100, 200, and 300 can establish a communication connection through a network. This network can be a wired network or a wireless network, such as wireless fidelity (WiFi), Bluetooth, or a mobile network. Computing devices 100, 200, and 300 can be any electronic device such as a computer, server, smart wearable device, smart home device, tablet computer, laptop computer, vehicle terminal, or smartphone. When the computing device is a server, it can be a single server, a server cluster consisting of multiple servers, or a cloud computing service center, etc. The communication connection established between computing devices 100, 200, and 300 enables floating-point number-based data storage, calculation, and transmission.

[0076] It should be noted that in actual application scenarios, the system may contain more than 3 computing devices or less than 3 computing devices, and this application does not limit this.

[0077] The internal structures of computing devices 100, 200, and 300 can be identical. Taking computing device 100 as an example, as shown in Figure 2, computing device 100 may internally include a processor 110 and a memory 120. The processor 110 and the memory 120 can be interconnected via a bus, which can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc.

[0078] The memory 120 can be used to store software programs and modules. The processor 110 executes various functional applications and data processing of the computing device 100 by running the software programs and modules stored in the memory 120, such as the floating-point number processing method provided in the embodiments of this application.

[0079] The memory 120 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, application programs of at least one application, etc.; the data storage area may be used to store data, etc. Furthermore, the memory 120 may include random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and may also include non-volatile memory, such as hard disks, disk storage devices, or other non-volatile solid-state storage devices.

[0080] The processor 110 may be, but is not limited to, a central processing unit (CPU), a high-performance computing (HPC) service acceleration chip, a graphics processing unit (GPU), or a neural network processing unit (NPU) or tensor network processing unit (TPU) in the field of AI. The processor 110 may include an encoder 112 and computing units 113. The number of computing units 113 may be one or more; for example, the computing units 113 may include one or more of scalar computing units, vector computing units, matrix computing units, and tensor computing units.

[0081] The different computing units will be introduced below.

[0082] I. Scalar Computation Unit: A scalar, also known as a scalar, has only magnitude and no direction. A circuit for scalar computation is called a scalar computation unit. Scalar computation is often used in general-purpose computing. In the embodiments of this application, an arithmetic logic unit (ALU) based on the FMX data format can be embedded in the execution unit (EXU) section of a CPU multi-stage pipeline or the scalar computation section of other processors with similar functions.

[0083] II. Vector Computation Unit: A vector, also known as a vector array, typically refers to a one-dimensional array with a length greater than 1. A computational unit with a certain degree of parallelism specifically designed for vector computation is called a vector computation unit, such as a single instruction multiple data (SIMD) processor. Vector computation units are widely used in HPC (High-Performance Computing) and AI machine learning fields, including solving mathematical problems such as linear programming, Fourier transforms, filtering calculations, and linear algebra, partial differential equations, and integrals. In the embodiments of this application, an arithmetic execution unit (vector unit) based on the FMX data format can be embedded in the vector computation acceleration unit or the vector processor.

[0084] III. Matrix Computation Unit: A matrix is ​​a 2D array arranged in a rectangular pattern. A computational unit specifically designed for matrix computations and possessing corresponding parallelism is called a matrix computation unit, such as a systolic array processor. Matrix computation units are widely used in matrix computations in fields such as HPC (High-Performance Computing) and AI machine learning, including operations such as matrix multiplication, matrix inversion, and matrix factorization. In this embodiment, matrix units based on the FMX data format can be embedded within the matrix computation acceleration unit.

[0085] IV. Tensor Computation Unit: A tensor is a multidimensional array with a dimension exceeding 2, commonly a 3-dimensional array. A computation unit specifically designed for tensor computation and possessing corresponding parallelism is called a tensor integration unit. Tensor computation units are widely used in AI machine learning, such as convolution operations. In the embodiments of this application, tensor units based on the FMX data format can be embedded in the tensor computation acceleration unit.

[0086] For example, when the computing device 100 performs general computing, high-performance computing, or AI training, it needs to use a large amount of floating-point data. In some embodiments, the computing device 100 can use the encoder 112 to convert the acquired floating-point numbers of other formats to obtain the floating-point numbers provided in this application embodiment. The floating-point number input calculation unit 113 provided in this application embodiment performs the corresponding calculation and obtains the calculation result, which can be saved to the memory 120.

[0087] In other embodiments, the processor 110 may further include a decoder 111. The computing device 100 can decode the acquired floating-point number provided in this embodiment of the application through the decoder 111 to obtain decoded data. Exemplarily, the floating-point number can be read from the local memory 120 of the computing device 100, or obtained from the computing device 200 or 300 via a network, or from other devices in the network. After decoding the acquired floating-point number, the decoder 111 can transmit the decoded data to the computing unit 113, where the computing unit 113 performs the corresponding calculation. The computing unit 113 transmits the calculation result to the encoder 112, which re-encodes the calculation result into the floating-point number provided in this embodiment of the application. This floating-point number can be used for data storage and data transfer. For example, the processor 110 can save the floating-point number encoded by the encoder 112 into the memory 120.

[0088] Optionally, the structure and function of computing devices 200 and 300 in FIG1 can be specifically referred to computing device 100 shown in FIG2. In some possible embodiments, computing devices 100, 200 and 300 may include more or fewer components than those shown in FIG2. This application embodiment does not specifically limit this.

[0089] To facilitate understanding, the data format of floating-point numbers provided in this application embodiment will be described below. As shown in Figure 3, the data format of floating-point numbers provided in this application embodiment includes two parts: a shared data portion (scale) and a variable data portion (element). The shared data portion includes a second exponent field and a second mantissa field. In some embodiments, the variable data portion may include a second sign field, a third exponent field, and a third mantissa field. The shared data portion and the variable data portion together represent a single data point, and multiple data points within the data range to which this data point belongs use the same shared data portion. This data format is based on block floating-point, where a data range can be considered a data block or a data set. For example, a data range may include 32 consecutive data points, meaning that every 32 data points use the same shared data portion, or in other words, the shared data portion of the 32 consecutive data points represents the same scale value. The following embodiments use binary as an example for illustration.

[0090] In some embodiments, the shared data portion of a floating-point number can be represented as ExMy, where Ex represents the second exponent field with a bit width of x bits, used to characterize the exponent of the shared data portion; My represents the second mantissa field with a bit width of y bits, used to characterize the mantissa of the shared data portion. The shared data portion of the floating-point number is unsigned, and the value represented by the shared data portion of the floating-point number is positive or negative by default. In one embodiment, the bit width of the shared data portion of the floating-point number can be 8, i.e., x + y = 8. Where x = 6, y = 2; x = 5, y = 3; x = 4, y = 4, that is, the shared data portion of the floating-point number can be represented in different formats such as E6M2, E5M3, and E4M4.

[0091] The second-order code field Ex is used to characterize the exponent of the shared data portion. The exponent of the shared data portion is the difference between the value of the second-order code field Ex and the static offset Ebias. The original range of the value of the second-order code field Ex is Ex = [0, ..., 2]. x -1], the static offset Ebias is Ebias = 2 x- 1 -1; where x is the bit width of the second exponent field Ex, and the value of the exponent in the shared data part is Ex-Ebias. The mantissa of the shared data part is the value of the second mantissa field My, and the range of the value represented by the second mantissa field My is My=[0,…,2]. y -1]. The values ​​represented by the shared data portion of floating-point numbers are shown in Table 2.

[0092] Table 2

[0093] As shown in Table 2, when the value of the second exponent field Ex is Ex==2 x -1, the last digit of the shared data portion, My == 2 y When -1, the shared data portion of the floating-point number represents the special value infinity (INF); when the value of the second exponent field Ex is 2... x -1, the last digit of the shared data portion My = [0,…,2] y When -2], the shared data portion of the floating-point number represents a value of 2. Ex-Ebias ×(1+My / 2 y When the value of the second-order code field Ex is Ex = [1,…,2], ... x When -2], the shared data portion of the floating-point number represents a value of 2. Ex-Ebias ×(1+My / 2 y When the value of the second exponent field Ex is 0, the value represented by the shared data part of the floating-point number is 2. 1-Ebias ×(My / 2 y ).

[0094] In addition to representing the special value INF, the shared data portion of a floating-point number can also represent: zero (indistinguishable between positive and negative) and not a number (NaN). For example, when every bit of the shared data portion of a floating-point number is 1, it can represent NaN; when every bit of the shared data portion of a floating-point number is 0, it can represent Zero. For instance, when the bit width of the shared data portion of a floating-point number is 8 bits, "11111111" represents NaN, and "00000000" represents 0.

[0095] When the second-order code field Ex! = 0, that is, Ex = [1,…,2] x When the value of the shared data portion of the floating-point number is -1, the value is normal. When the value of the second exponent field Ex is 0, the value is denormal. The shared data portion of the floating-point number can be represented in different formats such as E6M2, E5M3, and E4M4. The maximum and minimum values ​​of the data that can be represented by the shared data portion in different formats are shown in Table 3.

[0096] Table 3

[0097] In Table 3, "scale" refers to the shared data portion of the floating-point number. As shown in Table 3, when the shared data portion of the floating-point number is in E6M2 format, x = 6, y = 2, and the value of the second exponent field Ex is [1,…,2]. x -1] = [1,…,2] 6 When [-1] = [1,…,63], the exponent representation of the shared data portion is Ex-Ebias, where Ebias = 2. x-1 -1 = 2 6-1 -1 = 32 - 1 = 31, the exponent range of the normal represented by the shared data part is [1-31,…,63-31] = [-30,…,32] = [-30:32]. At this time, the minimum value represented by the shared data part is 2. -30 That is, the minimum value represented by the second mantissa field of the shared data part is 1 + My / 2 y =1+0 / 2 2 =1; the maximum value represented by the shared data portion is 2. 32 *1.5, meaning the maximum value represented by the second mantissa field of the shared data portion is 1 + My / 2. y =1+2 y -2 / 2 y =1+2 / 2 2 =1.5. When the value of the second exponent field Ex is Ex==0, the value of the exponent representation of the shared data part is 1-Ebias, where Ebias=2. x-1 -1 = 26-1 -1 = 31, so the exponent of the shared data portion is represented by 1 - 31 = -30. The minimum value represented by the shared data portion is 2. -30 *(My / 2 y ) = 2 -30 *(1 / 2 2 ) = 2 -30 *2 -2 =2 -30 *0.25, the minimum exponent of the denormal represented by the shared data portion is 2. -30 *2 -2 =2 -32 The maximum value represented by the shared data portion is 2. -30 *(My / 2 y ) = 2 -30 *

[0098] (3 / 2 2 ) = 2 -30 *1.5*2 -1 =2 -31 *1.5=2 -30 *0.75, the maximum exponent of the denormal represented by the shared data portion is 2. -31 Therefore, the exponent range of the denormal represented by the shared data part is [-32:-31].

[0099] When the shared data portion of the floating-point number is in E5M3 format, x = 5, y = 3, and the value of the second exponent field Ex is [1,…,2], then… x -1] = [1,…,2] 5 When [-1] = [1,…,31], the exponent representation of the shared data portion is Ex-Ebias, where Ebias = 2. x-1 -1 = 2 5-1 -1 = 16 - 1 = 15, the exponent range of the normal represented by the shared data part is [1-15, ..., 31-15] = [-14, ..., 16] = [-14: 16]. At this time, the minimum value represented by the shared data part is 2. -14 That is, the minimum value represented by the second mantissa field of the shared data part is 1 + My / 2 y =1+0 / 2 3 =1; the maximum value represented by the shared data portion is 2. 16 *1.75, meaning the maximum value represented by the second mantissa field of the shared data portion is 1 + My / 2. y =1+2 y -2 / 2 y =1 + 6 / 2 3=1.75. When the value of the second exponent field Ex is 0, the value of the exponent representation of the shared data part is 1 - Ebias, where Ebias = 2. x-1 -1 = 2 5-1 -1 = 15, so the exponent of the shared data portion is represented by 1 - 15 = -14. The minimum value represented by the shared data portion is 2. -14 *(My / 2 y ) = 2 -14 *(1 / 2 3 ) = 2 -14 *2 -3 =2 -14 *0.125, the minimum exponent of the denormal represented by the shared data portion is 2. -14 *2 -3 =2 -17 The maximum value represented by the shared data portion is 2. -14 *(My / 2 y ) = 2 -14 *(7 / 2 3 ) = 2 -14 *1.75*2 -1 =2 -15 *1.75=2 -14 *0.875, the maximum exponent of the denormal represented by the shared data portion is 2. -15 Therefore, the exponent range of the denormal represented by the shared data part is [-17:-15].

[0100] When the shared data portion of a floating-point number is in E4M4 format, x = 4, y = 4, and the value of the second exponent field Ex is [1,…,2]. x -1] = [1,…,2] 4 When [-1] = [1,…,15], the exponent representation of the shared data portion is Ex-Ebias, where Ebias = 2. x-1 -1 = 2 4-1 -1 = 8 - 1 = 7. The exponent range of the normal represented by the shared data part is [1-7, ..., 15-7] = [-6, ..., 8] = [-6: 8]. At this time, the minimum value represented by the shared data part is 2. -6 That is, the minimum value represented by the second mantissa field of the shared data part is 1 + My / 2 y =1+0 / 2 4 =1; the maximum value represented by the shared data portion is 2. 8 *1.875, meaning the maximum value represented by the second mantissa field of the shared data portion is 1 + My / 2. y =1+2 y -2 / 2 y =1+14 / 24 =1.875. When the value of the second exponent field Ex is 0, the value of the exponent representation of the shared data part is 1 - Ebias, where Ebias = 2. x-1 -1 = 2 4-1 -1 = 7, so the exponent of the shared data portion is represented by 1 - 7 = -6. The minimum value represented by the shared data portion is 2. -6 *(My / 2 y ) = 2 -6 *(1 / 2 4 ) = 2 -6 *2 - 4 =2 -6 *0.0625, the minimum exponent of the denormal represented by the shared data portion is 2. -6 *2 -4 =2 -10 The maximum value represented by the shared data portion is 2. -6 *(My / 2 y ) = 2 -6 *(15 / 2 4 ) = 2 -6 *1.875*2 -1 =2 -7 *1.875=2 -6 *0.9375, the maximum exponent of the denormal represented by the shared data portion is 2. -7 Therefore, the exponent range of the denormal represented by the shared data portion is [-10:-7].

[0101] In the above embodiments, the floating-point number is represented by a shared data part and a variable data part. The shared data part includes a second exponent field Ex and a second mantissa field My. The second mantissa field My of the shared data part can be used to supplement the numerical precision that the third mantissa field of the variable data part can represent, thereby improving the numerical precision of the data that the floating-point number can represent and meeting the needs of AI training and inference.

[0102] In other embodiments, the shared data portion of the floating-point number can be represented as Ex[My / EEy], where Ex represents a second exponent field with a bit width of x bits, used to characterize the exponent of the shared data portion; My represents a second mantissa field with a bit width of y bits, used to characterize the mantissa of the shared data portion. EEy and My share the same bit position, that is, EEy represents a second mantissa field with a bit width of y bits, used to characterize the offset of the exponent of the shared data portion. For example, when the second exponent field Ex! = 0, the second mantissa field is My, used to characterize the mantissa of the shared data portion. When the second exponent field Ex == 0, the second mantissa field is EEy, used to characterize the offset of the exponent of the shared data portion. The shared data portion of the floating-point number has no sign bit. In one embodiment, the bit width of the shared data portion of the floating-point number can be 8, i.e., x + y = 8. When x = 6, y = 2; when x = 5, y = 3; when x = 4, y = 4. That is, the shared data part of the floating-point number can be represented in different formats such as E6M2, E5M3, and E4M4.

[0103] The second exponent field Ex is used to characterize the exponent of the shared data portion. Similar to the above embodiment, the exponent of the shared data portion is the difference between the value of the second exponent field Ex and the static offset Ebias, which can be expressed as Ex - Ebias. When the value of the second exponent field Ex of the shared data portion is not 0, the second mantissa field of the shared data portion is My, and the range of values ​​represented by the second mantissa field My is My = [0, ..., 2]. y -1]. When the second exponent field Ex of the shared data portion is 0, the second mantissa field of the shared data portion is EEy. In an optional embodiment, EEy can represent [0,…,2]. y -1] Total 2 y There are several values, where EEy is all 0, i.e., Ex == 0. When EEy == 0, the shared data part can be used to represent the zero value. Therefore, after excluding the case where EEy is all 0, the range of values ​​represented by the second mantissa field EEy can be EEy = [1, ..., 2]. y -1], the offset of the exponent in the shared data part is the difference between the value of the second mantissa field EEy and the offset EEbias in the mantissa field, which can be expressed as EEy-EEbias, where EEbias can be 1. The values ​​represented by the shared data part of the floating-point number are shown in Table 4.

[0104] Table 4

[0105] As shown in Table 4, unlike the above embodiments, when the value of the second exponent field Ex is Ex==0, EEy=[1,…,2 yWhen the value is -1, the shared data part of the floating-point number represents the value 2Ex - Ebias - (EEy - EEbias). At this time, the mantissa of the shared data part of the floating-point number is the default value 1. When the value of the second exponent field Ex is Ex == 0 and EEy == 0, the shared data part of the floating-point number represents the value zero.

[0106] When the second-order code field Ex = [1,…,2] x When the value of the shared data part of the floating-point number is -1, the value is normal. When the value of the second exponent field Ex is 0, the value of the shared data part of the floating-point number is denormal. The shared data part of the floating-point number can be represented in different formats such as E6M2, E5M3, and E4M4. The maximum and minimum values ​​of the data that the shared data part can represent in different formats are shown in Table 5.

[0107] Table 5

[0108] In Table 5, "scale" refers to the shared data portion of the floating-point number. Unlike the ExMy format floating-point numbers shown in Table 3, in Table 5, when the shared data portion of the floating-point number is in E6[M2 / EE2] format, x = 6, y = 2, and the value of the second exponent field Ex is Ex = 0, the exponent representation of the shared data portion is Ex - Ebias, where Ebias = 2. x-1 -1 = 2 6-1 -1 = 31, so the exponent of the shared data portion represents the value 0 - 31 = -31. The second mantissa field EEy represents a value in the range [1, ..., 2]. y -1], assuming EEbias = 1, the range of values ​​represented by the exponent offset EEy - EEbias in the shared data portion is [0,…,2]. y -2] = [0,…,2] 2 -2] = [0,…,2], therefore, the minimum value of the denormal represented by the shared data portion is 2. -31-2 =2 -33 The maximum value of the denormalization represented by the shared data portion is 2. -31-0 =2 -31 The exponent range of the denormal represented by the shared data part is [-33:-31].

[0109] When the shared data portion of a floating-point number is in E5[M3 / EE3] format, x = 5, y = 3, and the value of the second exponent field Ex is Ex = 0, the value of the exponent representation of the shared data portion is Ex - Ebias, where Ebias = 2. x-1 -1 = 2 5-1-1 = 15, so the exponent of the shared data portion represents a value of 0 - 15 = -15. The range of values ​​represented by the offset EEy - EEbias of the exponent in the shared data portion is [0, ..., 2]. y -2] = [0,…,2] 3 -2] = [0,…,6], therefore, the minimum value of the denormal represented by the shared data portion is 2. -15-6 =2 -21 The maximum value of the denormalization represented by the shared data portion is 2. - 15-0 =2 -15 The exponent range of the denormal represented by the shared data part is [-21:-15].

[0110] When the shared data portion of a floating-point number is in E4[M4 / EE4] format, x = 4, y = 4, and the value of the second exponent field Ex is Ex = 0, the value of the exponent representation of the shared data portion is Ex - Ebias, where Ebias = 2. x-1 -1 = 2 4-1 -1 = 7, so the exponent of the shared data portion represents a value of 0 - 7 = -7. The range of values ​​represented by the offset EEy - EEbias of the exponent in the shared data portion is [0, ..., 2]. y -2] = [0,…,2] 4 -2] = [0,…,14], therefore, the minimum value of the denormal represented by the shared data portion is 2. -7-14 =2 -21 The maximum value of the denormalization represented by the shared data portion is 2. -7- 0 =2 -7 The exponent range of the denormal represented by the shared data part is [-21:-7].

[0111] In the above embodiments, when the shared data portion is represented as Ex[My / EEy], if Ex == 0 and EEy == 0, the shared data portion can be used to represent a zero value. In other embodiments, when the shared data portion is represented as Ex[My / EEy], it can also represent a zero value. In this embodiment, the offset of the exponent of the shared data portion is the difference between the value of the second mantissa field EEy and the offset EEbias of the mantissa field, which can be represented as EEy - EEbias, where EEbias can be 0. The values ​​represented by the shared data portion of floating-point numbers are shown in Table 6.

[0112] Table 6

[0113] As shown in Table 6, unlike Table 4, when the value of the second exponent field Ex is Ex==0, EEy=[0,…,2 y When -1], the value represented by the shared data portion of the floating-point number is 2Ex-Ebias-(EEy-EEbias).

[0114] In this embodiment, the shared data portion of the floating-point number can be represented in different formats such as E6M2, E5M3, and E4M4. The maximum and minimum values ​​of the data that can be represented by the shared data portion in different formats are shown in Table 7.

[0115] Table 7

[0116] Since it does not represent a zero value, the range of values ​​represented by EEy can be [0,…,2]. y -1], assuming EEbias = 0, the range of values ​​represented by the exponent offset EEy - EEbias in the shared data portion is [0,…,2]. y Therefore, as shown in Table 7, when the shared data portion of a floating-point number is in E6[M2 / EE2] format, the range of values ​​represented by the exponent offset EEy-EEbias of the shared data portion is [0,…,2]. y -1] = [0,…,2] 2 -1] = [0,…,3], where the minimum value of the denormalization represented by the shared data portion can be 2. -31-3 =2 -34 The exponent range of the denormalized value represented by the shared data portion is [-34:-31]. When the shared data portion of a floating-point number is in E5[M3 / EE3] format, the value represented by the exponent offset EEy-EEbias of the shared data portion ranges from [0,…,2]. y -1] = [0,…,2] 3 -1] = [0,…,7], the minimum value of the denormalization represented by the shared data portion can be 2. -15-7 =2 -22 The exponent range of the denormalized portion represented by the shared data is [-22:-15]. When the shared data portion of a floating-point number is in E4 [M4 / EE4] format, the value represented by the exponent offset EEy-EEbias of the shared data portion ranges from [0,…,2]. y -1] = [0,…,2] 4 -1] = [0,…,15], the minimum value of the denormalization represented by the shared data portion can be 2. -7-15 =2 -22 The exponent range of the denormal represented by the shared data part is [-22:-7].

[0117] In the above embodiments, the shared data portion of the floating-point number includes a second exponent field Ex and a second mantissa field [My / EEy]. The second mantissa field [My / EEy] can be used to characterize the mantissa of the shared data portion and also to characterize the offset of the exponent of the shared data portion. By reusing the second mantissa field, it is possible to ensure that the numerical precision of the floating-point number is minimally affected, while increasing the encoding space of the exponent, thereby expanding the dynamic range of the numerical value represented by the floating-point number. For example, taking E5[M3 / EE3] as an example, the dynamic range of the exponent that E5[M3 / EE3] can represent can be expanded to [-21, 16].

[0118] The data format of the shared data portion of a floating-point number has been introduced above. The data format of the variable data portion of a floating-point number can be any of the following formats, but is not limited to: FP8, FP6, FP4, INT4, INT3, or INT2. Among them, FP8, FP6, and FP4 are floating-point formats. When the variable data portion adopts a floating-point format, it can include a second sign field, a third exponent field, and a third mantissa field. As shown in Table 8, the FP8 format represents an 8-bit floating-point number. FP8 format floating-point numbers can include E4M3 and E5M2 formats. The E4M3 format includes a 1-bit second sign field, a 4-bit third exponent field, and a 3-bit third mantissa field; the E5M2 format includes a 1-bit second sign field, a 5-bit third exponent field, and a 2-bit third mantissa field. FP6 format represents 6-bit floating-point numbers. FP6 format floating-point numbers can include E2M3 and E3M2 formats. E2M3 format floating-point numbers include a 1-bit second sign field, a 2-bit third exponent field, and a 3-bit third mantissa field; E3M2 format floating-point numbers include a 1-bit second sign field, a 3-bit third exponent field, and a 2-bit third mantissa field. FP4 format represents 4-bit floating-point numbers. FP4 format floating-point numbers can include E2M1 format. E2M1 format floating-point numbers include a 1-bit second sign field, a 2-bit third exponent field, and a 1-bit third mantissa field. INT4 format represents 4-bit integer data, INT3 format represents 3-bit integer data, and INT2 format represents 2-bit integer data.

[0119] Table 8

[0120] The floating-point numbers provided in this application can be applied not only to AI scenarios but also to other scenarios such as high-performance computing, and can be used for data computation, data storage, or data transfer. Correspondingly, this application also provides a floating-point number processing method, which can be executed by a computing device, which may include an encoder and a memory. For example, the computing device may be the computing device 100 shown in Figure 2. As shown in Figure 4, the method may include the following steps:

[0121] S401, the encoder obtains the first floating-point number.

[0122] The encoder can read the first floating-point number from the local memory of the computing device, or it can obtain the first floating-point number from another computing device via a network. The first floating-point number can be a normalized floating-point number as described above, and it can include a first sign field, a first exponent field, and a first mantissa field. For example, the data format used by the first floating-point number can be either the FP16 format or the BF16 format shown in Table 1. The FP16 format floating-point number includes a 1-bit first sign field, a 5-bit first exponent field, and a 10-bit first mantissa field; the BF16 format floating-point number includes a 1-bit first sign field, an 8-bit first exponent field, and a 7-bit first mantissa field.

[0123] S402, the encoder obtains the second floating-point number based on the first symbol field, the first exponent field and the first mantissa field.

[0124] The second floating-point number uses a data format that may include a shared data portion and a variable data portion. The shared data portion is determined by the data range to which the data represented by the first floating-point number belongs. This shared data portion is stored in a shared area of ​​memory and is shared by multiple data points within the data range to which the first floating-point number belongs. The second floating-point number represents the same data as the first floating-point number. The shared data portion may include a second exponent field and a second mantissa field.

[0125] After obtaining the first floating-point number, the encoder can determine the data represented by the first floating-point number and the data range to which the data belongs based on the first sign field, the first exponent field, and the first mantissa field of the first floating-point number. It then obtains the shared data portion corresponding to this data range, which is used by multiple data points within that range. The shared data portions corresponding to different data ranges can be pre-stored in memory, and the encoder can read the shared data portion corresponding to the data range to which the data belongs from memory; alternatively, the encoder can determine the shared data portion based on the data range to which the data belongs. After obtaining the shared data portion, the encoder can determine the variable data portion based on the first floating-point number and the shared data portion, thereby obtaining the second floating-point number.

[0126] In some embodiments, the first floating-point number may be in FP16 format, and the second floating-point number may be in ExMy format. The process of converting the first floating-point number to the second floating-point number, as shown in Figure 5, may include the following steps:

[0127] S501, the encoder determines the data range to which the data represented by the first floating-point number belongs.

[0128] The encoder can determine the data range to which the data represented by the first floating-point number belongs based on the first sign field, the first exponent field, and the first mantissa field of the first floating-point number.

[0129] S502, the encoder determines the maximum absolute value aMAX of multiple data points contained within the data range.

[0130] For example, a data range may contain 32 data points. The encoder can determine the absolute values ​​of the 32 data points within the data range to which the data represented by the first floating-point number belongs, and denote the maximum value among the 32 absolute values ​​as aMAX. For example, aMAX may be data in FP16 format.

[0131] S503, the encoder determines the first ratio of the absolute maximum value aMAX to the set maximum value MAX.

[0132] Here, the set maximum value MAX represents the maximum value that the data format used by the second floating-point number can represent. Assuming the data format of the second floating-point number includes a shared data portion ExMy and a variable data portion FP4, the maximum value that the shared data portion ExMy can represent is ExMy. MAX The maximum value that FP4 can represent in the variable data part is FP4. MAX Then the maximum value that this data format can represent is MAX = ExMy MAX *FP4 MAX Among them, FP4 MAX =6, taking E5M3 as an example, E5M3 MAX =2 16 *1.75. After determining the set maximum value MAX, the reciprocal of the set maximum value MAX, 1 / MAX, can be determined. Since the maximum value MAX that each data format can represent is fixed, 1 / MAX can be predetermined and stored in memory as a constant in the quantization operator.

[0133] The first ratio can be denoted as X. 真值 X 真值 = aMAX*(1 / MAX), X 真值 The value used to characterize the shared data portion of the second floating-point number, which is a positive number, X. 真值 It can be data in FP16 format.

[0134] S504, the encoder performs data format conversion on the first ratio to obtain the shared data part X of the second floating-point number.

[0135] Assume the shared data portion X uses the E5M3 format. The first ratio X 真值 This is FP16 format data. FP16 format data includes a 5-bit first-order code field and a 10-bit first-mantissa field, which can be represented as E5M10. The 5-bit first-order code field is used to represent X. 真值 The exponent, the 10-bit first mantissa field is used to represent X. 真值 The last digit. The encoder can convert X... 真值 The 5-bit first-order code field is used as the second-order code field E5 of the shared data part X, that is, based on X 真值 The exponent is used to determine the value of the second exponent field in the shared data portion. The encoder can be based on X. 真值 The value of the second mantissa field of the shared data portion is determined by the mantissa of the first mantissa field M10. For example, the encoder can perform a mantissa shift and rounding operation on the 10-bit first mantissa field M10 to obtain the second mantissa field M3 of the shared data portion X. This process can be represented as M10→M3. After determining the second exponent field E5 and the second mantissa field M3 of the shared data portion X, the shared data portion X is obtained. The rounding strategy can be a user-defined strategy, such as a truncation and rounding strategy. For example, assume the ratio X... 真值 It is binary data in FP16 format: 1-10101-1010101010 (E5M10), for X 真值 After truncating and rounding the first mantissa field, the resulting shared data portion X can be 10101-101 (E5M3).

[0136] S505, the encoder determines the reciprocal 1 / X of the shared data portion X.

[0137] When X! = 0, 1 / X can be calculated, and 1 / X can be data in FP16 format. When X == 0, then 1 / X is also 0.

[0138] S506, the encoder determines the variable data portion of the second floating-point number based on the first floating-point number and the reciprocal 1 / X of the shared data portion X.

[0139] The encoder can perform element-wise multiplication of the first floating-point number in FP16 format with 1 / X. Multiplying the first floating-point number by 1 / X is equivalent to performing a data scaling operation to obtain the data MUL, which can be data in FP16 format.

[0140] The encoder can perform data format conversion on the data MUL to obtain the variable data part of the second floating-point number. Assume the variable data part uses the FP4 format, i.e., E2M1 format. The encoder can use the value of the first symbol field of the data MUL as the value of the second symbol field of the variable data part, perform a round operation on the 5-bit first exponent field E5 of the data MUL to obtain the third exponent field E2 of the variable data part; this process can be represented as E5→E2. Similarly, it can perform a round operation on the 10-bit first mantissa field M10 of the data MUL to obtain the third mantissa field M1 of the variable data part; this process can be represented as M10→M1.

[0141] After determining the shared data portion of the variable data portion of the second floating-point number, the second floating-point number is obtained.

[0142] In other embodiments, the first floating-point number may be in BF16 format, and the second floating-point number may be in ExMy format. The process of converting the first floating-point number to the second floating-point number, as shown in Figure 6, may include the following steps:

[0143] S601, the encoder determines the data range to which the data represented by the first floating-point number belongs.

[0144] S602, the encoder determines the maximum absolute value aMAX of multiple data points contained within the data range.

[0145] For example, a data range may contain 32 data points. The encoder can determine the absolute values ​​of the 32 data points within the data range to which the data represented by the first floating-point number belongs, and denote the maximum value among the 32 absolute values ​​as aMAX. For example, aMAX may be data in BF16 format.

[0146] S603, the encoder determines the first ratio of the absolute maximum value aMAX to the set maximum value MAX.

[0147] Here, the set maximum value MAX represents the maximum value that the data format used for the second floating-point number can represent. The encoder can determine the reciprocal of the set maximum value MAX, 1 / MAX. Since the maximum value MAX that each data format can represent is fixed, 1 / MAX can be predetermined and stored in memory as a constant in the quantization operator.

[0148] The first ratio can be denoted as X. 真值 X 真值 = aMAX*(1 / MAX), X 真值 The value used to characterize the shared data portion of the second floating-point number, which is a positive number, X. 真值It can be data in BF16 format.

[0149] S604, the encoder determines the second exponent field of the shared data portion X based on the sum of the exponent of the first ratio and the first preset exponent, and determines the second mantissa field of the shared data portion X based on the mantissa of the first ratio.

[0150] The encoder can perform data format conversion on the first ratio to obtain the shared data portion X of the second floating-point number. Assume the shared data portion X uses the E5M3 format. First ratio X 真值 This is BF16 format data, which includes an 8-bit first-order code field and a 7-bit first-mantissa field, and can be represented as E8M7. The 8-bit first-order code field E8 is used to represent X. 真值 The exponent, the 7-bit first mantissa field M7 is used to represent X. 真值 The mantissa. Since BF16 format data includes an 8-bit first exponent field E8, directly truncating E8 to E5 would lead to overflow risk because 5 < 8. Therefore, a first preset exponent BFbias can be configured. The encoder can determine the sum of the exponent of the first ratio and the first preset exponent E8 + BFbias, and perform a round operation on E8 + BFbias to obtain the second exponent field E5 of the shared data part X. This process can be represented as E8 + BFbias → E5. In some embodiments, all data can use the same BFbias. For example, the value of BFbias can be 18, 20, or 22. In other embodiments, in AI training scenarios, a globally dynamic BFbias can be set for backpropagation gradient data, that is, the BFbias used by different network parameters in the AI ​​model can be different. For example, BFbias can be set dynamically in a coarse-grained manner, such as for three different datasets A / W / E in an AI training and inference scenario, where A represents the input data in the AI ​​model, W represents the weight parameters in the AI ​​model, and E represents the variance data in the AI ​​model. All datasets A, W, and E use the same BFbias. Alternatively, BFbias can be set dynamically in a fine-grained manner, for example, using the same BFbias for each tensor in the computation process, and different tensors using different BFbias.

[0151] The encoder can perform a round operation on the first mantissa field M7 (7 bits) to obtain the second mantissa field M3 of the shared data part X. This process can be represented as M7→M3. After determining the second exponent field E5 and the second mantissa field M3 of the shared data part X, the shared data part X of the second floating-point number is obtained.

[0152] S605, the encoder determines the reciprocal 1 / X of the shared data portion X.

[0153] When X! = 0, 1 / X can be calculated, and 1 / X can be data in BF16 format. When X == 0, then 1 / X is also 0.

[0154] S606, the encoder determines the variable data portion of the second floating-point number based on the first floating-point number and the reciprocal 1 / X of the shared data portion X.

[0155] The encoder can perform bit-by-bit multiplication of the first floating-point number in FP16 format with 1 / X to obtain the data MUL. The data MUL can be data in BF16 format, including an 8-bit first exponent field and a 7-bit first mantissa field.

[0156] The encoder can perform data format conversion on the data MUL to obtain the variable data part of the second floating-point number. Assume the variable data part uses the FP4 format, i.e., E2M1 format. The encoder can use the value of the first symbol field of the data MUL as the value of the second symbol field of the variable data part, perform a round operation on the 8-bit first exponent field E8 of the data MUL to obtain the third exponent field E2 of the variable data part; this process can be represented as E8→E2. Similarly, it can perform a round operation on the 7-bit first mantissa field M7 of the data MUL to obtain the third mantissa field M1 of the variable data part; this process can be represented as M7→M1.

[0157] After determining the shared data portion of the variable data portion of the second floating-point number, the second floating-point number is obtained.

[0158] In other embodiments, the first floating-point number may be in FP16 format, and the second floating-point number may be in Ex[MyEEy] format. The process of converting the first floating-point number to the second floating-point number, as shown in Figure 7, may include the following steps:

[0159] S701, the encoder determines the data range to which the data represented by the first floating-point number belongs.

[0160] S702, the encoder determines the maximum absolute value aMAX of multiple data points contained within the data range.

[0161] S703, the encoder determines the first ratio of the absolute maximum value aMAX to the set maximum value MAX.

[0162] The specific implementation process of the above steps S701 to S703 can be executed by parameters S501 to S503, and will not be repeated here.

[0163] S704, the encoder determines the second exponent field of the shared data portion X of the second floating-point number based on the exponent of the first ratio.

[0164] Assume the shared data portion X uses the E5[M3EE3] format. The first ratio X 真值 This is FP16 format data. FP16 format data includes a 5-bit first-order code field and a 10-bit first-mantissa field, which can be represented as E5M10. The 5-bit first-order code field is used to represent X. 真值 The exponent, the 10-bit first mantissa field is used to represent X. 真值 The last digit. The encoder can convert X... 真值 The 5-bit first-order code field is used as the second-order code field E5 of the shared data part X, that is, based on X 真值 The exponent is used to determine the second exponent field E5 of the shared data portion X.

[0165] S705, the encoder determines whether the value of the second-order code field of the shared data part X is the set value; if yes, then proceed to step S706, if no, then proceed to step S707.

[0166] The set value can be 0.

[0167] S706, the encoder determines the second mantissa field of the shared data portion X of the second floating-point number based on the exponent of the first ratio; then, step S708 is executed.

[0168] When the value of the second exponent field E5 of the shared data portion X is the set value of 0, that is, when the second exponent field E5 == 0, the second mantissa field EE3 of the shared data portion X is used to characterize the offset of the exponent of the shared data portion X. The encoder can be based on X 真值 The exponent of X is used to determine the second mantissa field EE3 of the shared data portion X.

[0169] S707, the encoder determines the second mantissa field of the shared data portion X of the second floating-point number based on the mantissa of the first ratio.

[0170] When the value of the second-order code field E5 of the shared data part X is not the set value of 0, the encoder can base it on X. 真值 The second mantissa field M3 of the shared data portion X is determined by the mantissa of X. For example, the encoder can determine the second mantissa field M3 of X. 真值 The first mantissa field M10 of the 10 bits is rounded to obtain the second mantissa field M3 of the shared data part X. This process can be represented as M10→M3.

[0171] After determining the second exponent field E5 and the second mantissa field M3 of the shared data part X, the shared data part X is obtained.

[0172] S708, the encoder determines the reciprocal 1 / X of the shared data portion X.

[0173] When X! = 0, 1 / X can be calculated, and 1 / X can be data in FP16 format. When X == 0, then 1 / X is also 0.

[0174] S709, the encoder determines the variable data portion of the second floating-point number based on the first floating-point number and the reciprocal 1 / X of the shared data portion X.

[0175] The specific implementation process of step S709 can be executed with reference to step S506, and will not be described in detail here.

[0176] After determining the shared data portion of the variable data portion of the second floating-point number, the second floating-point number is obtained.

[0177] In other embodiments, the first floating-point number may be in BF16 format, and the second floating-point number may be in Ex[MyEEy] format. The process of converting the first floating-point number to the second floating-point number, as shown in Figure 8, may include the following steps:

[0178] S801, the encoder determines the data range to which the data represented by the first floating-point number belongs.

[0179] S802, the encoder determines the maximum absolute value aMAX of multiple data points contained within the data range.

[0180] S803, the encoder determines the first ratio of the absolute maximum value aMAX to the set maximum value MAX.

[0181] The specific implementation process of the above steps S801 to S803 can be executed by parameters S601 to S603, and will not be repeated here.

[0182] S804, the encoder determines the second exponent field of the shared data portion X of the second floating-point number based on the sum of the exponent of the first ratio and the first preset exponent.

[0183] Assume the shared data portion X uses the E5[M3EE3] format. The first ratio X 真值 This is BF16 format data, which includes an 8-bit first-order code field and a 7-bit first-mantissa field, and can be represented as E8M7. The 8-bit first-order code field is used to represent X. 真值 The exponent, the 7-bit first mantissa field is used to represent X. 真值The mantissa. As shown in the embodiment in Figure 6, a first preset exponent BFbias can be configured. The encoder can determine the sum of the exponent of the first ratio and the first preset exponent E8+BFbias, and perform a round operation on E8+BFbias to obtain the second exponent field E5 of the shared data part X. This process can be represented as E8+BFbias→E5.

[0184] S805, the encoder determines whether the value of the second-order code field of the shared data part X is the set value; if yes, then proceed to step S806, if no, then proceed to step S807.

[0185] The set value can be 0.

[0186] S806, the encoder determines the second mantissa field of the shared data portion X of the second floating-point number based on the sum of the exponent of the first ratio and the first preset exponent; then, step S808 is executed.

[0187] When the value of the second exponent field E5 of the shared data portion X is the set value of 0, that is, when the second exponent field E5 == 0, the second mantissa field EE3 of the shared data portion X is used to characterize the offset of the exponent of the shared data portion X. The encoder can be based on X 真值 The exponent of the first preset exponent, E8 + BFbias, is used to determine the second mantissa field EE3 of the shared data portion X. This process can resolve the issue that the data range represented by E8 exceeds the data range represented by E5. If the data represented by E8 is -20, which exceeds the data range that E5 - Ebias can represent, then the exponent exceeding the data range can be represented by the second mantissa field EE3. If the data represented by E8 still exceeds the data range that E5 - Ebias - (EE3 - EEbias) can represent, the shared data portion X can be set to 0, which is equivalent to the shared data portion X underflowing.

[0188] S807, the encoder determines the second mantissa field of the shared data portion X of the second floating-point number based on the mantissa of the first ratio.

[0189] When the value of the second-order code field E5 of the shared data part X is not the set value of 0, the encoder can base it on X. 真值 The second mantissa field M3 of the shared data portion X is determined by the mantissa of X. For example, the encoder can determine the second mantissa field M3 of X. 真值 The first mantissa field M7 of the 7 bits is rounded to obtain the second mantissa field M3 of the shared data part X. This process can be represented as M7→M3.

[0190] After determining the second exponent field E5 and the second mantissa field M3 of the shared data part X, the shared data part X is obtained.

[0191] S808, the encoder determines the reciprocal 1 / X of the shared data portion X.

[0192] When X! = 0, 1 / X can be calculated, and 1 / X can be data in FP16 format. When X == 0, then 1 / X is also 0.

[0193] S809, the encoder determines the variable data portion of the second floating-point number based on the first floating-point number and the reciprocal 1 / X of the shared data portion X.

[0194] The specific implementation process of step S809 can be executed with reference to step S606, and will not be described in detail here.

[0195] After determining the shared data portion of the variable data portion of the second floating-point number, the second floating-point number is obtained.

[0196] After converting the first floating-point number to a second floating-point number, the computing unit in the computing device can use the second floating-point number to participate in the computing task. The second floating-point number is an FMX format floating-point number. The computing unit can use a hardware architecture that supports FMX format floating-point number calculations to perform matrix operations. The hardware architecture can include two forms: single-sided FMX and dual-sided FMX. Single-sided FMX supports matrix operations between FMX format floating-point numbers and other floating-point numbers of other formats, while dual-sided FMX supports matrix operations between FMX format floating-point numbers.

[0197] Taking the matrix multiplication engine in inner product form as an example, each inner product engine in the matrix multiplication engine can be used to perform the inner product operation of two vectors A[] and B[], resulting in ∑A[]*B[]. Here, vector A[] can include one or more floating-point numbers, and vector B[] can also include one or more floating-point numbers. Suppose that vector A[] includes the second floating-point number mentioned above, and vector B[] includes the third floating-point number.

[0198] In some embodiments, both the third and second floating-point numbers are floating-point numbers in FMX format. The second floating-point number includes a shared data portion and a variable data portion, with the shared data portion in ExMy format. The third floating-point number also includes a shared data portion and a variable data portion, with the shared data portion in ExMy format, and can be applied to a matrix multiplication engine with two-sided FMX. The calculation unit can perform a multiplication calculation using the second and third floating-point numbers. The sum of the exponents of the shared data portions of the second and third floating-point numbers is used as the first part of the exponent of the calculation result; the product of the mantissas of the shared data portions of the second and third floating-point numbers is used as the first part of the mantissa of the calculation result; and the calculation result is obtained based on the first part of the exponent, the first part of the mantissa, and the product of the variable data portions of the second and third floating-point numbers.

[0199] For example, the internal structure of the calculation unit 113 shown in FIG2 can be as shown in FIG9. The calculation unit may include a shared data operation module for performing operations on the shared data part of the second floating-point number and the shared data part of the third floating-point number. The shared data part of the second floating-point number can be represented as Ex2My2, and the shared data part of the third floating-point number can be represented as Ex3My3. After the shared data part Ex2My2 of the second floating-point number and the shared data part Ex3My3 of the third floating-point number are processed by the shared data operation module, the first part exponent E1 and the first part mantissa M1 can be obtained. As shown in FIG10, the shared data operation module may include a decoding operation module and a scaling operation module. The decoding operation module is used to split the input Ex2My2 into Ex2 and My2, and the input Ex3My3 into Ex3 and My3. The scaling operation module is used to calculate the product of the shared data parts Ex2My2 and Ex3My3, which includes the first part exponent E1 and the first part mantissa M1. The product operation module can determine the sum of Ex2-Ebias2 and Ex3-Ebias3 of the shared data part of the second floating-point number and Ex3-Ebias3 of the shared data part of the third floating-point number, Ex2-Ebias2+Ex3-Ebias3, to obtain the first part of the exponent E1, E1 = Ex2-Ebias2+Ex3-Ebias3, where Ebias2 = 2. x2-1 -1, Ebias3 = 2 x3-1 -1. The product operation module can also determine the product My2*My3 of the shared data part of the second floating-point number and the My3 of the shared data part of the third floating-point number, and use My2*My3 as the first part mantissa M1. Combining the first part exponent E1 with the first part mantissa M1, the shared data 2 can be obtained. E1 *(1.M1).

[0200] As shown in Figure 9, the computation unit may also include a multiplier, used to perform operations on the variable data parts of multiple floating-point numbers in vector A[] and vector B[] to obtain the vector inner product. Taking the second floating-point number in vector A[] and the third floating-point number in vector B[] as examples, the variable data part of the second floating-point number can be represented as En2Mz2, and the variable data part of the third floating-point number can be represented as En3Mz3. The multiplier can perform exponential addition (add) on the value of the third exponent field En2 of the variable data part of the second floating-point number and the value of the third exponent field En3 of the variable data part of the third floating-point number, obtaining an exponent sum, which is then used as the exponent of the vector inner product. When both vector A[] and vector B[] contain multiple floating-point numbers, the values ​​of the exponent fields of the variable data parts of the corresponding pairs of floating-point numbers in vector A[] and vector B[] can be added to obtain multiple exponent sums. The resulting exponent sums are then subjected to exponent maximization (max), also known as exponent alignment. This involves determining the maximum value among the multiple exponent sums and changing all exponent sums to this maximum value. Simultaneously, the mantissa product needs to be shifted, and this maximum value is used as the exponent of the vector inner product. The multiplier can perform mantissa multiplication (mul) between the value of the third mantissa field Mz2 of the variable data part of the second floating-point number and the value of the third mantissa field Mz3 of the variable data part of the third floating-point number to obtain the mantissa product. The mantissa product is then shifted in conjunction with the exponent alignment process described above. For example, during the process of changing the exponent sum to the maximum value, for every 1 increase in the exponent sum, the decimal point of the corresponding mantissa product is shifted to the left. Then, the mantissas are accumulated to obtain the mantissa of the vector inner product. The above process yields the exponent and mantissa of the vector dot product. Multiplying the value of the second sign field of the second floating-point number's variable data part with the value of the second sign field of the third floating-point number's variable data part gives the sign of the vector dot product, thus obtaining the vector dot product ∑A[]*B[]. The shared data 2 E1 Multiplying *(1.M1) into the vector dot product ∑A[]*B[] means adding the first part of the exponent E1 to the exponent of the vector dot product, and multiplying the first part of the mantissa M1 into the mantissa of the vector dot product, yields the result (∑A[]*B[])*2. E1 *(1.M1). The sign of the vector inner product can be used as the sign of the calculation result. In some embodiments, the sum of the calculation result and the set reference value C can be converted into a floating-point number in FP16 format, FP32 format, or other formats for storage.

[0201] In other embodiments, both the third and second floating-point numbers are floating-point numbers in FMX format. The second floating-point number includes a shared data portion and a variable data portion, with the shared data portion in Ex[MyEEy] format. The third floating-point number also includes a shared data portion and a variable data portion, with the shared data portion also in Ex[MyEEy] format. This can be applied to a matrix multiplication engine with two-sided FMX. The computation unit can multiply the second and third floating-point numbers to obtain the calculation result.

[0202] Similar to Figure 9, the computation unit may include a shared data operation module for performing operations on the shared data portions of the second and third floating-point numbers. The difference lies in that the shared data portion of the second floating-point number can be represented as Ex2[My2EEy2], and the shared data portion of the third floating-point number can be represented as Ex3[My3EEy3]. After processing by the shared data operation module, the second part exponent E2 and the second part mantissa M2 can be obtained.

[0203] As shown in Figure 11, the shared data operation module may include a decoding operation module and a product operation module. The decoding operation module is used to determine the exponent E22 and mantissa M22 of the shared data part of the second floating-point number, and the exponent E33 and mantissa M33 of the shared data part of the third floating-point number. The decoding operation module selects the output result through a multiplexer (MUX), thereby achieving the following: when the shared data part Ex2 of the second floating-point number is 0, the exponent E22 of the shared data part of the second floating-point number is Ex2 - Ebias2 - (EEy2 - EEbias2), and the mantissa M22 of the shared data part of the second floating-point number is 0; when the shared data part Ex2 is 0, the exponent E22 of the shared data part of the second floating-point number is Ex2 - Ebias2, and the mantissa M22 of the shared data part of the second floating-point number is My2. When the shared data part Ex3 of the third floating-point number is 0, the exponent E33 of the shared data part of the third floating-point number is Ex3 - Ebias3 - (EEy3 - EEbias3), and the mantissa M33 of the shared data part of the third floating-point number is 0; when the shared data part Ex3! is 0, the exponent E33 of the shared data part of the third floating-point number is Ex3 - Ebias3, and the mantissa M33 of the shared data part of the third floating-point number is My3.

[0204] The product operation module is used to calculate the product of the shared data parts Ex2[My2EEy2] and Ex3[My3EEy3], which includes the second part exponent E2 and the second part mantissa M2. The decoding operation module inputs the exponent E22 and mantissa M22 of the shared data part of the second floating-point number, and the exponent E33 and mantissa M33 of the shared data part of the third floating-point number into the product operation module. The product operation module can determine the sum of the exponent E22 of the shared data part of the second floating-point number and the exponent E33 of the shared data part of the third floating-point number, E22+E33, to obtain the second part exponent E2, E2=E22+E33. The product operation module can also determine the product M22*M33 of the mantissa M22 of the shared data part of the second floating-point number and the mantissa M33 of the shared data part of the third floating-point number, M22*M33, to obtain the second part mantissa M2. Combining the second part exponent E2 and the second part mantissa M2, the shared data 2 can be obtained. E2 *(1.M2).

[0205] Similar to Figure 9, the computation unit may also include a multiplier to perform operations on the variable data parts of multiple floating-point numbers in vector A[] and vector B[], obtaining the vector inner product ∑A[]*B[]. This will share the data 2. E2 Multiplying *(1.M2) into the vector dot product ∑A[]*B[] means adding the second part of the exponent E2 to the exponent of the vector dot product, and multiplying the second part of the mantissa M2 into the mantissa of the vector dot product, yields the result (∑A[]*B[])*2. E2 *(1.M2). The sign of the vector inner product can be used as the sign of the calculation result. In some embodiments, the sum of the calculation result and the set reference value C can be converted into a floating-point number in FP16 format, FP32 format, or other formats for storage.

[0206] In other embodiments, the second floating-point number is an FMX format floating-point number. The second floating-point number includes a shared data portion and a variable data portion, with the shared data portion in ExMy format. The third floating-point number also includes a shared data portion and a variable data portion, with the shared data portion in Ex format, such as E8 format, which can be applied to a single-sided FMX matrix multiplication engine. The calculation unit can perform a multiplication operation using the second and third floating-point numbers. The sum of the exponents of the shared data portions of the second and third floating-point numbers is used as the third part of the exponent of the calculation result; the product of the mantissa of the shared data portion of the second and third floating-point numbers is used as the third part of the mantissa of the calculation result; and the calculation result is obtained based on the third part of the exponent, the third part of the mantissa, and the product of the variable data portions of the second and third floating-point numbers.

[0207] Similar to Figure 9, the computation unit may include a shared data operation module for performing operations on the shared data portions of the second and third floating-point numbers. The difference lies in that the shared data portion of the third floating-point number can be represented as E8, and the shared data portion of the second floating-point number can be represented as Ex2My2. After processing by the shared data operation module, the shared data portions Ex2My2 and E8 of the second and third floating-point numbers yield the third part exponent E3 and the third part mantissa M3. As shown in Figure 12, the shared data operation module may include a decoding operation module and a product operation module. The decoding operation module splits the input Ex2My2 into Ex2 and My2. The product operation module calculates the product of the shared data portions Ex2My2 and Ex3My3, which includes the third part exponent E3 and the third part mantissa M3. The product operation module can determine the sum of the exponents Ex2-Ebias2 of the shared data part of the second floating-point number and E8-Ebias3 of the shared data part of the third floating-point number, Ex2-Ebias2+E8-Ebias3, to obtain the third part exponent E3, E3 = Ex2-Ebias2+E8-Ebias3, where Ebias2 = 2. x2-1 -1, Ebias3 = 2 8-1 -1. The product operation module can also determine the product of the mantissa My2 of the shared data part of the second floating-point number and the preset mantissa to obtain the third mantissa M3. The preset mantissa can be 1, therefore the second mantissa M3 = My2. Combining the third exponent E3 and the third mantissa M3 yields the shared data 2. E3 *(1.M3).

[0208] Similar to Figure 9, the computation unit may also include a multiplier to perform operations on the variable data parts of multiple floating-point numbers in vector A[] and vector B[], obtaining the vector inner product ∑A[]*B[]. This will share the data 2. E3 Multiplying *(1.M3) into the vector dot product ∑A[]*B[] means adding the third part exponent E3 to the exponent of the vector dot product, and multiplying the third part mantissa M3 into the mantissa of the vector dot product, yields the result (∑A[]*B[])*2. E3 *(1.M3). The sign of the vector inner product can be used as the sign of the calculation result. In some embodiments, the sum of the calculation result and the set reference value C can be converted into a floating-point number in FP16 format, FP32 format, or other formats for storage.

[0209] In other embodiments, the second floating-point number is an FMX format floating-point number. The second floating-point number includes a shared data portion and a variable data portion, with the shared data portion in Ex[MyEEy] format. The third floating-point number also includes a shared data portion and a variable data portion, with the shared data portion in Ex format, such as E8 format, which can be applied to a single-sided FMX matrix multiplication engine. The computation unit can multiply the second floating-point number and the third floating-point number to obtain the calculation result.

[0210] Similar to Figure 9, the computation unit may include a shared data operation module for performing operations on the shared data portions of the second and third floating-point numbers. The difference lies in that the shared data portion of the third floating-point number can be represented as E8, and the shared data portion of the second floating-point number can be represented as Ex2[My2EEy2]. After processing by the shared data operation module, the shared data portions Ex2[My2EEy2] and E8 of the second and third floating-point numbers yield the fourth part exponent E4 and the fourth part mantissa M4. As shown in Figure 13, the shared data operation module may include a decoding operation module and a product operation module. The decoding module is used to determine the exponent E22 and mantissa M22 of the shared data part of the second floating-point number. The decoding module selects the output result through a multiplexer (MUX) to achieve the following: when the shared data part Ex2 of the second floating-point number is 0, the exponent E22 of the shared data part of the second floating-point number is Ex2 - Ebias2 - (EEy2 - EEbias2), and the mantissa M22 of the shared data part of the second floating-point number is 0; when the shared data part Ex2 is 0, the exponent E22 of the shared data part of the second floating-point number is Ex2 - Ebias2, and the mantissa M22 of the shared data part of the second floating-point number is My2.

[0211] The decoding module inputs the exponent E22 and mantissa M22 of the shared data portion of the second floating-point number into the product module. The product module calculates the product of the shared data portion Ex2[My2EEy2] and E8, which includes the fourth part exponent E4 and the fourth part mantissa M4. The product module can determine the sum of the exponent E22 of the shared data portion of the second floating-point number and the exponent E8-Ebias3 of the shared data portion of the third floating-point number, E22+E8-Ebias3, to obtain the fourth part exponent E4, E4=E22+E8-Ebias3, where Ebias3=2 8- 1-1. The product operation module can also determine the product of the mantissa M22 of the shared data part of the second floating-point number and the preset mantissa to obtain the fourth mantissa M4, where the preset mantissa can be 1, so the fourth mantissa M4 = M22. Combining the fourth exponent E4 and the fourth mantissa M4, the shared data 2 can be obtained. E4 *(1.M4).

[0212] Similar to Figure 9, the computation unit may also include a multiplier to perform operations on the variable data parts of multiple floating-point numbers in vector A[] and vector B[], obtaining the vector inner product ∑A[]*B[]. This will share the data 2. E4 Multiplying *(1.M4) into the vector dot product ∑A[]*B[] means adding the fourth part exponent E4 to the exponent of the vector dot product, and multiplying the fourth part mantissa M4 into the mantissa of the vector dot product, yields the result (∑A[]*B[])*2. E4 *(1.M4). The sign of the vector inner product can be used as the sign of the calculation result. In some embodiments, the sum of the calculation result and the set reference value C can be converted into a floating-point number in FP16 format, FP32 format, or other formats for storage.

[0213] The matrix multiplication engine of single-sided FMX can basically meet the data accuracy requirements of AI training and inference scenarios in a variety of situations.

[0214] The matrix multiplication operation described above can be applied to AI model training or inference scenarios. For example, as shown in Figure 14, in the forward propagation process of an AI model training scenario, the input data A of the (i-1)th training round... i-1 [M,K] and the weight parameters W of the AI ​​model i Matrix multiplication can be performed on [K,N] to obtain A. i [M,N]. Where A i-1 [M,K] can be understood as an M-row, K-column matrix, W i [K,N] can be understood as a matrix with K rows and N columns, A i [M,N] can be understood as an M-row, N-column matrix, A i-1 [M,K]、W i [K,N] and A i The data in [M,N] can be floating-point numbers in either FP16 or BF16 format.

[0215] In some embodiments, to improve computational efficiency, input data A i-1 [M,K] can be quantized to convert the data format, transforming it into the FMX format floating-point number FMX*[M,K] provided in this application embodiment. Q], weight parameter W i [K,N] can also be quantized, and converted into a floating-point number FMX*[K] in the FMX format provided in this application embodiment. Q [,N], then use FMX*[M,K] Q ] and FMX*[K Q [,N], through the matrix multiplication engine of the two-sided FMX provided in the embodiments of this application, A is obtained by performing multiplication operations. i [M,N].

[0216] During the reverse propagation process, input data A i-1 After transposing [M,K], it can be compared with the variance data E. i Perform matrix multiplication on [M,N] to obtain △W i [K,N]. In some embodiments, A i-1 A obtained after transposing [M,K] i-1 T Quantization can be performed, converting the data into a floating-point number FMX*[K,M] in the FMX format provided in this application embodiment. Q Variance data E i [M,N] can also be quantized, and converted into a floating-point number FMX*[M] in the FMX format provided in this application embodiment. Q [,N], then use FMX*[K,M] Q ] and FMX*[M Q [,N], by performing a multiplication operation using the dual-sided FMX matrix multiplication engine provided in this application embodiment, △W is obtained. i [K,N]. Using △W i [K,N] represents the weight parameter W. i After optimization, the updated weight parameters W can be obtained. i [K,N].

[0217] During the backpropagation process, the weight parameter W i After transposing [K,N], it can be compared with the variance data E. i Perform matrix multiplication on [M,N] to obtain the updated A. i-1 [M,K]. In some embodiments, the weighting parameter W i W obtained after transposing [K,N] i T Quantization can be performed, converting the data into a floating-point number FMX*[N] in the FMX format provided in this application embodiment. Q [,K], variance data E i[M,N] can also be quantized, and converted into a floating-point number FMX*[M,N] in the FMX format provided in this application embodiment. Q Then use FMX*[N Q [,K] and FMX*[M,N] Q The matrix multiplication operation is performed using the dual-sided FMX matrix multiplication engine provided in this application embodiment to obtain E. i-1 [M,K].

[0218] In the above embodiments, the input data A, weight parameter W, and variance data E of the input matrix multiplication engine can all include a shared data portion and a variable data portion. The shared data portion of the input data A, weight parameter W, and variance data E can all adopt the ExMy format or Ex[MyEEy]. Matrix multiplication operations can be implemented using the two-sided FMX matrix multiplication engine provided in the embodiments of this application.

[0219] In some embodiments of the application scenario shown in Figure 14, during the forward propagation process, input data A... i-1 [M,K] can be quantized, and data format conversion is performed through quantization to convert it into a floating-point number FMX*[M,K] in the FMX format provided in this application embodiment. Q ], weight parameter W i [K,N] can also be quantized, converting it to another floating-point format MX*[K]. Q The floating-point number format [,N] can include a shared data portion and a variable data portion. The shared data portion can use the Ex format, such as E8 format. Then, use FMX*[M,K] to... Q ] and MX*[K Q [,N], A is obtained by performing a multiplication operation using the single-sided FMX matrix multiplication engine provided in the embodiments of this application. i [M,N].

[0220] During the reverse propagation process, A i-1 A obtained after transposing [M,K] i-1 T Quantization can be performed, converting the data into another floating-point format MX*[K,M]. Q This format of floating-point numbers can include a shared data portion and a variable data portion. The shared data portion can use Ex format, such as E8 format. Variance data E i [M,N] can also be quantized, and converted into a floating-point number FMX*[M] in the FMX format provided in this application embodiment. Q [,N], then use MX*[K,M] Q ] and FMX*[M Q[,N], by performing a multiplication operation using the matrix multiplication engine of the single-sided FMX provided in this application embodiment, △W is obtained. i [K,N].

[0221] During the backpropagation process, the weight parameter W i W obtained after transposing [K,N] i T Quantization can be performed, converting the number to another floating-point format MX*[N]. Q [,K], this format of floating-point numbers can include a shared data portion and a variable data portion. The shared data portion can use Ex format, such as E8 format. Variance data E i [M,N] can also be quantized, and converted into a floating-point number FMX*[M,N] in the FMX format provided in this application embodiment. Q ], then use MX*[N Q [,K] and FMX*[M,N] Q The matrix multiplication operation is performed using the single-sided FMX matrix multiplication engine provided in this application embodiment to obtain A. i-1 [M,K].

[0222] In this embodiment, the input data A and variance data E used in the forward propagation process can both include shared data portions and variable data portions. The shared data portions of the input data A and variance data E used in the forward propagation process can both adopt the ExMy format or Ex[MyEEy]. Similarly, the input data A and weight parameter W used in the backward propagation process can both include shared data portions and variable data portions. The shared data portions of the input data A and weight parameter W used in the backward propagation process can both adopt the Ex format, such as E8 format. Matrix multiplication can be implemented using the single-sided FMX matrix multiplication engine provided in this embodiment.

[0223] The above embodiments describe the process of converting a first floating-point number in FP16 or BF16 format into a second floating-point number in FMX format provided in this application embodiment, and then using the second floating-point number to participate in the computation task. In some other embodiments, the second floating-point number in FMX format provided in this application embodiment can also be converted into a first floating-point number, and this process can be performed by a decoder in a computing device. As shown in Figure 15, this process may include the following steps:

[0224] S1501, the decoder obtains the second floating-point number.

[0225] The decoder can read the second floating-point number from the local memory of the computing device, or it can obtain the second floating-point number from another computing device via a network. The second floating-point number may include a shared data portion and a variable data portion. The shared data portion specifically includes a second exponent field and a second mantissa field. Multiple data points within the data range to which the second floating-point number belongs contain the same shared data portion. The data format of the shared data portion can be ExMy format or Ex[MyEEy] format.

[0226] When the data format of the shared data part is ExMy format, the second exponent field is used to represent the exponent of the shared data part, and the second mantissa field is used to represent the mantissa of the shared data part.

[0227] When the shared data portion is in Ex[MyEEy] format, the second exponent field is used to represent the exponent of the shared data portion. When the value of the second exponent field is a set value, the second mantissa field is used to represent the offset of the exponent of the shared data portion. When the value of the second exponent field is not a set value, the second mantissa field is used to represent the mantissa of the shared data portion. For example, the set value can be 0.

[0228] S1502, the decoder decodes the second floating-point number to obtain the first floating-point number.

[0229] The first floating-point number uses a data format that includes a first sign field, a first exponent field, and a first mantissa field. The first floating-point number and the second floating-point number represent the same data.

[0230] After obtaining the second floating-point number, the decoder can determine the first sign field of the first floating-point number based on the sign of the variable data portion of the second floating-point number. In some embodiments, the variable data portion of the second floating-point number may include a second sign field, a third exponent field, and a third mantissa field. The decoder can use the value of the second sign field of the variable data portion of the second floating-point number as the value of the first sign field of the first floating-point number.

[0231] The decoder can determine the first exponent field and the first mantissa field of the first floating-point number based on the product of the variable data part and the shared data part of the second floating-point number.

[0232] In some embodiments, when the data format of the shared data portion is ExMy, assuming the value of the second exponent field of the shared data portion of the second floating-point number is Ex, and the exponent of the shared data portion of the second floating-point number is Ex-Ebias, the exponent of the variable data portion of the second floating-point number is obtained as Eb based on the value of the third exponent field of the variable data portion of the second floating-point number. The decoder can obtain the value of the first exponent field of the first floating-point number based on the sum of the exponents Ex-Ebias and Eb of the shared data portion of the second floating-point number, Ex-Ebias+Eb. Assuming the value of the second mantissa field of the shared data portion of the second floating-point number is My, i.e., the mantissa of the shared data portion of the second floating-point number is My, and the mantissa of the variable data portion of the second floating-point number is Mb, the decoder can obtain the value of the first mantissa field of the first floating-point number based on the product My*Mb of the mantissa My and the mantissa Mb of the shared data portion of the second floating-point number.

[0233] In other embodiments, when the data format of the shared data portion is Ex[MyEEy], assuming the value of the second exponent field of the shared data portion of the second floating-point number is Ex, when Ex! = 0, the second mantissa field of the shared data portion of the second floating-point number is used to characterize the mantissa of the shared data portion. The exponent of the shared data portion of the second floating-point number is Ex - Ebias, and the exponent of the variable data portion of the second floating-point number is Eb. The decoder can obtain the value of the first exponent field of the first floating-point number from Ex - Ebias + Eb. The mantissa of the shared data portion of the second floating-point number is My, and the mantissa of the variable data portion of the second floating-point number is Mb. The decoder can obtain the value of the first mantissa field of the first floating-point number from My * Mb. When Ex == 0, the second mantissa field of the shared data portion of the second floating-point number is used to characterize the offset of the exponent of the shared data portion. Assuming the value of the second exponent field of the shared data portion of the second floating-point number is Ex, the value of the second mantissa field of the shared data portion of the second floating-point number is EEy, the exponent of the shared data portion of the second floating-point number is Ex - Ebias - (EEy - EEbias), and the exponent of the variable data portion of the second floating-point number is Eb, the decoder can obtain the value of the first exponent field of the first floating-point number from the sum of the exponents of the shared data portion of the second floating-point number, Ex - Ebias - (EEy - EEbias), and the exponent of the variable data portion of the second floating-point number, Eb. The mantissa of the shared data portion of the second floating-point number is a set mantissa, and the mantissa of the variable data portion of the second floating-point number is Mb. The decoder can obtain the value of the first mantissa field of the first floating-point number from the product of the set mantissa and the mantissa Mb. Assuming the set mantissa is 1, the decoder can obtain the value of the first mantissa field of the first floating-point number from 1 * Mb.

[0234] In some embodiments, after the decoder decodes the second floating-point number to obtain the first floating-point number, the computing unit can use the first floating-point number to participate in the calculation task and obtain the calculation result. For example, the decoder can transmit the decoded second floating-point number to the computing unit in the computing device. The computing unit receives the second floating-point number and performs corresponding calculations to obtain the calculation result. The data format of the calculation result can be the same as the data format of the first floating-point number, and may include a first sign field, a first exponent field, and a first mantissa field. The first sign field indicates the sign of the calculation result, the first exponent field indicates the exponent of the calculation result, and the first mantissa field indicates the mantissa of the calculation result. The calculation result can be directly saved to memory, or it can be converted to the FMX format floating-point number provided in this application embodiment by the encoder using the method shown in FIG4, and then saved to memory, or transmitted to other computing devices. This application embodiment can use a limited bit width to express floating-point numbers, saving the total bit width of the floating-point number. This floating-point number can be used for data storage or data transfer, saving the cost of data storage or data transfer.

[0235] Based on the same design concept as the above-described method embodiments, this application also provides a computing device. This computing device may include an encoder and a memory. The memory stores a second floating-point number, and the encoder can be used to execute the floating-point number processing method shown in FIG4. In some embodiments, the computing device may further include a computing unit, which can be used to perform a computing task using the second floating-point number and a third floating-point number. In some embodiments, the computing device may further include a decoder, which can be used to execute the floating-point number processing method shown in FIG15. Exemplarily, the internal structure of this computing device can be seen with reference to FIG2. This computing device can be used to implement the functions of the above-described method embodiments, and therefore can achieve the beneficial effects of the above-described method embodiments.

[0236] Based on the same design concept as the above method embodiments, this application also provides a computing chip that can be applied to any computing device. This computing chip can be used to implement the functions of the computing unit in the above method embodiments, thus achieving the beneficial effects of the above method embodiments.

[0237] In some embodiments, the structure of the computing chip 1600 can be as shown in FIG16, including a computing unit 1601 and an interface 1602 connected to the computing unit 1601. The computing unit 1601 and the interface 1602 can be interconnected via a bus, which can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0238] The computing unit 1601 may be a digital signal processor (DSP), ASIC, field programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, or other specific integrated circuit.

[0239] Interface 1602 can be used to obtain a second floating-point number and a third floating-point number, and calculation unit 1601 can be used to perform a multiplication calculation task using the second floating-point number and the third floating-point number. The internal structure of calculation unit 1601 can be seen with reference to Figure 9, and will not be described in detail here.

[0240] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the computing chip. In other embodiments of this application, the computing chip may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0241] Based on the same design concept as the above method embodiments, this application also provides an encoding chip that can be applied to any computing device. This encoding chip can be used to implement the function of the encoder in the above method embodiments, thus achieving the beneficial effects of the above method embodiments.

[0242] In some embodiments, the structure of the encoder chip 1700 can be as shown in Figure 17, including an encoder 1701 and an interface 1702 connected to the encoder 1701. The encoder 1701 and the interface 1702 can be interconnected via a bus, which can be a PCI bus or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0243] Interface 1702 can be used to obtain a first floating-point number, which can be a normalized floating-point number and may include a first sign field, a first exponent field, and a first mantissa field. Encoder 1701 can be used to encode based on the first floating-point number to obtain a second floating-point number. The data format of the second floating-point number includes a shared data portion and a variable data portion; the shared data portion is shared by multiple data within the data range to which the second floating-point number belongs, and is stored in a shared area of ​​memory. The shared data portion includes a second exponent field and a second mantissa field.

[0244] This application also provides a computer program product comprising computer-executable instructions. In one embodiment, the computer-executable instructions are used to cause a computer to perform the functions described in the method embodiments above.

[0245] Computer-executable instructions can be stored in a computer-readable storage medium. This application also provides a computer-readable storage medium storing executable instructions. In one embodiment, the computer-executable instructions are used to cause a computer to perform the functions described in the method embodiments above.

[0246] The computer-readable storage medium provided in the embodiments of this application may be random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), register, hard disk, portable hard disk, CD-ROM, or any other form of computer-readable storage medium known in the art.

[0247] Computer-executable instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; it can also be an optical medium, such as a digital video disc (DVD); or it can be a semiconductor medium, such as a solid-state drive (SSD).

[0248] In the various embodiments of this application, unless otherwise specified or logically conflicting, the terminology and / or descriptions between different embodiments are consistent and can be referenced mutually. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as including a series of steps or units. A method, system, product, or device is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices.

[0249] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made therein without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely illustrative examples of the solutions defined by the appended claims and are to be considered as covering any and all modifications, variations, combinations, or equivalents within the scope of this application.

[0250] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of this application. Therefore, if these modifications and variations of the embodiments of this application fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

Claims

1. A method of floating-point number processing, characterized by, The method is executed by a computing device, which includes an encoder and a memory, comprising: The encoder acquires a first floating-point number; the first floating-point number is a normalized floating-point number, and the first floating-point number includes a first sign field, a first exponent field, and a first mantissa field; The encoder obtains a second floating-point number based on the first symbol field, the first exponent field, and the first mantissa field. The data format of the second floating-point number includes a shared data portion and a variable data portion. The shared data portion is determined by the data range to which the data represented by the first floating-point number belongs. The shared data portion is stored in the shared area of ​​the memory and is shared by multiple data within the data range to which the first floating-point number belongs. The shared data portion includes a second exponent field and a second mantissa field. The data represented by the second floating-point number and the first floating-point number are the same.

2. The method of claim 1, wherein, The second exponent field is used to characterize the exponent of the shared data portion, and the second mantissa field is used to characterize the mantissa of the shared data portion.

3. The method of claim 1, wherein, The second exponent field is used to characterize the exponent of the shared data portion; the value of the second exponent field is a set value, and the second mantissa field is used to characterize the offset of the exponent of the shared data portion.

4. The method according to any one of claims 1 to 3, characterized in that, The encoder obtains the second floating-point number based on the first symbol field, the first exponent field, and the first mantissa field, including: The encoder determines the data range to which the data represented by the first floating-point number belongs based on the first symbol field, the first exponent field, and the first mantissa field; The encoder acquires the shared data portion corresponding to the data range; The encoder determines the variable data portion based on the first floating-point number and the shared data portion to obtain the second floating-point number.

5. The method of claim 4, wherein, The encoder determines the shared data portion corresponding to the data range in the following manner: The encoder determines a first ratio between the maximum absolute value of multiple data points contained in the data range and a set maximum value; The encoder obtains the shared data portion based on the first ratio.

6. The method of claim 5, wherein, The encoder obtains the shared data portion based on the first ratio, including: The encoder determines the second exponent field of the shared data portion based on the exponent of the first ratio; or, it determines the second exponent field based on the sum of the exponent of the first ratio and the first preset exponent. The encoder determines the second mantissa field of the shared data portion based on the mantissa of the first ratio.

7. The method of claim 5, wherein, The encoder obtains the shared data portion based on the first ratio, including: The encoder determines the second exponent field of the shared data portion based on the exponent of the first ratio; When the value of the second exponent field is a set value, the encoder determines the second mantissa field of the shared data portion based on the exponent of the first ratio.

8. The method of claim 5, wherein, The encoder obtains the shared data portion based on the first ratio, including: The encoder determines the second exponent field of the shared data portion based on the sum of the exponent of the first ratio and the first preset exponent. When the second exponent field is a set value, the encoder determines the second mantissa field of the shared data portion based on the sum of the exponent of the first ratio and the first preset exponent.

9. The method according to any one of claims 4 to 8, characterized in that, The encoder determines the variable data portion based on the first floating-point number and the shared data portion, including: The encoder determines a second ratio between the first floating-point number and the shared data portion; The encoder obtains the variable data portion based on the second ratio.

10. The method according to any one of claims 1 to 9, characterized in that, The computing device further includes a computing unit, and the method further includes: The calculation unit performs a multiplication calculation task using the second floating-point number and the third floating-point number to obtain the calculation result.

11. The method of claim 10, wherein, The computing unit includes a shared data operation module and a multiplier; the shared data portion of the third floating-point number includes the exponent; The calculation unit performs a multiplication operation using the second floating-point number and the third floating-point number to obtain the calculation result, including: The shared data operation module uses the sum of the exponent of the shared data part of the second floating-point number and the exponent of the shared data part of the third floating-point number as the first part of the exponent of the calculation result. The shared data processing module takes the product of the mantissa of the shared data portion of the second floating-point number and the preset mantissa as the first part of the mantissa of the calculation result; The multiplier obtains the calculation result based on the product of the first part of the exponent, the first part of the mantissa, and the variable data part of the second floating-point number and the variable data part of the third floating-point number.

12. The method of claim 10, wherein, The calculation unit includes a shared data operation module and a multiplier; the shared data portion of the third floating-point number includes an exponent and a mantissa; the calculation unit performs a multiplication operation between the second floating-point number and the third floating-point number to obtain a calculation result, including: The shared data operation module uses the sum of the exponent of the shared data part of the second floating-point number and the exponent of the shared data part of the third floating-point number as the second part of the exponent of the calculation result. The shared data processing module takes the product of the mantissa of the shared data portion of the second floating-point number and the mantissa of the shared data portion of the third floating-point number as the second part of the mantissa of the calculation result. The multiplier obtains the calculation result based on the product of the second part of the exponent, the second part of the mantissa, and the variable data part of the second floating-point number and the variable data part of the third floating-point number.

13. A computing device, comprising: It includes an encoder and a memory; the encoder is used to perform the method of any one of claims 1 to 9, and the memory is used to store the second floating-point number.

14. The computing device of claim 13, wherein, The computing device further includes a computing unit for performing the method according to any one of claims 10 to 12.

15. A computing chip, comprising: The computing chip includes a computing unit and an interface; The interface is used to obtain a second floating-point number and a third floating-point number. The data format of the second floating-point number includes a shared data part and a variable data part. The shared data part is shared by multiple data within the data range to which the second floating-point number belongs. The shared data part is stored in a shared area of ​​the memory. The shared data part includes a second exponent field and a second mantissa field. The computing unit is used to perform a multiplication calculation task with the third floating-point number using the shared data portion and the variable data portion of the second floating-point number.

16. The compute chip of claim 15, wherein, The third floating-point number includes a shared data portion and a variable data portion; the shared data portion of the third floating-point number includes an exponent; the calculation unit includes a shared data operation module and a multiplier. The shared data operation module is used to take the sum of the exponent of the shared data part of the second floating-point number and the exponent of the shared data part of the third floating-point number as the first part of the exponent of the calculation result. The shared data processing module is further configured to use the product of the mantissa of the shared data portion of the second floating-point number and the preset mantissa as the first part of the mantissa of the calculation result; The multiplier is used to obtain the calculation result based on the product of the first part of the exponent, the first part of the mantissa, and the variable data part of the second floating-point number and the variable data part of the third floating-point number.

17. The compute chip of claim 15, wherein, The third floating-point number uses the same data format as the second floating-point number; the calculation unit includes a shared data operation module and a multiplier. The shared data operation module is used to sum the exponent of the shared data part of the second floating-point number and the exponent of the shared data part of the third floating-point number as the second part of the exponent of the calculation result. The shared data operation module is further configured to take the product of the mantissa of the shared data part of the second floating-point number and the mantissa of the shared data part of the third floating-point number as the second part of the mantissa of the calculation result; The multiplier is used to obtain the calculation result based on the product of the second part of the exponent, the second part of the mantissa, and the variable data part of the second floating-point number and the variable data part of the third floating-point number.

18. An encoding chip, characterized by It includes an interface and an encoder, the interface being used to obtain a first floating-point number, and the encoder being used to perform the method of any one of claims 1 to 9.

Images