Arithmetic unit, related apparatus and method

Abstract

The present disclosure provides an operation unit, related devices and methods. The operation unit comprises: a first format conversion unit configured to convert data in a source data format into an intermediate data format according to a first rule; and a second format conversion unit configured to convert data in the intermediate data format into data in a target data format according to a second rule. The intermediate data format comprises all fields of the source data format and all fields of the target data format, and the number of bits in each field of the intermediate data format is not less than the number of bits in the corresponding field of either the source data format or the target data format. The embodiments of the present disclosure improve the versatility of hardware implementation of data type conversion and reduce hardware resource consumption caused by data type conversion.

Description

Technical Field

[0001] This disclosure relates to the field of chips, and more specifically, to a computing unit, related apparatus and method. Background Technology

[0002] Data processing frequently involves operations on various data types, such as integers and floating-point numbers. Currently, to improve computational efficiency, chip design is used to support operations on multiple data types at the hardware level. These data types include, but are not limited to, 32-bit floating-point numbers (F32), 32-bit tensor floating-point numbers (TF32), 16-bit floating-point numbers (F16), 16-bit brain-type floating-point numbers (Bf16), 32-bit signed integers (S32), 32-bit unsigned integers (U32), 16-bit signed integers (S16), 16-bit unsigned integers (U16), 8-bit signed integers (S8), and 8-bit unsigned integers (U8). Data type conversion is essential. Existing technologies typically require dedicated hardware for each pair of different data types, following specific conversion methods, i.e., dedicated data paths. The number of combinations of any two different data types corresponds to the number of data paths. This approach suffers from poor versatility, requires numerous data paths, and cannot reuse existing data paths. Summary of the Invention

[0003] In view of this, the present disclosure aims to improve the versatility of hardware implementation of data type conversion and reduce the hardware resource consumption caused by data type conversion.

[0004] According to one aspect of this disclosure, a computing unit is provided, comprising:

[0005] The first format conversion unit is used to convert data in the source data format into an intermediate data format according to the first rule;

[0006] The second format conversion unit is used to convert the data in the intermediate data format into the data in the target data format according to the second rule. The intermediate data format includes at least all the fields of the source data format and all the fields of the target data format, and the number of bits in the fields of the intermediate data format is not less than the number of bits in the corresponding fields of either the source data format or the target data format.

[0007] Optionally, the intermediate data format includes a sign bit, an exponent bit, an integer bit, and a mantissa bit. The value represented by the intermediate data format is: (-1)^sign*2^(exponent-255)*(integer[0]+fraction*2^-31-2*integer[1]), where ^ represents the exponent, sign represents the sign bit, sign=0 represents a positive number, sign=1 represents a negative number, exponent is the value corresponding to the exponent bit, integer[0] is the last bit of the integer bit, 0 represents a non-normalized floating-point number, 1 represents a normalized floating-point number, fraction represents the mantissa bit, integer[1] is the second to last bit of the integer bit, 0 represents a positive mantissa, and 1 represents a negative mantissa.

[0008] Optionally, the exponent is 9 bits and the mantissa is 31 bits.

[0009] Optionally, the arithmetic unit further includes:

[0010] A leading zero counting unit is used to check the number of consecutive 0s starting from the most significant bit in the sequence formed by the last bit of the integer part and the mantissa part;

[0011] A left shift unit is used to shift the number of consecutive 0s to the left out of the intermediate data format;

[0012] An exponent adjustment unit is used to adjust the exponent position so that the value corresponding to the exponent position is reduced by the number of values.

[0013] Optionally, the arithmetic unit further includes:

[0014] An overflow detection unit is used to determine that the data in the intermediate data format overflows after being converted into the target data format.

[0015] The overflow processing unit is used to perform pre-defined processing on overflowing data in the target data format.

[0016] Optionally, the overflow detection unit determines the overflow by at least one of the following:

[0017] If the target data format is a 32-bit tensor floating-point number and the value corresponding to the exponent bit is 383, then an upward overflow is determined.

[0018] If the target data format is a 16-bit floating-point number, and the value corresponding to the exponent bit is greater than or equal to 271, then it is determined to overflow upwards; if the value corresponding to the exponent bit is less than or equal to 240, then it is determined to overflow downwards.

[0019] If the target data format is a 16-bit human brain floating-point number, and the value corresponding to the exponent bit is greater than or equal to 383, then it is determined to overflow upwards; if the value corresponding to the exponent bit is less than or equal to 128, then it is determined to overflow downwards.

[0020] If the target data format is a 32-bit signed integer, and the value corresponding to the exponent bit is greater than or equal to 286 and the sign bit is 0, then it is determined to overflow upwards; if the value corresponding to the exponent bit is greater than or equal to 286 and the sign bit is 1, then it is determined to overflow downwards.

[0021] If the target data format is a 32-bit unsigned integer, and the value corresponding to the exponent bit is greater than or equal to 287, then it is determined to overflow upwards; if the value corresponding to the exponent bit is less than or equal to 254, then it is determined to overflow downwards.

[0022] If the target data format is a 16-bit signed integer, and the value corresponding to the exponent bit is greater than or equal to 270 and the sign bit is 0, then it is determined to overflow upwards; if the value corresponding to the exponent bit is greater than or equal to 270 and the sign bit is 1, then it is determined to overflow downwards.

[0023] If the target data format is a 16-bit unsigned integer, and the value corresponding to the exponent bit is greater than or equal to 271, then it is determined to overflow upwards; if the value corresponding to the exponent bit is less than or equal to 254, then it is determined to overflow downwards.

[0024] If the target data format is an 8-bit signed integer, and the value corresponding to the exponent bit is greater than or equal to 2^62 and the sign bit is 0, then it is determined to overflow upwards; if the value corresponding to the exponent bit is greater than or equal to 2^62 and the sign bit is 1, then it is determined to overflow downwards.

[0025] If the target data format is an 8-bit unsigned integer, and the value corresponding to the exponent bit is greater than or equal to 263, then it is determined to overflow upwards; if the value corresponding to the exponent bit is less than or equal to 254, then it is determined to overflow downwards.

[0026] Optionally, the arithmetic unit further includes:

[0027] The denormalization determination unit is used to determine whether the data in the intermediate data format is denormalized or an integer after being converted into the target data format;

[0028] The right shift unit is used to shift the mantissa of the intermediate data format to the right based on the determination result of whether it is denormalized data or an integer, so that the data after shifting and converting the intermediate data format into the target data format is normalized data.

[0029] Optionally, the arithmetic unit further includes:

[0030] The rounding unit is used to determine whether the rounding operation is performed on the mantissa of the intermediate data format after shifting the right bit out of the right bit shifting unit.

[0031] Optionally, the right shift unit shifts the mantissa bits of the intermediate data format to the right according to at least one of the following:

[0032] If the target data format is a 32-bit floating-point number or a 32-bit tensor floating-point number, and it is determined to be denormalized data, the value corresponding to the exponent bit is T, and the intermediate data format is shifted to the right by (137-T) bits; if it is determined to be normalized data, the intermediate data format is shifted to the right by 8 bits.

[0033] If the target data format is a 16-bit floating-point number, and it is determined to be denormalized data, the value corresponding to the exponent bit is T, and the intermediate data format is shifted to the right by (2^62 - T) bits; if it is determined to be normalized data, the intermediate data format is shifted to the right by 21 bits.

[0034] If the target data format is a 16-bit human brain floating-point number, and it is determined to be denormalized data, the value corresponding to the exponent bit is T, and the intermediate data format is shifted to the right by (153-T) bits; if it is determined to be normalized data, the intermediate data format is shifted to the right by 24 bits.

[0035] If the target data format is a 32-bit signed integer, a 32-bit unsigned integer, a 16-bit signed integer, a 16-bit unsigned integer, an 8-bit signed integer, or an 8-bit unsigned integer, and the value corresponding to the exponent bit is T, then the intermediate data format is shifted to the right by (286-T) bits.

[0036] Optionally, the first rule includes at least one of the following:

[0037] If the source data format is a 32-bit floating-point number or a 32-bit tensor floating-point number, fill the sign bit of the 32-bit floating-point number or 32-bit tensor floating-point number into the sign bit of the intermediate data format; fill the last 7 exponent bits of the 32-bit floating-point number or 32-bit tensor floating-point number into the last 7 exponent bits of the intermediate data format; fill the first exponent bit of the 32-bit floating-point number or 32-bit tensor floating-point number into the first exponent bit of the intermediate data format; and fill the negative of the first exponent bit of the 32-bit floating-point number or 32-bit tensor floating-point number into the intermediate data format. The 5th to 8th exponent bits from the end of the intermediate data format are used; if the 32-bit floating-point number or 32-bit tensor floating-point number is a denormalized floating-point number, the last bit of the integer part of the intermediate data format is filled with 0; if the 32-bit floating-point number or 32-bit tensor floating-point number is a normalized floating-point number, the last bit of the integer part of the intermediate data format is filled with 1; starting from the first bit of the mantissa of the intermediate data format, the mantissa of the 32-bit floating-point number or 32-bit tensor floating-point number is inserted sequentially, and the remaining bits of the mantissa of the intermediate data format are filled with 0;

[0038] If the source data format is a 16-bit floating-point number, fill the sign bit of the 16-bit floating-point number into the sign bit of the intermediate data format, fill the last 4 exponent bits of the 16-bit floating-point number into the last 4 exponent bits of the intermediate data format, fill the first exponent bit of the 16-bit floating-point number into the first exponent bit of the intermediate data format, and fill the opposite of the first exponent bit of the 16-bit floating-point number into the second exponent bit of the intermediate data format; if the 16-bit floating-point number is a denormalized floating-point number, fill the last bit of the integer part of the intermediate data format with 0; if the 16-bit floating-point number is a normalized floating-point number, fill the last bit of the integer part of the intermediate data format with 1; starting from the first bit of the mantissa part of the intermediate data format, sequentially insert the mantissa parts of the 16-bit floating-point number, and fill the remaining bits of the mantissa parts of the intermediate data format with 0.

[0039] If the source data format is a 16-bit floating-point number, fill the sign bit of the 16-bit floating-point number into the sign bit of the intermediate data format, fill the last 7 exponent bits of the 16-bit floating-point number into the last 7 exponent bits of the intermediate data format, fill the first exponent bit of the 16-bit floating-point number into the first exponent bit of the intermediate data format, and fill the opposite of the first exponent bit of the 16-bit floating-point number into the second exponent bit of the intermediate data format; if the 16-bit floating-point number is a denormalized floating-point number, fill the last bit of the integer part of the intermediate data format with 0; if the 16-bit floating-point number is a normalized floating-point number, fill the last bit of the integer part of the intermediate data format with 1; starting from the first bit of the mantissa of the intermediate data format, sequentially insert the mantissa of the 16-bit floating-point number, and fill the remaining bits of the mantissa of the intermediate data format with 0.

[0040] If the source data format is a 32-bit signed integer, a 32-bit unsigned integer, a 16-bit signed integer, a 16-bit unsigned integer, an 8-bit signed integer, or an 8-bit unsigned integer, convert the source data format to a 33-bit signed integer; set the sign bit of the intermediate data format to 0; set the exponent bit of the intermediate data format to the binary value corresponding to 286; and fill the integer bits and mantissa bits of the intermediate data format with the 33-bit signed integer.

[0041] Optionally, if the source data format is a 32-bit signed integer, a 32-bit unsigned integer, a 16-bit signed integer, a 16-bit unsigned integer, an 8-bit signed integer, or an 8-bit unsigned integer, converting the source data format to a 33-bit signed integer includes:

[0042] Add 1, 17, or 25 zeros to the end of the 32-bit signed integer, 16-bit signed integer, or 8-bit signed integer respectively to make a 33-bit signed integer.

[0043] Add one 0 to the front of the 32-bit unsigned integer to make a 33-bit signed integer;

[0044] Add one 0 to the front of the 16-bit unsigned integer or the 8-bit unsigned integer, and add 16 or 24 0s to the back respectively to make a 33-bit signed integer.

[0045] Optionally, the second rule includes at least one of the following:

[0046] If the target data format is a 32-bit floating-point number or a 32-bit tensor floating-point number, fill the sign bit of the intermediate data format into the sign bit of the 32-bit floating-point number or the 32-bit tensor floating-point number, fill the last 7 exponent bits of the intermediate data format into the last 7 exponent bits of the 32-bit floating-point number or the 32-bit tensor floating-point number, and fill the first exponent bit of the intermediate data format into the first exponent bit of the 32-bit floating-point number or the 32-bit tensor floating-point number; use the last 23 bits of the mantissa of the intermediate data format as the mantissa of the 32-bit floating-point number, or use the last 10 bits of the mantissa of the intermediate data format as the mantissa of the 32-bit tensor floating-point number.

[0047] If the target data format is a 16-bit floating-point number, the sign bit of the intermediate data format is filled into the sign bit of the 16-bit floating-point number, the last 4 exponent bits of the intermediate data format are filled into the last 4 exponent bits of the 16-bit floating-point number, and the first exponent bit of the intermediate data format is filled into the first exponent bit of the 16-bit floating-point number; the last 10 bits of the mantissa of the intermediate data format are used as the mantissa of the 16-bit floating-point number.

[0048] If the target data format is a 16-bit floating-point number, the sign bit of the intermediate data format is filled into the sign bit of the 16-bit floating-point number, the last 7 exponent bits of the intermediate data format are filled into the last 7 exponent bits of the 16-bit floating-point number, and the first exponent bit of the intermediate data format is filled into the first exponent bit of the 16-bit floating-point number; the last 7 bits of the mantissa of the intermediate data format are used as the mantissa of the 16-bit floating-point number.

[0049] If the target data format is a 32-bit signed integer, a 16-bit signed integer, or an 8-bit signed integer, the integer part of the intermediate data format and the mantissa parts after removing the first 1, 17, or 25 bits are concatenated to form the 32-bit signed integer, 16-bit signed integer, or 8-bit signed integer, respectively.

[0050] If the target data format is a 32-bit unsigned integer, extract the integer part and mantissa part of the intermediate data format, remove the first bit, and use it as the 32-bit unsigned integer;

[0051] If the target data format is a 16-bit unsigned integer, the integer part after removing the first bit and the mantissa part after removing the first 16 bits of the intermediate data format are concatenated to form the 16-bit unsigned integer.

[0052] If the target data format is an 8-bit unsigned integer, the integer part after removing the first bit and the mantissa part after removing the first 24 bits of the intermediate data format are concatenated to form the 8-bit unsigned integer.

[0053] Optionally, the arithmetic unit includes an integer data path and a floating-point data path. The integer data path is used for integer arithmetic processing, and the floating-point data path is used for floating-point arithmetic processing. The first format conversion unit and the second format conversion unit are included in the integer data path.

[0054] According to one aspect of this disclosure, a processing unit is provided, comprising:

[0055] The instruction execution unit serves as the arithmetic unit as described above;

[0056] A register, wherein data in the source data format is read from the register by the arithmetic unit, and data in the target data format is written to the register by the arithmetic unit.

[0057] According to one aspect of this disclosure, an acceleration unit is provided, comprising:

[0058] Tensor engines serve as the computational units described above;

[0059] On-chip memory, wherein data in the source data format is read from the on-chip memory by the tensor engine, and data in the target data format is written to the on-chip memory by the tensor engine.

[0060] According to one aspect of this disclosure, a computing device is provided, including at least one of the processing unit and the acceleration unit described above.

[0061] According to one aspect of this disclosure, a system-on-a-chip is provided, including at least one of the processing unit as described above and the acceleration unit as described above.

[0062] According to one aspect of this disclosure, a data center is provided, including the computing device described above.

[0063] According to one aspect of this disclosure, a data format conversion method is provided, comprising:

[0064] Convert the source data format into an intermediate data format according to the first rule;

[0065] The data in the intermediate data format is converted into the data in the target data format according to the second rule. The intermediate data format includes at least all the fields of the source data format and all the fields of the target data format, and the number of bits in the fields of the intermediate data format is not less than the number of bits in the corresponding fields of either the source data format or the target data format.

[0066] In this embodiment, unlike existing technologies that set different data paths for each of two different data types according to the conversion method between the two types, the source data format is converted into a universal intermediate data format (the field settings of this intermediate data format take into account existing fields that can encompass various possible data types, and the number of bits is greater than or equal to the number of bits of existing fields of various possible data types, thereby removing obstacles for any data type to be converted with the intermediate data format). Then, the intermediate data format is converted into the target data format to be converted. This data path is unique and can be reused with existing data paths in the arithmetic unit (such as integer data paths), which improves the versatility of the hardware implementation of data type conversion and reduces the hardware resource consumption caused by data type conversion. Attached Figure Description

[0067] The above and other objects, features, and advantages of this disclosure will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0068] Figure 1 This is a structural diagram of a data center used in one embodiment of this disclosure;

[0069] Figure 2 This is an internal structure diagram of a server in a data center according to an embodiment of this disclosure;

[0070] Figure 3 This is an internal structural diagram of a processing unit and an acceleration unit inside a server according to an embodiment of the present disclosure, which shows the computing unit that serves as the execution subject of the embodiment of the present disclosure;

[0071] Figure 4 This is an internal structural diagram of an acceleration unit core according to an embodiment of the present disclosure;

[0072] Figure 5 This is an internal structural diagram of a computing unit according to an embodiment of the present disclosure;

[0073] Figure 6A -J shows the data structure diagrams for 32-bit floating-point numbers, 32-bit tensor floating-point numbers, 16-bit floating-point numbers, 16-bit human brain floating-point numbers, 32-bit signed integers, 32-bit unsigned integers, 16-bit signed integers, 16-bit unsigned integers, 8-bit signed integers, and 8-bit unsigned integers, respectively.

[0074] Figure 7 A data structure diagram of an intermediate data format according to an embodiment of the present disclosure is shown;

[0075] Figure 8 The present disclosure illustrates, according to one embodiment, the conditions for overflow and underflow when converting an intermediate data format to a target data format for different target data formats, and the number of bits by which the mantissa of the intermediate data format is shifted to the right to ensure that the converted target data format is normalized data.

[0076] Figure 9 A flowchart of a data format conversion method according to an embodiment of the present disclosure is shown. Detailed Implementation

[0077] The present disclosure is described below based on embodiments, but it is not limited to these embodiments. In the detailed description of the present disclosure below, certain specific details are described in detail. Those skilled in the art will fully understand the present disclosure even without these details. To avoid obscuring the substance of the present disclosure, well-known methods, processes, and procedures are not described in detail. Furthermore, the accompanying drawings are not necessarily drawn to scale.

[0078] The following terms are used in this document.

[0079] Data format: Describes the rules for storing data in files or records. It corresponds to the data type; generally, each data type is represented by a different data format. It typically includes several fields, each representing a specific element of the data. For example, for floating-point numbers, the integer part, mantissa, exponent, and sign all affect the magnitude of the represented floating-point number and are all elements of the floating-point number, which may be represented by different fields in the data format.

[0080] Fixed-point numbers: Numbers in which the position of the decimal point is fixed. In computers, two simple conventions are commonly used: placing the decimal point before the most significant digit or after the least significant digit. The former is generally called a fixed-point decimal (mantissa), and the latter a fixed-point integer (integrity).

[0081] Floating-point numbers: A numerical representation of a number belonging to a specific subset of rational numbers, used in computers to approximate any real number. Specifically, this real number is obtained by multiplying an integer or fixed-point decimal (i.e., the mantissa) by an integer power of a base (usually 2 in computers), similar to scientific notation with base 10. A floating-point number 'a' is represented by two numbers 'm' and 'e': a = m × b^e, where ^ is the exponent symbol. In any such architecture, we choose a base 'b' (the base of the number system) and a precision 'p' (i.e., how many bits to use for storage). 'm' is a 'p'-bit number of the form ±d.ddd...ddd (each bit is an integer between 0 and b-1, inclusive). In the above floating-point representation a=m×b^e, e is called the exponent, which is generally an integer; the part of m after the decimal point, such as ddd...ddd in ±d.ddd...ddd, is called the mantissa; the part of m after the decimal point, such as d before the decimal point in ±d.ddd...ddd, is called the integer. The sign of the above floating-point number indicates whether it is positive or negative.

[0082] Tensor floating-point numbers: Floating-point numbers that are elements of a tensor. A tensor is an N-dimensional (where N can be very large) data structure, most commonly a scalar, vector, or matrix. A scalar is a 0-dimensional tensor, a vector is a 1-dimensional tensor, and a matrix is ​​a 2-dimensional tensor. The number of bits in the mantissa of a tensor floating-point number generally needs to be reduced to ensure the efficiency of tensor operations.

[0083] Human brain floating-point numbers: Floating-point numbers used when simulating human brain operations using neural networks. Similar to tensor floating-point numbers, their main difference from regular floating-point numbers is that they significantly reduce the number of bits in the mantissa, ensuring the execution efficiency of neural network operations.

[0084] 32-bit floating-point numbers (F32): Floating-point numbers represented using 32 bits, with the following data structure: Figure 6AAs shown. Bits 0-22 are the mantissa bits, used to represent the mantissa of the floating-point number, i.e., the ddd...ddd after the decimal point of m in the above a = m × b^e. The d before the decimal point of m is determined by the 8 exponent bits. When the 8 exponent bits are all 0, the d before the decimal point of m is 0. When the 8 exponent bits are not all 0, the d before the decimal point of m is 1. Bits 23-30 are the exponent bits. Bit 31 is the sign bit. When the sign bit S is 0, it represents a positive number; when the sign bit S is 1, it represents a negative number. Assuming the exponent is represented by 8 bits (exponent), the mantissa by 23 bits (fraction), and the sign bit by S, then when the exponent bits are not all 0s, the value of the 32-bit floating-point number is (-1)^S * (1.fraction) * 2^(exponent - 127); when the exponent bits are all 0s, the value is (-1)^S * (0.fraction) * 2^(-126). When S = 0, (-1)^S = 1, and the above 32-bit floating-point number is positive; when S = 1, (-1)^S = -1, and the above 32-bit floating-point number is negative. The 1s or 0s in the integer bits of 1.fraction or 0.fraction actually represent normalized or denormalized floating-point numbers (explained in detail later). The reason for subtracting 127 from exponent is that it is necessary to express that the floating-point number is not necessarily a number greater than 1; sometimes it is necessary to represent a number less than 1, in which case the exponent of 2 needs to be negative. Generally, about half of the numbers need to be less than 1, and about half need to be greater than 1. This provides a reasonable dynamic range in reality, allowing the representation of numbers of various orders of magnitude. The maximum value of an 8-bit exponent is 11111111, equivalent to 255 in decimal, half of which is approximately 127. Subtracting 127 from the exponent ensures a more balanced distribution of numbers larger and smaller than 127. When the exponent is 00000001, the value of the 32-bit floating-point number is (-1)^S*(1.fraction)*2^(-126), not (-1)^S*(1.fraction)*2. When the exponent is 11111111, the value of the 32-bit floating-point number is (-1)^S*(1.fraction)*2^(128), not (-1)^S*(1.fraction)*2^(255).

[0085] 32-bit Tensor Floating-Point Number (TF32): A tensor floating-point number represented using 32 bits, its data structure is as follows: Figure 6BAs shown. Bits 0-12 are idle bits, because to improve the processing efficiency of tensor operations, the number of bits in the mantissa cannot be too large. Bits 13-22 are the mantissa bits, used to represent the mantissa of this tensor floating-point number, that is, ddd...ddd after the decimal point of m in the above a = m × b^e. The d before the decimal point of m is determined by the 8 exponent bits. When the 8 exponent bits are all 0, the d before the decimal point of m is 0. When the 8 exponent bits are not all 0, the d before the decimal point of m is 1. Bits 23-30 are the exponent bits. Bit 31 is the sign bit. The meaning of the sign bit is the same as that of the 32-bit floating-point number above. The formula for calculating the value of a 32-bit tensor floating-point number is the same as that for a 32-bit floating-point number. That is, when the exponent is not all zeros, the value of the 32-bit floating-point number is (-1)^S * (1.fraction) * 2^(exponent - 127); when the exponent is all zeros, the value of the 32-bit floating-point number is (-1)^S * (0.fraction) * 2^(-126). fraction, exponent, and S represent the values ​​of the mantissa, exponent, and sign bits, respectively.

[0086] 16-bit floating-point number (F16): A floating-point number represented using 16 bits, its data structure is as follows: Figure 6C As shown. Bits 0-9 are the mantissa bits, used to represent the mantissa of the floating-point number, i.e., the ddd...ddd after the decimal point of m in the above a = m × b^e. The d before the decimal point of m is determined by the 5 exponent bits. When all 5 exponent bits are 0, the d before the decimal point of m is 0. When the 5 exponent bits are not all 0, the d before the decimal point of m is 1. Bits 10-14 are the exponent bits. Bit 15 is the sign bit. The meaning of the sign bit is the same as for 32-bit floating-point numbers. Therefore, when the exponent bits are not all 0, the value of the 16-bit floating-point number = (-1)^S * (1.fraction) * 2^(exponent-15); when the exponent bits are all 0, the value of the 16-bit floating-point number = (-1)^S * (0.fraction) * 2^(-14). The formula above is similar to that for 32-bit floating-point numbers, except that the value subtracted from the exponent changes from 127 to 15, because the maximum value of the 5 mantissa digits is 2. 5 -1, where the value is 15. fraction, exponent, and S represent the values ​​of the mantissa, exponent, and sign bits, respectively.

[0087] 16-bit human brain floating-point number (Bf16): A human brain floating-point number represented by 16 bits, its data structure is as follows: Figure 6DAs shown. Bits 0-6 are the mantissa bits, used to represent the last digit of the floating-point number, which is fewer than the mantissa bits of a typical 16-bit floating-point number. The mantissa bits are the ddd...ddd bits after the decimal point of m in the above a = m × b^e. The d before the decimal point of m is determined by the 8 exponent bits. When the 8 exponent bits are all 0, the d before the decimal point of m is 0. When the 8 exponent bits are not all 0, the d before the decimal point of m is 1. Bits 7-14 are the exponent bits. Bit 15 is the sign bit. The meaning of the sign bit is the same as that of the 16-bit floating-point number. The calculation formula for the value of a 16-bit floating-point number is the same as that for a 32-bit floating-point number, and since its exponent is also 8 bits, it will not be repeated.

[0088] 32-bit signed integer (S32): An integer is a number where the decimal point is at the last digit. A 32-bit signed integer is a signed integer represented using 32 bits, and its data structure is as follows: Figure 6E As shown. Bits 0-30 represent the absolute value of the signed integer, and bit 31 represents the sign, i.e., whether the number is positive or negative. A sign bit of 0 indicates a positive number, and a sign bit of 1 indicates a negative number. When the sign bit is 0, the value of bits 0-30 is the value of the 32-bit signed integer. When the sign bit is 1, the value of the 32-bit signed integer = the value of bits 0-30 - 2. 31 For example, 1000000000000000000000000000000000 represents -2. 31 1 ...

[0089] 32-bit unsigned integer (U32): An unsigned integer represented using 32 bits, its data structure is as follows: Figure 6F As shown. Bits 0-31 represent the value of a 32-bit unsigned integer. For example, 00000000000000000000000000000000 represents 0, and 1111111111111111111111111111111111111 represents 2. 32 -1.

[0090] 16-bit signed integer (S16): A signed integer represented using 16 bits, its data structure is as follows: Figure 6G As shown. Bits 0-14 represent the absolute value of the signed integer, and bit 15 represents the sign, i.e., whether the number is positive or negative. A sign bit of 0 indicates a positive number, and a sign bit of 1 indicates a negative number. When the sign bit is 0, the value of bits 0-14 is the value of the 16-bit signed integer. When the sign bit is 1, the value of the 16-bit signed integer = the value of bits 0-14 - 2. 15 For example, 1000000000000000 represents -2. 15 1111111111111111 represents -1.

[0091] 16-bit unsigned integer (U16): An unsigned integer represented using 16 bits, its data structure is as follows: Figure 6H As shown. Its 0-15 bits represent the value of a 16-bit unsigned integer. For example, 0000000000000000 represents 0, and 1111111111111111 represents 2. 16 -1.

[0092] 8-bit signed integer (S8): A signed integer represented using 8 bits, its data structure is as follows: Figure 6I As shown. Bits 0-6 represent the absolute value of the signed integer, and bit 7 represents the sign, i.e., whether the number is positive or negative. A sign bit of 0 indicates a positive number, and a sign bit of 1 indicates a negative number. When the sign bit is 0, the value of bits 0-6 is the value of the 8-bit signed integer. When the sign bit is 1, the value of the 8-bit signed integer = the value of bits 0-6 - 2. 7 For example, 1000000000000000 represents -2. 7 1111111111111111 represents -1.

[0093] 8-bit unsigned integer (U8): An unsigned integer represented by 8 bits, its data structure is as follows: Figure 6J As shown. Its 0-7 bits represent the value of an 8-bit unsigned integer. For example, 0000000000000000 represents 0, and 1111111111111111 represents 2. 8 -1.

[0094] Normalized floating-point number (NF): A floating-point number whose exponent is not all zeros. In the above a = m × b^e, m is ±d.ddd...ddd. The hallmark of a normalized floating-point number is that the d before the decimal point in m is not zero; in binary, it is 1. It is a representation similar to scientific notation, where any number can be represented as a decimal number with only one decimal place multiplied by an integer power of a certain base. When a floating-point number is represented in this standard form, it is a normalized floating-point number. Its characteristics are that the d before the decimal point in m is 1, and the exponent is generally not zero.

[0095] Denormalized floating-point numbers: Floating-point numbers where all exponent bits are 0. In the above a = m × b^e, m is ±d.ddd...ddd. The hallmark of a normalized floating-point number is that the d before the decimal point of m is 0, for example, m is 0.0010001. Denormalized floating-point numbers can be converted to normalized floating-point numbers by making the d before the decimal point of m non-zero. For example, the denormalized floating-point number 0.0010001 × 10^ ... 5 This can be represented as a normalized floating-point number 1.0001 × 10⁻⁶. 2 .

[0096] Overflow: The value after conversion to a certain data format exceeds the range of values ​​that the data format can represent.

[0097] Overflow: The value after conversion to a certain data format is greater than the maximum value that the data format can represent.

[0098] Downflow: The value after conversion to a certain data format is less than the minimum value that the data format can represent.

[0099] Left shift: Shifts all bits of a field of data to the left by a certain number of positions, filling the empty bits on the right with 0.

[0100] Right shift: Shifts all bits of a data field to the right by a specified number of positions, filling in the vacated bits on the left according to a predetermined rule. Right shifts are divided into logical right shifts and arithmetic right shifts. For logical right shifts, the predetermined rule is to fill in the vacated bits on the left with 0s. For arithmetic right shifts, the predetermined rule is to fill in the vacated bits on the left with the sign bit.

[0101] Rounding: When you need to retain a certain number of decimal places, you decide whether to carry over the remaining digits based on the digits to be discarded. Specific rounding methods are existing technologies.

[0102] Data path: A data path is the sum of all the related units used in a chip to perform a type of processing. For example, the sum of all units performing integer arithmetic is the integer data path, and the sum of all units performing floating-point arithmetic is the floating-point data path.

[0103] Acceleration Unit: A unit designed to improve data processing speed in specialized fields (e.g., image processing, various operations for deep learning models, etc.) where traditional processing units are inefficient. Acceleration units include Central Processing Units (CPUs), Graphics Processing Units (GPUs), General-Purpose Graphics Processing Units (GPGPUs), Field-Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), and dedicated intelligent acceleration hardware (e.g., Neural Processing Units, NPUs).

[0104] Processing Unit: A unit that performs traditional processing (not the processing handled by the acceleration units mentioned above). In addition to these traditional processes, the processing unit also performs scheduling functions for the acceleration units, assigning tasks to them. Processing units can take various forms, such as CPUs, Application-Specific Integrated Circuits (ASICs), and Field-Programmable Gate Arrays (FPGAs).

[0105] Application Environment of This Disclosure

[0106] This disclosure proposes a data format conversion scheme. The entire data format conversion scheme is relatively universal. Examples include data centers, IoT (Internet of Things) devices that need to process data in different formats, and embedded devices. This data format conversion scheme is independent of the final deployed hardware environment. However, for illustrative purposes, the following description will primarily focus on data centers as an application scenario. Those skilled in the art should understand that this disclosure can also be applied to other application scenarios.

[0107] Data Center

[0108] Data centers are globally collaborative networks of specific equipment used to transmit, accelerate, display, compute, and store data information on the internet infrastructure. In the future, data centers will become a key competitive asset for businesses. Big data analytics in data centers often utilize data in various formats. When these data are processed together, format conversion is necessary beforehand.

[0109] In traditional large data centers, the network architecture is typically as follows: Figure 1 As shown, this is the hierarchical inter-networking model. This model includes the following components:

[0110] Server 140: Each server 140 is a processing and storage entity in the data center. The processing and storage of a large amount of data in the data center are all done by these servers 140.

[0111] Access Switch 130: Access Switch 130 is used to connect Server 140 to the data center. One Access Switch 130 connects multiple Servers 140. Access Switches 130 are typically located at the top of the rack, so they are also called Top of Rack switches, and they physically connect the servers.

[0112] Aggregation Switch 120: Each aggregation switch 120 connects to multiple access switches 130 and provides other services such as firewall, intrusion detection, network analysis, etc.

[0113] Core Switch 110: Core Switch 110 provides high-speed forwarding for packets entering and leaving the data center and provides connectivity for Aggregation Switch 120. The entire data center network is divided into an L3 routing network and an L2 routing network. Core Switch 110 typically provides a flexible L3 routing network for the entire data center network.

[0114] Typically, aggregation switch 120 serves as the boundary between L2 and L3 layer routing networks. Below aggregation switch 120 is the L2 network, and above it is the L3 network. Each aggregation switch group manages one Point of Delivery (POD), and each POD contains an independent VLAN network. Server migration within a POD does not require modification of IP addresses and default gateways, as one POD corresponds to one L2 broadcast domain.

[0115] The aggregation switch 120 and access switch 130 typically use the Spanning Tree Protocol (STP). STP ensures that only one aggregation switch 120 is available for a given VLAN network; other aggregation switches 120 are only used in case of failure (dashed lines in the diagram above). In other words, horizontal scaling is not possible at the aggregation switch 120 level, because even if multiple aggregation switches 120 are added, only one will be operational.

[0116] server

[0117] Since server 140 is the actual processing equipment in the data center. Figure 2 A block diagram of the internal structure of a server 140 is shown. Server 140 includes a bus-connected memory 210, a processing unit cluster 270, and an acceleration unit cluster 280. The processing unit cluster 270 includes multiple processing units 220. The acceleration unit cluster 280 includes multiple acceleration units 230. Acceleration units 230 are processing units designed to improve data processing speed in specialized applications. Acceleration units include central processing units (CPUs), graphics processing units (GPUs), general-purpose graphics processing units (GPGPUs), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), and dedicated intelligent acceleration hardware (e.g., neural network processors (NPUs)). A processing unit is a unit that schedules acceleration units and allocates sequences of instructions to be executed to each acceleration unit. It also performs some calculations and processing itself, and can take various forms such as processing units (CPUs), application-specific integrated circuits (ASICs), and field-programmable gate arrays (FPGAs).

[0118] Traditional processing unit architectures often dedicate significant space to control and storage units, leaving insufficient space for computation units. While efficient for logic control, this approach is inefficient for large-scale parallel computing. Therefore, specialized acceleration units have been developed to improve computational speed for different functions and domains. For example, data-driven parallel computing architectures are used to enhance the speed of deep learning models. These complex calculations involve large amounts of data and generate numerous intermediate results, which are frequently accessed during computation. Using existing processing unit architectures results in inefficiency due to the limited internal memory of the cores, necessitating frequent access to external memory. Acceleration units, with their on-chip memory for storing large amounts of input data and intermediate results within each core, avoid frequent access to external memory, significantly improving processing efficiency and computational performance.

[0119] The acceleration unit 230 needs to be scheduled by the processing unit 220. For example... Figure 2 As shown, memory 210 stores parameters needed for accelerated computation, such as node weights in a deep learning model. These accelerated computations are processed when needed. Figure 2 One of the processing units 220 is deployed to an acceleration unit 230. That is, the processing unit 220 can send the addresses of the parameters (such as the weights of each node) required for accelerated computation in memory 210 to the acceleration unit 230 in the form of instructions. When the acceleration unit 230 actually performs accelerated computation, it directly addresses these parameters (e.g., weights) in memory 210 based on their addresses and temporarily stores them in its on-chip memory. When the acceleration unit 230 actually performs accelerated computation, the processing unit 220 also sends the inputs for accelerated computation to the acceleration unit 230, which are then temporarily stored in the acceleration unit 230's on-chip memory. In this way, the acceleration unit 230 can perform inference computation based on these inputs and the parameters (e.g., weights) required for accelerated computation.

[0120] Internal structure of processing unit and acceleration unit

[0121] The following is combined Figure 3 The internal structure diagram of the processing unit 220 and the acceleration unit 230 is shown in detail, illustrating how the processing unit 220 schedules the acceleration unit 230 to perform its work.

[0122] like Figure 3 As shown, the processing unit 220 includes multiple processor cores 222 and a cache 221 shared by the multiple processor cores 222. Each processor core 222 includes an instruction fetch unit 203, an instruction decode unit 224, an instruction issue unit 225, and an instruction execution unit 226.

[0123] Instruction fetch unit 223 is used to move the instruction to be executed from memory 210 to instruction register (which may be...). Figure 3 The instruction is stored in one of the registers in the register file 229 shown, and the next fetch address is received or calculated according to the fetch algorithm, which may include, for example, incrementing or decrementing the address based on the instruction length.

[0124] After the instruction is fetched, the processing unit 220 enters the instruction decoding stage. The instruction decoding unit 224 decodes the fetched instruction according to a predetermined instruction format to obtain the operand acquisition information required by the fetched instruction, thereby preparing for the operation of the instruction execution unit (arithmetic unit) 226. Operand acquisition information includes, for example, pointers to immediate values, registers, or other software / hardware that can provide source operands.

[0125] The instruction issuing unit 225 is located between the instruction decoding unit 224 and the instruction execution unit (arithmetic unit) 226. It is used for instruction scheduling and control to efficiently allocate each instruction to different instruction execution units (arithmetic units) 226, making parallel operation of multiple instructions possible.

[0126] After instruction issuing unit 225 issues an instruction to instruction execution unit (arithmetic unit) 226, instruction execution unit (arithmetic unit) 226 begins executing the instruction. For general logical operations or simple arithmetic operations, instruction execution unit (arithmetic unit) 226 executes them directly. However, if instruction execution unit (arithmetic unit) 226 determines that the instruction should be executed by acceleration unit 230, it forwards the instruction to the corresponding acceleration unit 230 for execution. For example, if the instruction is a deep learning model inference instruction, instruction execution unit (arithmetic unit) 226 will not execute the instruction directly, but will instead send the instruction to acceleration unit 230 via the bus for execution by acceleration unit 230.

[0127] Since general logical operations or simple arithmetic operations are performed by the instruction execution unit 226, it can serve as the arithmetic unit in this embodiment of the present disclosure. It may need to perform operations on different types of data, thus requiring conversion of data in different formats. Therefore, the instruction execution unit (arithmetic unit) 226 may implement the data format conversion scheme of this embodiment of the present disclosure, as detailed later, serving as the execution entity of this embodiment of the present disclosure. Its structure will be detailed later.

[0128] Figure 3The acceleration unit 30 shown is not limited to an NPU, but can also be a GPGPU. A GPGPU, or General Purpose Graphics Processing Unit, is a graphics processor that uses graphics processing to perform general-purpose computing tasks that would otherwise be handled by a central processing unit (CPU). These general-purpose computing tasks are often unrelated to graphics processing. Due to the powerful parallel processing capabilities and programmable pipelines of modern graphics processors, stream processors can handle non-graphics data. Especially when dealing with Single Instruction Multiple Data (SIMD) applications, where the computational load of data processing far exceeds the needs of data scheduling and transmission, general-purpose graphics processors significantly outperform traditional CPU applications, thus playing an acceleration role and belonging to the acceleration unit 30, just like an NPU. Additionally, the acceleration unit 30 can also include a CPU, GPU, FPGA, ASIC, etc.

[0129] The general structure of the acceleration unit 30 is as follows: Figure 3 Its internal structure includes multiple 236 cores. Figure 3 Four cores are shown, but those skilled in the art will understand that the acceleration unit 230 may also contain other numbers of cores 236, command processor 237, direct memory access mechanism 235, and bus channel 231.

[0130] Bus channel 231 is the channel through which instructions enter and exit the acceleration unit 230 via the bus.

[0131] Direct Memory Access (DMA) mechanism 235 is a feature provided by some computer bus architectures that allows data to be written directly from an external device to the computer motherboard's memory. This method significantly improves data access efficiency compared to the previous method where all data transfers between devices had to go through a processing unit. Because of this mechanism, the core of the acceleration unit 230 can directly access memory 210 and read parameters from accelerated operations (such as the weights of each node), greatly improving data access efficiency.

[0132] Command processor 237 distributes instructions sent from processing unit 220 to acceleration unit 230 to core 236 for execution. Instruction execution unit 226 sends a sequence of instructions to be executed to acceleration unit 230. This sequence of instructions, after entering from bus channel 231, is cached in command processor 237, which then selects core 236 and assigns the instruction sequence to it for execution. Additionally, command processor 237 is also responsible for synchronization operations between cores 236.

[0133] Acceleration unit core

[0134] Figure 4 This is an internal structural diagram of an acceleration unit core according to an embodiment of the present disclosure.

[0135] In one embodiment, such as Figure 4 As shown, the acceleration unit core 236 includes a tensor engine 310, a pooling engine 320, a memory copy engine 330, a sequencer 350, an instruction cache 340, on-chip memory 360, and a constant buffer 370.

[0136] The instruction sequence allocated by the command processor 237 to the acceleration unit core 236 is first cached in the instruction register 340. Then, the sequencer 350 fetches instructions from the instruction register 340 in a first-in-first-out order and allocates them to the tensor engine 310 or the pooling engine 320 for execution, depending on the nature of the instructions.

[0137] Tensor engine 310 is responsible for handling operations such as convolution and matrix multiplication in deep learning models. When performing these operations, calculations may involve data in various formats, such as integers and floating-point numbers, requiring data format conversion. Therefore, tensor engine 310 may also serve as a computation unit, becoming the main body executing the data format conversion scheme of this embodiment.

[0138] Pooling Engine 320 is responsible for handling pooling operations in deep learning models.

[0139] The memory copy engine 330 is a unit specifically designed to handle data copying. This data copying includes copying some data from the on-chip memory 360 to the shared memory of each core 236, or to the on-chip memory 360 of other cores 236, when the on-chip memory 360 may overflow.

[0140] The sequencer 350 determines whether to assign the instruction to the tensor engine 310, the pooling engine 320, or the memory copy engine 330 based on the nature of the retrieved instruction, such as convolution, matrix multiplication, pooling, or data copying.

[0141] On-chip memory 360 is the core memory for parameters (such as weights) required for accelerated computation, as well as the actual inputs and various intermediate results used in the accelerated computation. Constant buffer 370 is a buffer that stores some constants involved in the accelerated computation (e.g., hyperparameters in a neural network model). As described above, during the process of processing unit 220 pre-configuring the accelerated computation in acceleration unit 230, processing unit 220 sends the addresses of the parameters and constants required for the accelerated computation in memory 210 to acceleration unit 230 in the form of instructions. For weights, acceleration unit 230 fetches them from the corresponding location in memory 210 and places them in on-chip memory 360 before the actual accelerated computation. For constants (such as hyperparameters), acceleration unit 230 fetches them from the corresponding location in memory 210 and places them in constant buffer 370 before the actual accelerated computation. Furthermore, when the instructions for the actual accelerated computation are allocated to core 236 by command processor 237 for execution, the inputs in the instructions (inputs for the accelerated computation) are also stored in on-chip memory 360. In addition, after the tensor engine 310 and the pooling engine 320 perform convolution or pooling operations, the various intermediate results are also stored in the on-chip memory 360.

[0142] arithmetic unit

[0143] As mentioned above, Figure 3 The instruction execution unit 226 can be used as a computation unit to perform data format conversion when the processing unit 220 needs to process data of different data formats while performing tasks. Figure 4 The tensor engine 310 can also be used as a computing unit to perform data format conversion when the acceleration unit 230 needs to process data of different formats while performing tasks. Figure 5 The structure of the arithmetic unit is shown to be suitable for both Figure 3 The instruction execution unit 226 is also applicable to Figure 4 Tensor Engine 310 in the middle.

[0144] Figure 5The arithmetic units 226, 310 shown include multiple data paths, such as an integer data path 410 and a floating-point data path 420. If the arithmetic units 226, 310 also need to process operations on other data types, they may also include data paths for other data types. The sum of the related units used for processing operations on a certain data type constitutes one data path. For example, the sum of all units used for integer operations is the integer data path 410, and the sum of all units used for floating-point operations is the floating-point data path 420. A feature of this embodiment is that the related units for data format conversion are placed in one of the data paths, such as the integer data path 410. Therefore, it reuses the same data path to support arbitrary conversions between multiple data types. Instead of creating a separate data path for each conversion between two different data formats, it reuses an existing data path, improving the versatility of the hardware implementation for data type conversion and reducing the hardware resource consumption caused by data type conversion.

[0145] Integer data path

[0146] Since integer data path 410 needs to perform various operations on integers, it must include a logic operation unit 440 for regular integer logical operations and an integer adder 450 for integer addition. Integer subtraction is generally converted into addition. It may also contain units for integer multiplication and division (not shown).

[0147] In addition to the commonly used integer arithmetic components such as the logic operation unit 440 and the integer adder 450 mentioned above, the integer data path 410 of this embodiment may also include a first format conversion unit 411, a leading zero counting unit 412, a left shift unit 413, an exponent adjustment unit 414, an overflow detection unit 415, an overflow processing unit 416, a conversion-unformatted determination unit 417, a right shift unit 418, a rounding unit 419, and a second format conversion unit 421.

[0148] The first format conversion unit 411 converts the source data format into an intermediate data format according to a first rule. The second format conversion unit 421 converts the intermediate data format into the target data format according to a second rule.

[0149] The source data format is the data format before conversion in this embodiment of the disclosure, and the target data format is the data format after conversion in this embodiment of the disclosure. This embodiment of the disclosure converts the source data format into the target data format. Each data format corresponds to a data type. In the foregoing explanation of terms, combined with... Figure 6AThe `-J` directive describes the data structures for the following data formats: 32-bit floating-point numbers, 32-bit tensor floating-point numbers, 16-bit floating-point numbers, 16-bit human brain floating-point numbers, 32-bit signed integers, 32-bit unsigned integers, 16-bit signed integers, 16-bit unsigned integers, 8-bit signed integers, and 8-bit unsigned integers. It should be understood that the above data formats are merely examples, and other data formats are possible. The source data format can be any of these. The target data format can also be any of these.

[0150] The existing approach to converting source data format to target data format involves setting up dedicated hardware, i.e., dedicated data paths, for each pair of different data types, according to the conversion method between these two types. The number of combinations between any two different data types corresponds to the number of data paths. Therefore, this approach has poor versatility, requires numerous data paths, and consumes significant resources. This disclosure cleverly constructs an intermediate data format. This intermediate data format considers existing fields encompassing all possible data formats in its field settings and has a bit length greater than or equal to the corresponding bit length of existing fields in various possible data formats, thus removing obstacles to conversion between any data type and the intermediate data format. In this way, converting the source data format to the universal intermediate data format, and then converting the intermediate data format to the target data format, completes the conversion between any source and target data formats. This data path is unique and can be reused with existing data paths in the processing unit (such as integer data paths), improving the versatility of the hardware implementation for data type conversion.

[0151] because Figure 6A The -D data format contains three fields: sign bit, exponent bit, and mantissa bit. The 6E-J data format contains two fields: sign bit and integer bit. Therefore, the intermediate data format includes four fields: sign bit, exponent bit, integer bit, and mantissa bit. Figure 7 As shown, this represents the union of fields from all possible data structures. For the sign bit, Figure 6A The sign bit in -J is always 1 bit, therefore Figure 7 The sign bit of the intermediate data format can also be set to 1 bit. For the exponent bit, Figure 6A The exponent of -J can be either 8 bits or 5 bits; the maximum value of 8 bits is used. Therefore... Figure 7 The exponent bits in the intermediate data format need to be set to greater than or equal to 8 bits. Specifically, in Figure 7 The middle digit is set to 9 bits. For the mantissa digits... Figure 6A The mantissa in -J can have 23, 10, or 7 digits, therefore... Figure 7 The mantissa of the intermediate data format needs to be greater than or equal to 23 bits, specifically 31 bits. Figure 6AThe integer bits in -J can be 31, 32, 15, 16, 7, or 8 bits. Since the mantissa bits are not needed when the intermediate data format represents an integer, the integer bits and mantissa bits can be shared when the intermediate data format represents an integer. This way, it is only necessary to ensure... Figure 7 The sum of the number of digits in the integer part and the mantissa part is greater than or equal to 32. Specifically, in Figure 7 In this intermediate data format, the integer part consists of 2 bits, and the mantissa part consists of 31 bits, the sum of which is greater than or equal to 32. When representing floating-point numbers using this intermediate data format, the mantissa part represents the last digit of the floating-point number, the last integer part represents the integer digits before the decimal point, and the second-to-last integer part represents the sign of the floating-point number. When representing integers using this intermediate data format, the entire integer is represented by a total of 33 bits, including the integer and mantissa parts.

[0152] like Figure 7 The intermediate data format shown includes sign bits, exponent bits, integer bits, and mantissa bits, encompassing all possible data formats and ensuring that the number of bits is greater than or equal to the maximum number of bits in the corresponding fields of various data formats. This allows any source data format to be converted to... Figure 7 The intermediate data format can also be Figure 7 It can convert intermediate data formats into any target data format.

[0153] Figure 7 The intermediate data format represents the value as: (-1)^sign*2^(exponent-255)*(integer[0]+fraction*2^-31-2*integer[1]), where ^ represents the exponent. sign represents the sign bit. When sign=0, (-1)^sign=1, and the whole value is positive. When sign=1, (-1)^sign=-1, and the whole value is negative. exponent is the value corresponding to the exponent bit. For example, the value corresponding to 111111111 is 2. 9-1 = 511. The reason for subtracting 255 from exponent is to balance the probabilities of 2^(exponent-255) being greater than 1 and less than 1. 255 is the middle number between 0000000000 and 111111111. Thus, the minimum value of 2^(exponent-255) is 2^(-255), and the maximum value is 2^(256), achieving a balance between the probabilities of being greater than 1 and less than 1. integer[0] is the last bit of the integer part. 0 represents a denormalized floating-point number, and 1 represents a normalized floating-point number because the value before the decimal point in a denormalized floating-point number is 0, while the value before the decimal point in a normalized floating-point number is 1. fraction represents the value corresponding to the mantissa. Since its decimal point is actually before the 31st digit of fraction, its actual represented mantissa value is fraction * 2^ - 31. Adding the integer part integer[0], integer[0] + fraction * 2^ - 31 is the absolute value of m in a = m × b^e. integer[1] is the second to last digit of the integer part, with 0 indicating a positive mantissa and 1 indicating a negative mantissa. Thus, if Figure 7 The last 33 bits are 100000000000000000000000000000000, which actually represents -1. If Figure 7 The last 33 bits are 10 11111111111111111111111111111111, which actually represents -0.000000000000000000000000000001. This setting avoids the situation where 100000000000000000000000000000000 and 0 ...

[0154] The first rule for converting source data format to intermediate data format is described in detail below. That is, for... Figure 6A How can I convert the following data formats (using -J) as source data: 32-bit floating-point numbers, 32-bit tensor floating-point numbers, 16-bit floating-point numbers, 16-bit human brain floating-point numbers, 32-bit signed integers, 32-bit unsigned integers, 16-bit signed integers, 16-bit unsigned integers, 8-bit signed integers, and 8-bit unsigned integers? Figure 7 The intermediate data format.

[0155] against Figure 6A -B represents a 32-bit floating-point number or a 32-bit tensor floating-point number, and the sign bit of the 32-bit floating-point number or the 32-bit tensor floating-point number is... Figure 6A -B bit 31) is filled into the sign bit of the intermediate data format. Figure 7(bit 42). The last 7 exponent bits of the 32-bit floating-point number or 32-bit tensor floating-point number ( Figure 6A -B bits 23-29) are filled into the last 7 exponent bits of the intermediate data format. Figure 7 Bits 33-39). The first exponent bit of the 32-bit floating-point number or 32-bit tensor floating-point number ( Figure 6A -B bit 30) is filled into the first exponent bit of the intermediate data format. Figure 7 (bit 41). The first exponent bit of the 32-bit floating-point number or 32-bit tensor floating-point number ( Figure 6A The negative of bit 30 of -B is filled into the second exponent bit of the intermediate data format. Figure 7 (bit 40). If the 32-bit floating-point number or 32-bit tensor floating-point number is a denormalized floating-point number, the last bit of the integer bits of the intermediate data format ( Figure 7 Fill the last bit (31) with 0. If the 32-bit floating-point number or 32-bit tensor floating-point number is a normalized floating-point number, fill the last bit (31) of the integer bits of the intermediate data format with 0. Figure 7 The second-to-last integer bit (31) is filled with 1. Since the source data format is a floating-point number, not an integer, and does not involve the sign of the integer, the second-to-last integer bit of the intermediate data format can be filled with 1. Figure 7 For digit 32), fill in 0. From Figure 7 The first digit of the mantissa in the intermediate data format ( Figure 7 Starting from bit 30, the mantissa bits of the 32-bit floating-point number or 32-bit tensor floating-point number are sequentially inserted. For Figure 6A For example, the mantissa has 23 bits, which, after being inserted, will occupy [a certain number of bits]. Figure 7 The first 23 bits of the mantissa (bits 30 to 8), and the remaining bits 0-7, can be filled with 0. For Figure 6B For example, the last digit has 10 digits; after inserting it, it occupies [a certain amount of space Figure 7 The first 10 digits of the last digit (digits 30-21), and the remaining digits 0-20, can be filled with 0.

[0156] against Figure 6C The 16-bit floating-point number, the sign bit of the 16-bit floating-point number ( Figure 6C Bit 15) is filled into the sign bit of the intermediate data format. Figure 7 (bit 42). The last 4 bits of the 16-bit floating-point number (exponent bits) Figure 6C Bits 10-13 are filled into the last 4 exponent bits (bits 33-36) of the intermediate data format. The first exponent bit of the 16-bit floating-point number is then filled into the first exponent bit (bits 10-13). Figure 6C Bit 14) is filled into the first index bit of the intermediate data format. Figure 7 (bit 41), the first exponent bit of the 16-bit floating-point number ( Figure 6CThe opposite of bit 14 is filled into the 5th-8th exponent bits from the end of the intermediate data format. Figure 7 Bits 37-40). If the 16-bit floating-point number is a denormalized floating-point number, the last bit of the integer part of the intermediate data format (bits 37-40) will be used. Figure 7 Fill bit 31 with 0. If the 16-bit floating-point number is a normalized floating-point number, fill the last bit of the integer part of the intermediate data format with 0. Figure 7 Fill bit 31 with 1. Fill the second to last integer bit of the intermediate data format with 1. Figure 7 The 32nd digit of the intermediate data format is uniformly filled with 0. The first digit of the last digit (of the intermediate data format) is taken from the first digit of the last digit. Figure 7 Starting from bit 30, the mantissa bits of the 16-bit floating-point number are sequentially inserted. A 16-bit floating-point number has 10 mantissa bits, therefore... Figure 7 The digits 30 to 21. The remaining... Figure 7 The digit is 0-20, and you can fill in 0.

[0157] against Figure 6D The 16-bit human brain floating-point number, the sign bit (bit 15) of the 16-bit human brain floating-point number is filled into the sign bit of the intermediate data format. Figure 7 (bit 42). The last 7 exponent bits of the 16-bit human brain floating-point number ( Figure 6D Bits 7-13) are filled into the last 7 exponent bits of the intermediate data format. Figure 7 (bits 33-39). The first exponent of the 16-bit human brain floating-point number ( Figure 6D Bit 14) is filled into the first index bit of the intermediate data format. Figure 7 (bit 41). The first exponent of the 16-bit human brain floating-point number ( Figure 6D The opposite of bit 14 is filled into the second exponent bit of the intermediate data format. Figure 7 (bit 40). If the 16-bit human brain floating-point number is a denormalized floating-point number, the last bit of the integer part of the intermediate data format ( Figure 7 Fill bit 31 with 0; if the 16-bit human brain floating-point number is a normalized floating-point number, fill the last bit of the integer part of the intermediate data format with 0; Figure 7 Fill bit 31 with 1. Fill the second to last integer bit of the intermediate data format with 1. Figure 7 The 32nd digit of the intermediate data format is uniformly filled with 0. The first digit of the last digit (of the intermediate data format) is taken from the first digit of the last digit. Figure 7 Starting with bit 30, the mantissa bits of the 16-bit floating-point number are inserted sequentially. The mantissa bits of a 16-bit floating-point number are 7 bits. After insertion, Figure 7 Bits 30 to 24 are the mantissa bits of a floating-point number in the human brain, while bits 0 to 23 are unoccupied and can be filled with 0.

[0158] against Figure 6E32-bit signed integers Figure 6F 32-bit unsigned integer, Figure 6G 16-bit signed integer, Figure 6H 16-bit unsigned integer, Figure 6I 8-bit signed integer, or Figure 6J The 8-bit unsigned integers are converted into 33-bit signed integers, and then converted to an intermediate data format. A 33-bit signed integer is an integer in which the highest bit (bit 32) represents the sign of the number, and the remaining bits 0-31 represent the absolute value of the number.

[0159] for Figure 6E For a 32-bit signed integer, it is already signed and does not require an additional sign bit. It differs from a 33-bit signed integer only in the last bit; therefore, adding a 0 to the end of the 32-bit signed integer yields a 33-bit signed integer. Figure 6F For a 32-bit unsigned integer, it is inherently unsigned. To obtain a 33-bit signed integer, a sign bit of 0 (representing a positive number) needs to be added before it. Figure 6G For a 16-bit signed integer, it is already signed and does not require an additional sign bit. It differs from a 33-bit signed integer by 17 bits at the end; therefore, adding 17 zeros to the end of the 16-bit signed integer yields the 33-bit signed integer. Figure 6H For a 16-bit unsigned integer, it is inherently unsigned, so a sign bit of 0 (indicating a positive number) needs to be added before it. However, since 17 bits and 33 bits still differ by 16 bits, 16 zeros need to be added to the end to obtain a 33-bit signed integer. Figure 6I For an 8-bit signed integer, it is already signed and does not require an additional sign bit. It differs from a 33-bit signed integer by 25 bits at the end; therefore, adding 25 zeros to the end of the 8-bit signed integer yields a 33-bit signed integer. Figure 6J For an 8-bit unsigned integer, it is unsigned by itself. A sign bit 0 (to indicate a positive number) needs to be added in front of it. However, there is still a 24-bit difference between 9 bits and 33 bits. 24 bits need to be added to the end of it to get a 33-bit signed integer.

[0160] Then, convert the 33-bit signed integer to... Figure 7 The intermediate data format. Since the sign of an integer is not represented by a sign bit, it can be... Figure 7 The sign bit (bit 42) of the intermediate data format is uniformly set to 0. Figure 7In essence, integer values ​​are represented using 31 mantissa bits. Since the decimal point is placed before the first mantissa bit by default, it essentially represents the integer as a fraction with the decimal point before the 31st mantissa bit multiplied by 2 to the power of 31. Because the 9 exponent bits are around the mean 255 (2... 8 -1) The layout aims to balance the probability of the power being greater than 1 and less than 1. Therefore, when the 9 exponent bits are set to the binary value corresponding to 286, 286 - 255 = 31, which is exactly equivalent to multiplying by 2 to the power of 31. Therefore, when converting a 33-bit signed integer to an intermediate data format, the 9 exponent bits are fixed to the binary value corresponding to 286. The integer and mantissa bits of the intermediate data format are filled with the 33-bit signed integer. The integer and mantissa bits of the intermediate data format together are exactly 33 bits, as shown below. Figure 7 As shown.

[0161] The intermediate data format converted as described above may not be normalized; it is very likely to be unnormalized. For example, the last bit of the integer part of the intermediate data format may be 0, and the first few bits of the mantissa may also be 0. Considering that the embodiments of this disclosure implement the conversion of various source data formats to different target data formats through intermediate data formats, in order to ensure that the conversion to different target data formats is error-free, the data in the intermediate data format should follow certain standards. In the embodiments of this disclosure, the data in the intermediate data format must first be normalized. That is, if the last bit of the integer part of the intermediate data format is 0, and the first few bits of the mantissa are also 0, it needs to be normalized by shifting the data so that the last bit of the integer part of the intermediate data format is 1, becoming a normalized floating-point number. For example, assuming the last bit of the integer part in the intermediate data format is 0, the mantissa is 0000010101010001110110001111101, and the exponent is 2^67, the actual floating-point number it represents is 0.0000010101010001110110001111101 × 2 267 -255 = 0.0000010101010001110110001111101 × 2 12 The floating-point number is unnormalized. Converting it to a normalized floating-point number results in 1.0101010001110110001111101 × 2 6 .

[0162] The process of converting a denormalized floating-point number to a normalized floating-point number is as follows: First, the leading zero counting unit 412 checks the number of consecutive 0s starting from the highest bit in the sequence formed by the last bit of the integer part and the mantissa part of the intermediate data format. In the example above, there are 6 consecutive 0s. Then, the left shift unit 413 shifts the number of consecutive 0s to the left out of the intermediate data format. In the example above, 0.0000010101010001110110001111101 is shifted 6 bits to the left. In this way, the first 6 0s of the floating-point number are all shifted to the left out of the intermediate data format, leaving 6 empty bits on the right to be filled with 0s. In the example above, 0.0000010101010001110110001111101 becomes 1.0101010001110110001111101000000 after shifting all bits to the left by 6 bits. This transforms it into a normalized floating-point number. Shifting left by a certain number of bits is equivalent to multiplying m by 2 raised to the power of that number of bits in a = m × b^e. In the example above, shifting left by 6 bits is equivalent to m multiplied by 2 raised to the power of 6. Therefore, e needs to be subtracted by that number of bits to keep a constant. Thus, the exponent adjustment unit 414 adjusts the exponent bits, reducing the value corresponding to the exponent bits by the specified number of bits. In the example above, the exponent represented by the exponent bits is 267, which is changed to 261. 261 - 255 = 6, thus transforming the denormalized floating-point number 0.0000010101010001110110001111101 × 2 12 This becomes a normalized floating-point number 1.0101010001110110001111101×2 6 .

[0163] By converting the denormalized floating-point numbers in the intermediate data structure into normalized floating-point numbers, a unified standard is achieved after conversion to the second data structure, which is beneficial for the consistency of the conversion results.

[0164] To prevent overflow when converting intermediate data format to the target data format (e.g., the target data format is...), Figure 6C The 16-bit floating-point number shown has 5 exponent bits and 10 mantissa bits, representing a maximum value of 1.1111111111 × 2. 16 If the value exceeds the target data format after conversion, an overflow occurs. The overflow detection unit 415 determines that the data in the intermediate data format overflows after being converted to the target data format.

[0165] Overflow occurs when a value, after being converted to a certain data format, exceeds the range of values ​​that format can represent. Upflow occurs when a value, after being converted to a certain data format, exceeds the maximum value that that format can represent. Downflow occurs when a value, after being converted to a certain data format, is less than the minimum value that that format can represent.

[0166] Figure 8 The conditions for determining overflow and underflow are shown for various target data formats.

[0167] If the target data format is a 32-bit floating-point number, no upward or downward overflow will occur. This is because... Figure 6A As shown in -J, 32-bit floating-point numbers have the most bits in both the exponent and mantissa, thus representing the widest dynamic range. Figure 6B The J data structure will not overflow upwards or downwards when converted to a 32-bit floating-point number.

[0168] If the target data format is a 32-bit tensor floating-point number, Figure 6A -J shows all the data structures where the exponent is 8 bits. Figure 6A 32-bit floating-point numbers are the most common, but Figure 6A A 32-bit floating-point number has more mantissa bits than its base value. Therefore, if an overflow occurs when converting to a 32-bit tensor floating-point number, the source data format is a 32-bit floating-point number, which is equivalent to... Figure 7 The exponent digits (digits 41-33) represent the value 383. Since the value represented by the exponent digits needs to maintain a balance between the probabilities of the entire power value being greater than 1 and less than 1, the intermediate value is 255. 383 - 255 = 128. The value 128 represents... Figure 6B The 8-bit exponent will exceed the upper limit and cannot be represented, thus causing an upward overflow. When the target data format is a 32-bit tensor floating-point number, downward overflow will not occur.

[0169] If the target data format is a 16-bit floating-point number Figure 7 If the value corresponding to the 9-bit exponent in the intermediate data format is greater than or equal to 271, then an upward overflow is determined. The reason is that... Figure 7 The median value of the 9-bit exponent in the intermediate data format is 255. 271 - 255 = 16. At this point, the actual exponent value represented by the 9-bit exponent is greater than or equal to 16. Figure 6C The 5 exponent bits represent a dynamic range of -14 to 15. Therefore, the actual exponent value represented by the 9 exponent bits exceeds... Figure 6C The 5-bit exponent represents the upper limit of the exponent value, which overflows upwards. Figure 7 If the value corresponding to the 9-bit exponent in the intermediate data format is less than or equal to 240, then it is determined to overflow downwards. This is because 240 - 255 = -15, which is less than... Figure 6C The 5-bit exponent represents the lower limit of the exponent value as -14.

[0170] If the target data format is 16-bit floating-point numbers... Figure 7 If the value corresponding to the 9-bit exponent in the intermediate data format is greater than or equal to 383, then an upward overflow is determined. The reason is that... Figure 7 The median value of the 9-bit exponent in the intermediate data format is 255. 383 - 255 = 128. In this case, the actual exponent value represented by the 9-bit exponent is greater than or equal to 128. Figure 6D The 8-bit exponent represents a dynamic range of -126 to 127. Therefore, the actual exponent value represented by the 9-bit exponent exceeds... Figure 6D The 8-bit exponent represents the upper limit of the exponent value, which overflows upwards. Figure 7 If the value corresponding to the 9-bit exponent in the intermediate data format is less than or equal to 128, then it is determined to overflow downwards. This is because 128 - 255 = -127, which is less than... Figure 6D The 8-bit exponent represents the lower limit of the exponent value as -126.

[0171] If the target data format is a 32-bit signed integer Figure 7 If the value corresponding to the 9-bit exponent in the intermediate data format is greater than or equal to 286 and the sign bit is 0, then an upward overflow is determined. Figure 7 The median value of the 9-bit exponent in the intermediate data format is 255. 286 - 255 = 31. Figure 7 The intermediate data format has a 9-bit exponent value greater than or equal to 286, which actually means that... Figure 7 The intermediate data format is represented by a decimal point. Figure 7 The number before 30 is multiplied by 2 to the power of 31 or higher. Multiplying by 2 to the power of 31 or higher converts to... Figure 6E A number with only 31 integer digits will always overflow. If the sign is 0, it indicates a positive number, and therefore overflows upwards. Figure 7 If the value corresponding to the 9-bit exponent in the intermediate data format is greater than or equal to 286 and the sign bit is 1, then it is determined to overflow downwards.

[0172] If the target data format is a 32-bit unsigned integer Figure 7 If the value corresponding to the 9-bit exponent in the intermediate data format is greater than or equal to 287, then an upward overflow is determined. Figure 7 The median value of the 9-bit exponent in the intermediate data format is 255. 287 - 255 = 31. Figure 7 The intermediate data format has a 9-bit exponent value greater than or equal to 287, which actually means that... Figure 7 The intermediate data format is represented by a decimal point. Figure 7 The number before 30 is multiplied by 2 to the power of 32 or higher. Multiplying by 2 to the power of 31 or higher converts to... Figure 6F A number with only 32 integer digits will always overflow. If Figure 7 If the value corresponding to the 9-bit exponent in the intermediate data format is less than or equal to 254, then it is determined to overflow downwards. 254-255=-1. Figure 7 The intermediate data format has a 9-bit exponent value that is less than or equal to 254, which actually means that... Figure 7 The intermediate data format is represented by a decimal point. Figure 7 Numbers before 30 multiplied by 2 to the power of -1 or less cannot be converted. Figure 6D The integer in the variable determines the downward overflow.

[0173] If the target data format is a 16-bit signed integer Figure 7 If the value corresponding to the 9-bit exponent in the intermediate data format is greater than or equal to 270 and the sign bit is 0, then an upward overflow is determined. Figure 7 The median value of the 9-bit exponent in the intermediate data format is 255. 270 - 255 = 15. Figure 7 The intermediate data format has a 9-bit exponent value greater than or equal to 270, which actually means that... Figure 7 The intermediate data format is represented by a decimal point. Figure 7 The number before 30 is multiplied by 2 to the power of 15 or higher. Multiplying by 2 to the power of 15 or higher converts to... Figure 6G A number with only 15 integer digits will always overflow. If the sign is 0, it indicates a positive number, and therefore overflows upwards. Figure 7 If the value corresponding to the 9-bit exponent in the intermediate data format is greater than or equal to 270 and the sign bit is 1, then it is determined to overflow downwards.

[0174] If the target data format is a 16-bit unsigned integer Figure 7 If the value corresponding to the 9-bit exponent in the intermediate data format is greater than or equal to 271, then an upward overflow is determined. Figure 7 The median value of the 9-bit exponent in the intermediate data format is 255. 271 - 255 = 16. Figure 7 The intermediate data format has a 9-bit exponent value greater than or equal to 271, which actually means that... Figure 7 The intermediate data format is represented by a decimal point. Figure 7 The number before 30 is multiplied by 2 to the power of 16 or higher. Multiplying by 2 to the power of 16 or higher converts to... Figure 6H A number with only 16 integer digits will always overflow. If Figure 7 If the value corresponding to the 9-bit exponent in the intermediate data format is less than or equal to 254, then it is determined to overflow downwards. 254-255=-1. Figure 7 The intermediate data format has a 9-bit exponent value that is less than or equal to 254, which actually means that... Figure 7 The intermediate data format is represented by a decimal point. Figure 7 Numbers before 30 multiplied by 2 to the power of -1 or less cannot be converted. Figure 6HThe integer in the variable determines the downward overflow.

[0175] If the target data format is an 8-bit signed integer Figure 7 If the value corresponding to the 9-bit exponent in the intermediate data format is greater than or equal to 262 and the sign bit is 0, then an upward overflow is determined. Figure 7 The median value of the 9-bit exponent in the intermediate data format is 255. 262 - 255 = 7. Figure 7 The intermediate data format has a 9-bit exponent value greater than or equal to 2^62, which actually means that... Figure 7 The intermediate data format is represented by a decimal point. Figure 7 The number before 30 is multiplied by 2 to the power of 7 or higher. Multiplying by 2 to the power of 7 or higher converts to... Figure 6I A number with only 7 integer digits will always overflow. If the sign is 0, it indicates a positive number, and therefore overflows upwards. Figure 7 If the value corresponding to the 9-bit exponent in the intermediate data format is greater than or equal to 262 and the sign bit is 1, then it is determined to overflow downwards.

[0176] If the target data format is an 8-bit unsigned integer Figure 7 If the value corresponding to the 9-bit exponent in the intermediate data format is greater than or equal to 263, then an upward overflow is determined. Figure 7 The median value of the 9-bit exponent in the intermediate data format is 255. 263 - 255 = 8. Figure 7 The intermediate data format has a 9-bit exponent value greater than or equal to 263, which actually means that... Figure 7 The intermediate data format is represented by a decimal point. Figure 7 The number before 30 is multiplied by 2 to the power of 8 or higher. Multiplying by 2 to the power of 8 or higher converts to... Figure 6J A number with only 8 integer digits will always overflow. If Figure 7 If the value corresponding to the 9-bit exponent in the intermediate data format is less than or equal to 254, then it is determined to overflow downwards. 254-255=-1. Figure 7 The intermediate data format has a 9-bit exponent value that is less than or equal to 254, which actually means that... Figure 7 The intermediate data format is represented by a decimal point. Figure 7 Numbers before 30 multiplied by 2 to the power of -1 or less cannot be converted. Figure 6J The integer in the variable determines the downward overflow.

[0177] The overflow processing unit 416 performs pre-processing on the overflow target data format. This pre-processing can be to set the exponent bit to a certain reserved value. For example, all 1s of the exponent bit can be reserved to represent the overflow situation. In this way, the all 1s of the exponent bit cannot express a normal exponent value, thus reducing the range of exponent values ​​that the exponent bit can express.

[0178] In another embodiment, it is necessary to convert the intermediate data format into the target data format to become standardized data, so that the conversion to various target data formats is uniform, that is, the result obtained by converting to a certain target data format is unique, thereby improving the uniformity of the results.

[0179] In this embodiment, the denormalization determination unit 417 determines whether the data in the intermediate data format is denormalized or an integer after being converted to the target data format. The determination method can be to determine the value represented by the intermediate data format according to the formula (-1)^sign*2^(exponent-255)*(integer[0]+fraction*2^-31-2*integer[1]), and then according to the above description... Figure 6A -J uses the data value calculation formula of each second data structure to determine each field in each second data structure and to determine whether the exponent bit is all 0. If it is all 0, it is normalized data; otherwise, it is denormalized data or an integer.

[0180] Then, the right shift unit 418 shifts the mantissa of the intermediate data format to the right according to the determination result of whether it is non-normalized data or an integer, so that the intermediate data format is shifted and converted into the target data format and then normalized data.

[0181] The following describes one implementation whereby the right shift unit 418 shifts the mantissa of the intermediate data format to the right based on the determination result of whether the data is denormalized or an integer. Those skilled in the art will understand that other methods can also be used to perform the right shift.

[0182] The shifted values ​​need to ensure that the meaningful data bits required by different target data formats are right-aligned, i.e. Figure 6A The significant data bits in the mantissa of -D are aligned to the right of the mantissa. Figure 6E The significant data bits in the integer part of -J are aligned to the right of the integer part. This is done so that uniform addition can be used when rounding.

[0183] In this implementation, if the target data format is a 32-bit floating-point number or a 32-bit tensor floating-point number, and it is determined to be denormalized data, the value corresponding to the exponent bit is T, and the intermediate data format is shifted to the right by (137-T) bits; if it is determined to be normalized data, the intermediate data format is shifted to the right by 8 bits. This is because... Figure 7 The intermediate data format has 31 mantissa bits, while Figure 6A -B has 23 mantissa bits, with an 8-bit difference in the middle. As mentioned above, it is necessary to ensure that the valid data bits are aligned to the right. Therefore, whether it is normalized or denormalized data, it needs to be shifted 8 bits to the right. If it is denormalized data, it needs to be shifted an additional (129-T) bits. As mentioned above, in the value T represented by the exponent bits of the intermediate format data, 128 is the middle offset, thus ensuring that the probability of b^e being greater than 1 and the probability of it being less than 1 in the final a = m × b^e are balanced. e = T - 128. When it is determined that the data in the intermediate data format is denormalized after being converted to the target data format, the mantissa bits of the intermediate data format should be shifted to the right by 1-e bits in addition to the fixed 8-bit shift (since the data is denormalized after conversion to the target data format, e here is mostly negative), thus making the converted data normalized. Substituting e = T - 128 into 1-e, we get that the intermediate data format should be shifted to the right by (137-T) bits. Only by converting it to the target data format can the obtained value be a normalized floating-point number.

[0184] In this implementation, if the target data format is a 16-bit floating-point number, and it is determined to be denormalized data, the value corresponding to the exponent bit is T, and the intermediate data format is shifted to the right by (2^62 - T) bits; if it is determined to be normalized data, the intermediate data format is shifted to the right by 21 bits. This is because... Figure 7 The intermediate data format has 31 mantissa bits, while Figure 6C The mantissa has 10 bits, with a difference of 21 bits in the middle. As mentioned above, the valid data bits need to be right-aligned. Therefore, whether the data is normalized or denormalized, it needs to be shifted 21 bits to the right. For denormalized data, an additional (241-T) bits need to be shifted. The process of deriving the additional (241-T) bits right shift is similar to the process of deriving the additional (129-T) bits right shift for 32-bit floating-point numbers or 32-bit tensor floating-point numbers, so it will not be repeated here.

[0185] In this implementation, if the target data format is a 16-bit human brain floating-point number, and it is determined to be denormalized data, the value corresponding to the exponent bit is T, and the intermediate data format is shifted to the right by (153-T) bits; if it is determined to be normalized data, the intermediate data format is shifted to the right by 24 bits. This is because... Figure 7The intermediate data format has 31 mantissa bits, while Figure 6D The mantissa is 7 bits, with a difference of 24 bits in the middle. As mentioned above, the valid data bits need to be right-aligned. Therefore, whether the data is normalized or denormalized, it needs to be shifted 24 bits to the right. If it is denormalized data, it needs to be shifted an additional (129-T) bits. The process of deriving the additional (129-T) bits right shift is similar to the process described above for 32-bit floating-point numbers or 32-bit tensor floating-point numbers, so it will not be repeated here.

[0186] In this embodiment, if the target data format is a 32-bit signed integer, a 32-bit unsigned integer, a 16-bit signed integer, a 16-bit unsigned integer, an 8-bit signed integer, or an 8-bit unsigned integer, and the value corresponding to the exponent bit is T, then the intermediate data format is shifted to the right by (286-T) bits. The derivation process is similar to that described above and will not be repeated here.

[0187] When the right shift unit 418 shifts the mantissa bits of the intermediate data format to the right, precision may be lost. This is because when the mantissa bits of the intermediate data format are shifted to the right, some mantissa bits may be shifted out of range. Figure 7 The right boundary of the 31-bit mantissa causes a loss of precision. For example, shifting the mantissa 0.1100010011000001100000101110101 to the right by 5 bits results in 0.0000011000100110000011000001011, where the rightmost 5 bits (10101) are shifted out of the field. If all bits shifted out of the field are automatically ignored, the loss of precision is significant. Therefore, in one embodiment, rounding is performed by rounding unit 419 to reduce the loss of precision.

[0188] The rounding unit 419, based on the right shift unit 418 shifting the mantissa bits of the intermediate data format to the right, determines whether a carry operation should be performed on the remaining mantissa bits in the intermediate data format. A carry operation involves adding 1 to the lowest remaining mantissa bit in the intermediate data format. Determining whether to perform a carry operation based on the shifted-out mantissa bits is existing technology. For example, shifting the mantissa 0.1100010011000001100000101110101 to the right by 5 bits results in 0.0000011000100110000011000001011; the 5 bits shifted out from the right boundary of the intermediate data format are 10101. Based on the value of these 5 bits, different existing criteria can be used to determine whether to perform the carry operation. These different criteria are existing and will not be elaborated here. If a carry is decided, add 1 to the last digit of 0.0000011000100110000011000001011, and it becomes 0.0000011000100110000011000001100.

[0189] The second format conversion unit 421 converts the intermediate data format into the target data format according to the second rule. The intermediate data format can be the intermediate data format that has undergone normalization, overflow detection and corresponding processing, and conversion followed by denormalization, or it can be the intermediate data format that omits one or more of the above processing steps.

[0190] The second rule according to one embodiment of this disclosure will now be described in detail.

[0191] In this embodiment, if the target data format is as follows Figure 6A The 32-bit floating-point number or 32-bit tensor floating-point number indicated by -B will be as follows: Figure 7 The sign bit of the intermediate data format shown ( Figure 7 Fill the 42nd bit with the following: Figure 6A -B indicates the sign bit of the 32-bit floating-point number or 32-bit tensor floating-point number. Figure 6A -B bit 31). Figure 7 The last 7 exponent bits of the intermediate data format shown ( Figure 7 Bits 33-39) are filled into the last 7 exponent bits of the 32-bit floating-point number or the 32-bit tensor floating-point number. Figure 6A -B bits 23-29). Figure 7 The first exponent bit of the intermediate data format shown ( Figure 7 Fill bit 41) to Figure 6A -B indicates the first exponent of a 32-bit floating-point number or a 32-bit tensor floating-point number. Figure 6A -B bit 30). Figure 7 The last 23 bits of the mantissa in the intermediate data format shown ( Figure 7 Bits 0-22) are used as the mantissa of the 32-bit floating-point number. Figure 6A (bits 0-22), or will Figure 7 The last 10 bits of the mantissa in the intermediate data format shown ( Figure 7 (bits 0-9) are used as the mantissa of the 32-bit tensor floating-point number. Figure 6B (0-9).

[0192] In this embodiment, if the target data format is as follows Figure 6C The 16-bit floating-point number shown will be as follows: Figure 7 The sign bit of the intermediate data format shown ( Figure 7 Fill the 42nd bit with the following: Figure 6C The sign bit of the 16-bit floating-point number shown Figure 6C (position 15). As follows Figure 7 The last 4 exponent bits of the intermediate data format shown ( Figure 7 Fill positions 33-36 into as shown Figure 6C The last 4 bits of the 16-bit floating-point number shown are the exponent bits. Figure 6C (Positions 10-13). As follows Figure 7 The first exponent bit of the intermediate data format shown ( Figure 7 Bit 41) is filled into the first exponent bit of the 16-bit floating-point number. Figure 6C (position 14). As follows Figure 7 The last 10 bits of the mantissa in the intermediate data format shown ( Figure 7 The bits 0-9 are used as the mantissa of the 16-bit floating-point number. Figure 6C (0-9).

[0193] In this embodiment, if the target data format is Figure 6D The 16-bit floating-point number shown will be as follows: Figure 7 The sign bit of the intermediate data format shown ( Figure 7 Fill the 42nd bit with the following: Figure 6D The sign bit of the 16-bit human brain floating-point number shown ( Figure 6D (position 15). As follows Figure 7 The last 7 exponent bits of the intermediate data format shown ( Figure 7 Fill bits 33-39) into Figure 6D The last 7 exponent bits of the 16-bit human brain floating-point number shown ( Figure 6D (positions 7-13). As follows Figure 7 The first exponent bit of the intermediate data format shown ( Figure 7 Fill bit 41) to Figure 6D The first exponent of the 16-bit human brain floating-point number shown ( Figure 6D (40th position). (Will be as follows) Figure 7 The last 7 bits of the mantissa in the intermediate data format shown ( Figure 7 (bits 0-6) as Figure 6D The mantissa of the 16-bit human brain floating-point number shown ( Figure 6D (0-6).

[0194] In this embodiment, if the target data format is Figure 6E The 32-bit signed integer shown will Figure 7 The integer bits of the intermediate data format shown ( Figure 7 (31-32) and the mantissa after removing the first digit ( Figure 7 (0-29) continues as Figure 6E The 32-bit signed integer shown. Figure 7 Bit 32 represents the sign of the integer, which is related to... Figure 6E The digit 31 is consistent.

[0195] In this embodiment, if the target data format is Figure 6FThe 32-bit unsigned integer shown will Figure 7 The integer and mantissa bits of the intermediate data format shown ( Figure 7 Take out the bits 0-32, remove the first bit (32), and use it as... Figure 6F The 32-bit unsigned integer shown.

[0196] In this embodiment, if the target data format is Figure 6G The 16-bit signed integer shown will Figure 7 The integer bits of the intermediate data format shown ( Figure 7 (31-32) and the last 17 digits removed ( Figure 7 The bits 0-13) continue as Figure 6G The 16-bit signed integer shown. Figure 7 Bit 32 represents the sign of the integer, which is related to... Figure 6G The digit 15 is consistent.

[0197] In this embodiment, if the target data format is Figure 6H The 16-bit unsigned integer shown will Figure 7 The intermediate data format shown is the integer part with the leading digits removed. Figure 7 (31) and removing the last 16 bits ( Figure 7 (bits 0-14) continue as Figure 6H The 16-bit unsigned integer shown.

[0198] In this embodiment, if the target data format is Figure 6I The 8-bit signed integer shown will Figure 7 The integer bits of the intermediate data format shown ( Figure 7 (31-32) and the last 25 digits removed ( Figure 7 The digits 0-5 continue as... Figure 6I The 8-bit signed integer shown. Figure 7 Bit 32 represents the sign of the integer, which is related to... Figure 6I The digit 7 is consistent.

[0199] In this embodiment, if the target data format is Figure 6J The 8-bit unsigned integer shown will Figure 7 The intermediate data format shown is the integer part with the leading digits removed. Figure 7 (31) and removing the last 24 bits ( Figure 7 The digits 0-6) continue as Figure 6J The 8-bit unsigned integer shown.

[0200] The computational unit that performs data format conversion is Figure 3In the case of instruction execution unit 226, data in the source data format is read from register 430 in register file 229 by instruction execution unit 226. Data in the target data format is written to register 430 by instruction execution unit 226.

[0201] The computational unit that performs data format conversion is Figure 4 In the case of the Tensor Engine 310, data in the source data format is read from the on-chip memory 360 by the Tensor Engine 310. Data in the target data format is written to the on-chip memory 360 by the Tensor Engine 310.

[0202] Furthermore, embodiments of this disclosure can be embodied in the form of a system-on-a-chip, which includes, for example... Figure 3 At least one of the processing unit 220 and acceleration unit 230 shown. A system-on-a-chip (SoC) refers to a complete system integrated on a single chip, in which all or some of the necessary electronic circuitry is grouped together. A complete system generally includes processing units or acceleration units, memory, and peripheral circuitry, etc.

[0203] Data format conversion methods

[0204] like Figure 9 As shown, according to one embodiment of this disclosure, a data format conversion method is provided, including:

[0205] Step 510: Convert the source data format into an intermediate data format according to the first rule;

[0206] Step 520: Convert the data in the intermediate data format into the data in the target data format according to the second rule, wherein the intermediate data format includes at least all the fields of the source data format and all the fields of the target data format, and the number of bits in the fields of the intermediate data format is not less than the number of bits in the corresponding fields of either the source data format or the target data format.

[0207] Optionally, the intermediate data format includes a sign bit, an exponent bit, an integer bit, and a mantissa bit. The value represented by the intermediate data format is: (-1)^sign*2^(exponent-255)*(integer[0]+fraction*2^-31-2*integer[1]), where ^ represents the exponent, sign represents the sign bit, sign=0 represents a positive number, sign=1 represents a negative number, exponent is the value corresponding to the exponent bit, integer[0] is the last bit of the integer bit, 0 represents a non-normalized floating-point number, 1 represents a normalized floating-point number, fraction represents the mantissa bit, integer[1] is the second to last bit of the integer bit, 0 represents a positive mantissa, and 1 represents a negative mantissa.

[0208] Optionally, after converting the source data format into an intermediate data format according to the first rule, the method further includes:

[0209] Check the number of consecutive 0s starting from the most significant bit in the sequence formed by the last bit of the integer part and the mantissa part;

[0210] Shift the number of consecutive zeros to the left out of the intermediate data format;

[0211] Adjust the exponent bit so that the value corresponding to the exponent bit is reduced by the number of bits.

[0212] Optionally, after adjusting the exponent digit to subtract the number from the value corresponding to the exponent digit, the method further includes:

[0213] After the data in the intermediate data format is converted into the target data format, it overflows.

[0214] Perform pre-processing on the overflowing target data format.

[0215] Optionally, after performing pre-processing on the overflowing target data format, the method further includes:

[0216] Determine whether the data in the intermediate data format, after being converted to the target data format, is either denormalized data or an integer;

[0217] Based on the determination of whether it is non-normalized data or an integer, the mantissa of the intermediate data format is shifted to the right, so that the intermediate data format is shifted and converted into the target data format, which is then normalized data.

[0218] Optionally, after shifting the mantissa of the intermediate data format to the right based on the determination result of whether it is denormalized data or an integer, the method further includes: shifting the mantissa of the intermediate data format to the right based on the right shift unit, and determining whether it is a mantissa left in the intermediate data format for carry operation.

[0219] The commercial value of the embodiments disclosed herein

[0220] This disclosure embodiment supports arbitrary conversion between multiple data types using a single data path, and can be reused with existing data paths. It can also convert from data types with a large range to data types with a small range, greatly reducing the area overhead of the chip for data format conversion (by more than half), and has good market prospects.

[0221] It should be understood that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the method embodiments are basically similar to the methods described in the apparatus and system embodiments, so the description is relatively simple, and relevant parts can be referred to in the description of other embodiments.

[0222] It should be understood that the foregoing describes specific embodiments of this specification. Other embodiments are within the scope of the claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0223] It should be understood that the use of a singular form to describe an element or to show only one element in the accompanying drawings does not imply that the number of such element is limited to one. Furthermore, modules or elements described or shown as separate herein may be combined into a single module or element, and modules or elements described or shown as single herein may be broken down into multiple modules or elements.

[0224] It should also be understood that the terminology and expressions used herein are for descriptive purposes only, and one or more embodiments described herein should not be limited to these terms and expressions. The use of these terms and expressions does not exclude any illustrative and descriptive equivalent features (or parts thereof), and it should be recognized that various modifications that may exist should also be included within the scope of the claims. Other modifications, variations, and substitutions may also exist. Accordingly, the claims should be considered to cover all such equivalents.

Claims

1. A computational unit, used as an instruction execution unit in a processing core of a processing unit or as a tensor engine in an acceleration unit core of an acceleration unit, wherein, The computational unit, acting as an instruction execution unit, receives and processes instructions transmitted by the instruction issuing unit and decoded by the instruction decoding unit. The computational unit, acting as a tensor engine, processes convolution and matrix multiplication operations in the deep learning model. The computational unit includes: The first format conversion unit is used to convert data in the source data format into an intermediate data format according to the first rule; The second format conversion unit is used to convert the intermediate data format into the target data format according to the second rule. The intermediate data format includes at least all the fields of the source data format and all the fields of the target data format, and the number of bits in the fields of the intermediate data format is not less than the number of bits in the corresponding fields of either the source data format or the target data format. The source data format is any one of a variety of data types, and the target data format is any one of the variety of data types other than the source data format; The operation unit includes multiple data paths, and the sum of the related units used to perform operations on a data type is one data path; the first format conversion unit and the second format conversion unit are set in one of the multiple data paths and are reused by each data path for data format conversion. The arithmetic unit includes an integer data path and a floating-point data path. The integer data path is used for integer arithmetic processing, and the floating-point data path is used for floating-point arithmetic processing. The first format conversion unit and the second format conversion unit are included in the integer data path or the floating-point data path. The integer data path includes a logic arithmetic unit for performing integer logical operations and an integer adder for performing integer addition processing.

2. The arithmetic unit according to claim 1, wherein, The intermediate data format includes a sign bit, an exponent bit, an integer bit, and a mantissa bit. The value represented by the intermediate data format is: (-1)^sign * 2^(exponent-255) *(integer[0]+fraction*2^-31-2*integer[1]), where ^ represents the exponent, sign represents the sign bit, sign=0 represents a positive number, sign=1 represents a negative number, exponent is the value corresponding to the exponent bit, integer[0] is the last bit of the integer bit, 0 represents a non-normalized floating-point number, 1 represents a normalized floating-point number, fraction represents the mantissa bit, integer[1] is the second to last bit of the integer bit, 0 represents a positive mantissa, 1 represents a negative mantissa.

3. The arithmetic unit according to claim 2, wherein, The exponent has 9 bits and the mantissa has 31 bits.

4. The arithmetic unit according to claim 2 further includes: A leading zero counting unit is used to check the number of consecutive 0s starting from the most significant bit in the sequence formed by the last bit of the integer part and the mantissa part; A left shift unit is used to shift the number of consecutive 0s to the left out of the intermediate data format; An exponent adjustment unit is used to adjust the exponent position so that the value corresponding to the exponent position is reduced by the number of values.

5. The arithmetic unit according to claim 2, further comprising: An overflow detection unit is used to determine that the data in the intermediate data format overflows after being converted into the target data format. The overflow processing unit is used to perform pre-defined processing on overflowing data in the target data format.

6. The arithmetic unit according to claim 5, wherein, The overflow detection unit determines overflow by at least one of the following: If the target data format is a 32-bit tensor floating-point number and the value corresponding to the exponent bit is 383, then an upward overflow is determined. If the target data format is a 16-bit floating-point number, and the value corresponding to the exponent bit is greater than or equal to 271, then it is determined to overflow upwards; if the value corresponding to the exponent bit is less than or equal to 240, then it is determined to overflow downwards. If the target data format is a 16-bit human brain floating-point number, and the value corresponding to the exponent bit is greater than or equal to 383, then it is determined to overflow upwards; if the value corresponding to the exponent bit is less than or equal to 128, then it is determined to overflow downwards. If the target data format is a 32-bit signed integer, and the value corresponding to the exponent bit is greater than or equal to 286 and the sign bit is 0, then it is determined to overflow upwards; if the value corresponding to the exponent bit is greater than or equal to 286 and the sign bit is 1, then it is determined to overflow downwards. If the target data format is a 32-bit unsigned integer, and the value corresponding to the exponent bit is greater than or equal to 287, then it is determined to overflow upwards; if the value corresponding to the exponent bit is less than or equal to 254, then it is determined to overflow downwards. If the target data format is a 16-bit signed integer, and the value corresponding to the exponent bit is greater than or equal to 270 and the sign bit is 0, then it is determined to overflow upwards; if the value corresponding to the exponent bit is greater than or equal to 270 and the sign bit is 1, then it is determined to overflow downwards. If the target data format is a 16-bit unsigned integer, and the value corresponding to the exponent bit is greater than or equal to 271, then it is determined to overflow upwards; if the value corresponding to the exponent bit is less than or equal to 254, then it is determined to overflow downwards. If the target data format is an 8-bit signed integer, and the value corresponding to the exponent bit is greater than or equal to 2^62 and the sign bit is 0, then it is determined to overflow upwards; if the value corresponding to the exponent bit is greater than or equal to 2^62 and the sign bit is 1, then it is determined to overflow downwards. If the target data format is an 8-bit unsigned integer, and the value corresponding to the exponent bit is greater than or equal to 263, then it is determined to overflow upwards; if the value corresponding to the exponent bit is less than or equal to 254, then it is determined to overflow downwards.

7. The arithmetic unit according to claim 2, further comprising: The denormalization determination unit is used to determine whether the data in the intermediate data format is denormalized or an integer after being converted into the target data format; The right shift unit is used to shift the mantissa of the intermediate data format to the right based on the determination result of whether it is denormalized data or an integer, so that the data after shifting and converting the intermediate data format into the target data format is normalized data.

8. The arithmetic unit according to claim 7, further comprising: The rounding unit is used to determine whether the rounding operation is performed on the mantissa of the intermediate data format after shifting the right bit out of the right bit shifting unit.

9. The arithmetic unit according to claim 7, wherein, The right shift unit shifts the mantissa bits of the intermediate data format to the right according to at least one of the following: If the target data format is a 32-bit floating-point number or a 32-bit tensor floating-point number, and it is determined to be denormalized data, the value corresponding to the exponent bit is T, and the intermediate data format is shifted to the right by (137-T) bits; if it is determined to be normalized data, the intermediate data format is shifted to the right by 8 bits. If the target data format is a 16-bit floating-point number, and it is determined to be denormalized data, the value corresponding to the exponent bit is T, and the intermediate data format is shifted to the right by (2^62 - T) bits; if it is determined to be normalized data, the intermediate data format is shifted to the right by 21 bits. If the target data format is a 16-bit human brain floating-point number, and it is determined to be denormalized data, the value corresponding to the exponent bit is T, and the intermediate data format is shifted to the right by (153-T) bits; if it is determined to be normalized data, the intermediate data format is shifted to the right by 24 bits. If the target data format is a 32-bit signed integer, a 32-bit unsigned integer, a 16-bit signed integer, a 16-bit unsigned integer, an 8-bit signed integer, or an 8-bit unsigned integer, and the value corresponding to the exponent bit is T, then the intermediate data format is shifted to the right by (286-T) bits.

10. The arithmetic unit according to claim 2, wherein, The first rule includes at least one of the following: If the source data format is a 32-bit floating-point number or a 32-bit tensor floating-point number, fill the sign bit of the 32-bit floating-point number or 32-bit tensor floating-point number into the sign bit of the intermediate data format; fill the last 7 exponent bits of the 32-bit floating-point number or 32-bit tensor floating-point number into the last 7 exponent bits of the intermediate data format; fill the first exponent bit of the 32-bit floating-point number or 32-bit tensor floating-point number into the first exponent bit of the intermediate data format; and fill the negative of the first exponent bit of the 32-bit floating-point number or 32-bit tensor floating-point number with... The second exponent bit of the intermediate data format is used; if the 32-bit floating-point number or 32-bit tensor floating-point number is a denormalized floating-point number, the last bit of the integer part of the intermediate data format is filled with 0; if the 32-bit floating-point number or 32-bit tensor floating-point number is a normalized floating-point number, the last bit of the integer part of the intermediate data format is filled with 1; starting from the first bit of the mantissa part of the intermediate data format, the mantissa parts of the 32-bit floating-point number or 32-bit tensor floating-point number are sequentially inserted, and the remaining bits of the mantissa part of the intermediate data format are filled with 0; If the source data format is a 16-bit floating-point number, fill the sign bit of the 16-bit floating-point number into the sign bit of the intermediate data format, fill the last 4 exponent bits of the 16-bit floating-point number into the last 4 exponent bits of the intermediate data format, fill the first exponent bit of the 16-bit floating-point number into the first exponent bit of the intermediate data format, and fill the inverse of the first exponent bit of the 16-bit floating-point number into the 5th to 8th exponent bits from the end of the intermediate data format; if the 16-bit floating-point number is a denormalized floating-point number, fill the last bit of the integer part of the intermediate data format with 0; if the 16-bit floating-point number is a normalized floating-point number, fill the last bit of the integer part of the intermediate data format with 1; starting from the first bit of the mantissa of the intermediate data format, sequentially insert the mantissa of the 16-bit floating-point number, and fill the remaining bits of the mantissa of the intermediate data format with 0. If the source data format is a 16-bit floating-point number, fill the sign bit of the 16-bit floating-point number into the sign bit of the intermediate data format, fill the last 7 exponent bits of the 16-bit floating-point number into the last 7 exponent bits of the intermediate data format, fill the first exponent bit of the 16-bit floating-point number into the first exponent bit of the intermediate data format, and fill the opposite of the first exponent bit of the 16-bit floating-point number into the second exponent bit of the intermediate data format; if the 16-bit floating-point number is a denormalized floating-point number, fill the last bit of the integer part of the intermediate data format with 0; if the 16-bit floating-point number is a normalized floating-point number, fill the last bit of the integer part of the intermediate data format with 1; starting from the first bit of the mantissa of the intermediate data format, sequentially insert the mantissa of the 16-bit floating-point number, and fill the remaining bits of the mantissa of the intermediate data format with 0. If the source data format is a 32-bit signed integer, a 32-bit unsigned integer, a 16-bit signed integer, a 16-bit unsigned integer, an 8-bit signed integer, or an 8-bit unsigned integer, convert the source data format to a 33-bit signed integer; set the sign bit of the intermediate data format to 0; set the exponent bit of the intermediate data format to the binary value corresponding to 286; and fill the integer bits and mantissa bits of the intermediate data format with the 33-bit signed integer.

11. The arithmetic unit according to claim 10, wherein, If the source data format is a 32-bit signed integer, a 32-bit unsigned integer, a 16-bit signed integer, a 16-bit unsigned integer, an 8-bit signed integer, or an 8-bit unsigned integer, the source data format is converted to a 33-bit signed integer, including: Add 1, 17, or 25 zeros to the end of the 32-bit signed integer, 16-bit signed integer, or 8-bit signed integer respectively to make a 33-bit signed integer. Add one 0 to the front of the 32-bit unsigned integer to make a 33-bit signed integer; Add one 0 to the front of the 16-bit unsigned integer or the 8-bit unsigned integer, and add 16 or 24 0s to the back respectively to make a 33-bit signed integer.

12. The arithmetic unit according to claim 2, wherein, The second rule includes at least one of the following: If the target data format is a 32-bit floating-point number or a 32-bit tensor floating-point number, fill the sign bit of the intermediate data format into the sign bit of the 32-bit floating-point number or the 32-bit tensor floating-point number, fill the last 7 exponent bits of the intermediate data format into the last 7 exponent bits of the 32-bit floating-point number or the 32-bit tensor floating-point number, and fill the first exponent bit of the intermediate data format into the first exponent bit of the 32-bit floating-point number or the 32-bit tensor floating-point number; use the last 23 bits of the mantissa of the intermediate data format as the mantissa of the 32-bit floating-point number, or use the last 10 bits of the mantissa of the intermediate data format as the mantissa of the 32-bit tensor floating-point number. If the target data format is a 16-bit floating-point number, the sign bit of the intermediate data format is filled into the sign bit of the 16-bit floating-point number, the last 4 exponent bits of the intermediate data format are filled into the last 4 exponent bits of the 16-bit floating-point number, and the first exponent bit of the intermediate data format is filled into the first exponent bit of the 16-bit floating-point number; the last 10 bits of the mantissa of the intermediate data format are used as the mantissa of the 16-bit floating-point number. If the target data format is a 16-bit floating-point number, the sign bit of the intermediate data format is filled into the sign bit of the 16-bit floating-point number, the last 7 exponent bits of the intermediate data format are filled into the last 7 exponent bits of the 16-bit floating-point number, and the first exponent bit of the intermediate data format is filled into the first exponent bit of the 16-bit floating-point number; the last 7 bits of the mantissa of the intermediate data format are used as the mantissa of the 16-bit floating-point number. If the target data format is a 32-bit signed integer, a 16-bit signed integer, or an 8-bit signed integer, the integer part of the intermediate data format and the mantissa parts after removing the first 1, 17, or 25 bits are concatenated to form the 32-bit signed integer, 16-bit signed integer, or 8-bit signed integer, respectively. If the target data format is a 32-bit unsigned integer, extract the integer part and mantissa part of the intermediate data format, remove the first bit, and use it as the 32-bit unsigned integer; If the target data format is a 16-bit unsigned integer, the integer part after removing the first bit and the mantissa part after removing the first 16 bits of the intermediate data format are concatenated to form the 16-bit unsigned integer. If the target data format is an 8-bit unsigned integer, the integer part after removing the first bit and the mantissa part after removing the first 24 bits of the intermediate data format are concatenated to form the 8-bit unsigned integer.

13. A processing unit, comprising: As an instruction execution unit of any one of the arithmetic units according to claims 1-12; A register, wherein data in the source data format is read from the register by the arithmetic unit, and data in the target data format is written to the register by the arithmetic unit.

14. An acceleration unit, comprising: As a tensor engine of any one of the computational units according to claims 1-12; On-chip memory, wherein data in the source data format is read from the on-chip memory by the tensor engine, and data in the target data format is written to the on-chip memory by the tensor engine.

15. A computing device comprising a processing unit according to claim 13 or an acceleration unit according to claim 14.

16. A system-on-a-chip, comprising a processing unit according to claim 13 or an acceleration unit according to claim 14.

17. A data center comprising the computing device according to claim 15.

Images