Floating-point number processing method, apparatus, device, storage medium, and program product

Abstract

Embodiments of the present disclosure disclose a floating point number processing method, device, equipment, storage medium and program product. The floating point number processing method comprises: obtaining a first floating point number and a second floating point number to be processed currently; the data format of the first floating point number and the second floating point number is a first data format; performing format conversion on the first floating point number and the second floating point number respectively based on a loop instruction, and determining a first operation result after the first floating point number and the second floating point number are multiplied based on the converted first floating point number and the converted second floating point number; the loop instruction is used to indicate the processing mode of the format conversion performed by a tensor calculation engine and the loop times; the converted first floating point number and the converted second floating point number are a second data format, and the second data format is a simplification of the first data format. In this way, the structure of the tensor calculation engine does not need to be changed, and only a small amount of hardware resources are needed, and a performance improvement of 2.66 times can be obtained.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to, but is not limited to, the technical field of computer technology, and particularly relates to a floating point number processing method and device, equipment, a storage medium and a program product. BACKGROUND

[0002] At present, most artificial intelligence (AI) models are trained in the data format of single-precision floating point (FP32). However, the FP32 has a large number of mantissas, which makes it face many challenges in implementing floating point operations at the hardware level.

[0003] Some chips specially designed for artificial intelligence applications integrate efficient matrix calculation units. In these chips, if the FP32 floating point operation is to be implemented by hardware, a large number of calculation logics need to be introduced, which greatly changes the existing matrix calculation units, and the running efficiency is low in the actual running process of the chip. SUMMARY

[0004] Therefore, the embodiments of the present disclosure provide at least a floating point number processing method, device, equipment, storage medium and program product.

[0005] The technical solutions of the embodiments of the present disclosure are implemented as follows:

[0006] In one aspect, the embodiments of the present disclosure provide a floating point number processing method applied to a tensor calculation engine, and the floating point number processing method comprises the following steps.

[0007] Obtaining a first floating point number and a second floating point number to be processed currently, wherein the data format of the first floating point number and the second floating point number is a first data format;

[0008] Converting the first floating point number and the second floating point number based on a loop instruction, and determining a first operation result after the first floating point number is multiplied by the second floating point number based on the converted first floating point number and the converted second floating point number; the loop instruction is used to indicate the processing mode of the tensor calculation engine for format conversion and the number of loops; the converted first floating point number and the converted second floating point number are in a second data format, and the second data format is a simplification of the first data format.

[0009] In another aspect, the embodiments of the present disclosure provide a tensor calculation engine, which comprises the following.

[0010] a format conversion unit, configured to perform format conversion on the first floating-point number and the second floating-point number respectively based on a loop instruction; the loop instruction is used to indicate a processing manner of performing format conversion by the tensor calculation engine and a loop number; the data format of the first floating-point number and the second floating-point number is a first data format;

[0011] a dot product calculation unit, configured to determine a first operation result of multiplication of the first floating-point number and the second floating-point number based on the converted first floating-point number and the converted second floating-point number; the converted first floating-point number and the converted second floating-point number are in a second data format, and the second data format is a simplification of the first data format.

[0012] In the embodiments of the present disclosure, since the loop instruction is used to indicate a processing manner of performing format conversion by the tensor calculation engine and a loop number, the number of loops required for floating-point number multiplication and the processing manner used for format conversion of the first floating-point number and the second floating-point number in each loop can be quickly determined based on the loop instruction. Since the data format of the first floating-point number and the second floating-point number is the first data format, and the converted first floating-point number and the converted second floating-point number are in the second data format, the second data format is a simplification of the first data format, the format conversion of the first floating-point number and the second floating-point number based on the loop instruction can obtain simplified first floating-point numbers (converted first floating-point numbers) and simplified second floating-point numbers (converted second floating-point numbers) corresponding to different loops, and thus the first operation result of multiplication of the first floating-point number and the second floating-point number can be determined based on the converted first floating-point number and the converted second floating-point number. In this way, only the processing manner of format conversion needs to be added in the tensor calculation engine, and different processing manners can be called in different loops by the loop instruction to perform format conversion (simplification of data format) of the first floating-point number and the second floating-point number, and the multiplication result of the first floating-point number and the second floating-point number can be determined based on the converted first floating-point number and the converted second floating-point number, so that the floating-point number multiplication with high precision can be realized, the structure of the tensor calculation engine does not need to be changed, only a small amount of hardware resources are required, the performance can be improved by 2.66 times, the precision is higher than that of the scheme of directly truncating the FP32 mantissa, the complexity of software development is reduced compared with the software implementation scheme, and the demand for storage resources is reduced.

[0013] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, but not limiting the technical solutions of the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0014] The accompanying drawings, which are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the technical solutions of the present disclosure.

[0015] Figure 1 A schematic diagram of a floating-point number data format provided for an embodiment of the present disclosure;

[0016] Figure 2 An implementation flowchart of a floating-point number processing method provided for an embodiment of the present disclosure Figure 1 ;

[0017] Figure 3 An implementation flowchart of a floating-point number processing method provided for an embodiment of the present disclosure

[0018] Figure 4 An implementation flowchart of a floating-point number processing method provided for an embodiment of the present disclosure Figure 2 ;

[0019] Figure 5 An implementation flowchart of a floating-point number processing method provided for an embodiment of the present disclosure Figure 3 ;

[0020] Figure 6 An implementation flowchart of a floating-point number processing method provided for an embodiment of the present disclosure

[0021] Figure 7 An implementation flowchart of a floating-point number processing method provided for an embodiment of the present disclosure

[0022] Figure 8 An implementation flowchart of a floating-point number processing method provided for an embodiment of the present disclosure DETAILED DESCRIPTION

[0023] In order to make the purposes, technical solutions and advantages of the present disclosure clearer, the technical solutions of the present disclosure are further described in detail below in combination with the drawings and embodiments, and the described embodiments should not be regarded as limiting the present disclosure, and all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the scope of protection of the present disclosure.

[0024] In the following description, “some embodiments” are related to a subset of all possible embodiments, but it can be understood that “some embodiments” can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0025] The terms “first / second / third” involved only distinguish similar objects, and do not represent a specific order of the objects, and it can be understood that “first / second / third” can interchange specific order or sequence as allowed, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used in the description herein is for describing particular embodiments only and is not intended to be limiting of the disclosure.

[0027] Before the embodiments of the present disclosure are further described, the terms and names involved in the embodiments of the present disclosure are explained, which are applicable to the following explanations.

[0028] As shown in Figure 1 Half-Precision Floating Point (FP16) is a 16-bit floating-point format that follows the IEEE754 standard. The design purpose of FP16 is to provide lower precision than Single-Precision Floating Point (FP32) while reducing memory footprint and improving computing speed. The data format of FP16 includes 1 sign bit, 5 exponent bits, and 10 mantissa bits.

[0029] Brain Floating Point 16 (BF16) is a 16-bit floating-point format that has the same numerical range as FP32, but its precision is slightly lower than FP32. BF16 is particularly suitable for the field of deep learning, because deep learning models are usually more sensitive to numerical range, while the requirement for precision can be relatively low. The data format of BF16 includes 1 sign bit, 8 exponent bits, and 7 mantissa bits.

[0030] Tensor Float 32 (TF32) is a computing format for TensorCore introduced on GPUs with Ampere architecture. In some tasks, TF32 can have up to 10 times performance improvement compared to FP32. TF32 is a truncated Float32 data format, which truncates the 23 mantissa bits in FP32 to 10 bits, while the exponent bits remain 8 bits, and the total length is 19 bits (including 1 sign bit). In this way, TF32 maintains the same precision as FP16 (both have 10 mantissa bits), while also maintaining the dynamic range of FP32 (both have 8 exponent bits).

[0031] FP32 is a widely used floating-point representation standard in computer science, which follows the IEEE754 standard. The data format of FP32 includes 1 sign bit, 8 exponent bits, and 23 mantissa bits.

[0032] In order to better understand the floating-point processing method provided by the embodiments of the present disclosure, the following will first explain the scheme in the related art.

[0033] Under the constraints of chip area and power consumption, when a neural network processing unit (NPU) and a graphics processing unit (GPU) integrated on a chip are used to process an artificial intelligence model, in order to support higher computing power, most NPUs and GPUs usually use matrix calculation units with lower precision than FP32, such as FP16, BF16, INT8, and FP8 data formats.

[0034] If an AI model is run in the FP16 format, since FP16 and FP32 have certain similarity in data format, in most cases, data format conversion can be directly performed; or if a lower precision INT8 format is selected, the model needs to be quantized, and quantization mainly includes two ways: post-training quantization (PTQ) and quantization-aware training (QAT).

[0035] Whether data format conversion is directly performed or a quantization method is used, these operations increase additional complexity and workload.

[0036] TF32 and FP16 use the same 10 bits (bit) mantissa, and have the same 8 bits exponent as FP32, and in many scenarios, TF32 can replace FP32. One way is that TF32 is still stored in the form of FP32, and the tail 13 bits are truncated during calculation. This truncation method cannot meet the requirements of some high-precision scenarios. Another way is to convert FP32 to TF32 by using a software solution. However, converting FP32 at the software level not only needs complex instruction scheduling, but also needs additional space to store the converted data. At the same time, format conversion during runtime also brings additional burden, so that the final performance improvement is difficult to reach the theoretical value. If the hardware directly implements the FP32 matrix calculation unit, since the mantissa of FP32 is more, more calculation logic needs to be introduced, and the existing matrix calculation unit needs to be changed greatly.

[0037] Therefore, an embodiment of the present disclosure provides a floating-point number processing method applied to a tensor calculation engine. As shown in the method includes the following steps 201 to 202: Figure 2

[0038] Step 201, obtaining a first floating-point number and a second floating-point number to be processed; the data format of the first floating-point number and the second floating-point number is a first data format.

[0039] ​The first data format can be an FP32 data format. The first floating point number and the second floating point number are at least two FP32 floating point numbers to be processed currently. A tensor compute engine (TCE) is used to perform multiplication operation of the floating point numbers.

[0040] In some embodiments, a specific implementation of step 201 can be: based on the metadata information of the tensor and the information of the data block, reading the first floating point number and the second floating point number from the local memory.

[0041] Step 202, based on a loop instruction, respectively converting the first floating point number and the second floating point number, and based on the converted first floating point number and the converted second floating point number, determining a first operation result after multiplying the first floating point number and the second floating point number; the loop instruction is used to indicate a processing mode of the tensor compute engine to perform format conversion and a loop number; the converted first floating point number and the converted second floating point number are a second data format, and the second data format is a simplification of the first data format.

[0042] The second data format is a simplification of the first data format, which can mean that the second data format has less mantissa bits than the first data format. The second data format can be a TF32 data format. The loop instruction is used to indicate how many times the current floating point number needs to be processed and whether the first floating point number and the second floating point number are converted into a first type of TF32 floating point number or a second type of TF32 floating point number in each loop. The first type can include high-bit truncated data, which is obtained by high-bit truncation of the first data format, for example, big TF32. The second type can include low-bit rounding data, which can be obtained by rounding operation on the remaining low-bit part through a rounding algorithm (i.e., rounding to the nearest even number) after obtaining the first type of data, for example, small TF32.

[0043] Exemplarily, the conversion of an FP32 floating-point number into two TF32 floating-point numbers can be expanded by using the following multiplication formula: A * B = (big_TF32_A + small_TF32_A) * (big_TF32_B + small_TF32_B) = big_TF32_A * big_TF32_B + big_TF32_A * small_TF32_B + small_TF32_A * big_TF32_B + small_TF32_A * small_TF32_B. Wherein, A represents a first floating-point number, B represents a second floating-point number. big_TF32_A represents a first sub-floating-point number (i.e., the first floating-point number of the first type after conversion), small_TF32_A represents a fourth sub-floating-point number (i.e., the first floating-point number of the second type after conversion); big_TF32_B represents a second sub-floating-point number (i.e., the second floating-point number of the first type after conversion), small_TF32_B represents a third sub-floating-point number (i.e., the second floating-point number of the second type after conversion).

[0044] The processing mode of format conversion includes: a processing mode of converting an FP32 floating-point number into high-bit-truncated data (e.g., a big_TF32 floating-point number), and a processing mode of converting an FP32 floating-point number into low-bit-rounded data (e.g., a small_TF32_A floating-point number).

[0045] Exemplarily, the processing mode of converting an FP32 floating-point number into a big_TF32 floating-point number can be represented as: big_TF32 = FP32 & 0xffffe000; and the processing mode of converting an FP32 floating-point number into a small_TF32_A floating-point number can be represented as: small_TF32 = RNE((FP32 & 0x1fff) + 0x1000). Wherein, 0xffffe000 is a hexadecimal mask used to preserve specific bits during conversion. 0x1fff represents a specific exponent value used to adjust the exponent part during conversion to ensure that the exponent part maintains the correct range and offset during conversion. 0x1000 represents the offset of the mantissa in the TF32 format, used to adjust the position of the mantissa during conversion to adapt to the format requirements of TF32.

[0046] In some embodiments, the specific implementation of step 202 can be: based on the loop indication, in different loops, respectively format-convert the first floating-point number and the second floating-point number, and in each loop, multiply the converted first floating-point number and the converted second floating-point number, and after the loop ends, add the multiplication results of multiple loops to obtain the first operation result.

[0047] In some embodiments, the implementation of step 202 can also be: first, format conversion is performed on the first floating-point number and the second floating-point number respectively, then in different loops, different converted first floating-point numbers and different converted second floating-point numbers are directly called, and in each loop, multiplication operation is performed on the converted first floating-point number and the converted second floating-point number, and after the loop ends, the multiplication operation results of multiple loops are added to obtain the first operation result. The combination of the converted first floating-point number and the converted second floating-point number corresponding to each loop is different.

[0048] In some embodiments, the converted first floating-point number includes a first sub-floating-point number (for example, big_TF32_A) and a fourth sub-floating-point number (for example, small_TF32_A); and the converted second floating-point number includes a second sub-floating-point number (for example, big_TF32_B) and a third sub-floating-point number (for example, small_TF32_B). The first operation result refers to the result of multiplication of the first floating-point number and the second floating-point number. At this time, the implementation of step 202 can be: based on the loop indication, converting the first floating-point number A into big_TF32_A and small_TF32_A, and converting the second floating-point number B into big_TF32_B and small_TF32_B, and based on the loop indication, sequentially determining the result of multiplication of big_TF32_A and big_TF32_B, the result of multiplication of big_TF32_A and small_TF32_B, the result of multiplication of small_TF32_A and big_TF32_B, and the result of multiplication of small_TF32_A and small_TF32_B; and then adding the results of the four multiplications to obtain the first operation result of multiplication of the first floating-point number and the second floating-point number.

[0049] In some embodiments, the specific implementation of step 202 can also be: first converting the first floating-point number A into big_TF32_A and small_TF32_A, converting the second floating-point number B into big_TF32_B and small_TF32_B, then calling big_TF32_A and big_TF32_B, big_TF32_A and small_TF32_B, small_TF32_A and big_TF32_B, and small_TF32_A and small_TF32_B in turn based on the loop indication, and determining the results of multiplying big_TF32_A and big_TF32_B, multiplying big_TF32_A and small_TF32_B, multiplying small_TF32_A and big_TF32_B, and multiplying small_TF32_A and small_TF32_B in turn. In this process, after obtaining the result of this multiplication, the result of this multiplication is added to the result of the multiplication before this time to obtain the first operation result of multiplying the first floating-point number and the second floating-point number.

[0050] In the embodiments of the present disclosure, since the loop indication is used to indicate the processing manner of format conversion performed by the tensor calculation engine and the number of loops; therefore, based on the loop indication, it can be quickly determined how many times of loops are needed for floating-point number multiplication operation and which processing manner is used to respectively perform format conversion on the first floating-point number and the second floating-point number in each loop. Since the data format of the first floating-point number and the second floating-point number is the first data format, the converted first floating-point number and the converted second floating-point number are the second data format, and the second data format is a simplification of the first data format; therefore, based on the loop indication, the format conversion is respectively performed on the first floating-point number and the second floating-point number, which can simplify the data format of the first floating-point number and the second floating-point number in different loops, and thus, based on the converted first floating-point number and the converted second floating-point number, the first operation result of multiplying the first floating-point number and the second floating-point number is determined. In this way, only the processing manner of format conversion needs to be added in the tensor calculation engine, and different processing manners are called in different loops by the loop indication to respectively perform format conversion (simplification of data format) on the first floating-point number and the second floating-point number, and based on the converted first floating-point number and the converted second floating-point number, the multiplication result of the first floating-point number and the second floating-point number is determined, the floating-point number multiplication operation with high precision is realized, the structure of the tensor calculation engine does not need to be changed, only a small amount of hardware resources are needed, the performance is improved by 2.66 times, the precision is higher than that of the scheme of directly truncating the FP32 mantissa, the complexity of software development is reduced compared with the software implementation scheme, and the demand for storage resources is also reduced.

[0051] In some embodiments, step 201 can be implemented by step 2011 to step 2022 as follows:

[0052] Step 2011, determining a first number of floating-point multiplication operations that can be processed by the tensor computing engine at one time.

[0053] The first number refers to the number of floating-point multiplication operations that can be processed by the tensor computing engine at one time.

[0054] In some embodiments, the specific implementation of step 2011 can be: determining the first number based on the structure of the dot product calculation unit in the tensor computing engine.

[0055] In some embodiments, the dot product calculation unit can include 8*8 dot product units (Dot-Product, DP), and the first number can be 8*8.

[0056] Step 2012, reading the first number of floating-point numbers from a first address of the local memory as the first floating-point numbers, and reading the first number of floating-point numbers from a second address of the local memory as the second floating-point numbers.

[0057] The data format of the first floating-point numbers and the second floating-point numbers is a first data format.

[0058] The first address refers to the address of the first floating-point numbers in the local memory, and the second address refers to the address of the second floating-point numbers in the local memory.

[0059] In some embodiments, the first number can be 8*8, and at this time, 8*8 floating-point numbers can be read from the first address of the local memory as the first floating-point numbers, and 8*8 floating-point numbers can be read from the second address of the local memory as the second floating-point numbers.

[0060] In the embodiments of the present disclosure, the first floating-point numbers and the second floating-point numbers are read according to the first number of floating-point multiplication operations that can be processed by the tensor computing engine at one time, which can process multiplication operations of the first number of floating-point numbers at the same time, and improve the utilization rate of the tensor computing engine.

[0061] In some embodiments, the above step 202 can be implemented by step 2021 to step 2023 as follows:

[0062] Step 2021, determining a number of cycles based on the cycle indication; the number of cycles is used to indicate the processing mode of the corresponding format conversion.

[0063] In some embodiments, the cycle indication can be controlled by the instruction of the tensor computing engine, and the cycle indication can be 3 cycles or 4 cycles.

[0064] Step 2022, based on the number of cycles, in different cycles, different processing modes of format conversion are respectively used to extract different parts of the first floating-point number and the second floating-point number, to obtain at least two different simplified first floating-point numbers and at least two different simplified second floating-point numbers; the converted first floating-point number includes the simplified first floating-point number, and the converted second floating-point number includes the simplified second floating-point number.

[0065] The different processing modes of format conversion include: format conversion based on a first function and format conversion based on a second function. The first function can include a high-bit truncation format conversion function, which realizes format conversion by high-bit truncation on data in a first data format. Exemplarily, the first function can be a big_TF32 format conversion function, i.e. big_TF32=FP32&0xffffe000. The second function can include a low-bit rounding format conversion function, which realizes format conversion by rounding operation on the low-bit part of data in the first data format. Exemplarily, the second function can be a small_TF32 format conversion function, i.e. small_TF32=RNE((FP32&0x1fff)+0x1000).

[0066] In some embodiments, if the number of cycles is 4, in the first cycle, the first function is used to respectively convert the format of the first floating-point number and the second floating-point number; in the second cycle, the first function is used to convert the format of the first floating-point number, and the second function is used to convert the format of the second floating-point number; in the third cycle, the second function is used to convert the format of the first floating-point number, and the first function is used to convert the format of the second floating-point number; in the fourth cycle, the second function is used to respectively convert the format of the first floating-point number and the second floating-point number; thereby obtaining the simplified first floating-point number and the simplified second floating-point number corresponding to each cycle.

[0067] In some embodiments, if the number of cycles is 3, in the first cycle, the first function is used to respectively convert the format of the first floating-point number and the second floating-point number; in the second cycle, the first function is used to convert the format of the first floating-point number, and the second function is used to convert the format of the second floating-point number; in the third cycle, the second function is used to convert the format of the first floating-point number, and the first function is used to convert the format of the second floating-point number; thereby obtaining the simplified first floating-point number and the simplified second floating-point number corresponding to each cycle.

[0068] Step 2023, determining the first operation result based on the simplified first floating-point number and the simplified second floating-point number.

[0069] In some embodiments, the implementation of step 2023 can be: performing operation processing on the simplified first floating-point number and the simplified second floating-point number to obtain a first operation result.

[0070] Based on the above technical solution, the number of cycles of floating-point number processing can be determined based on the cycle indication, and the format conversion manner corresponding to each cycle can be determined based on the number of cycles. Then, in different cycles, different format conversion processing manners are used to extract different parts of the first floating-point number and the second floating-point number, to obtain the simplified first floating-point number and the simplified second floating-point number corresponding to each cycle. Thus, by performing operation processing on the simplified first floating-point number and the simplified second floating-point number, a first operation result is obtained. In this way, only by using the cycle indication, different processing manners can be called in different cycles to perform format conversion on the first floating-point number and the second floating-point number, respectively. This way of converting while looping does not require additional space to store the converted floating-point numbers, thereby reducing the demand for storage resources.

[0071] In some embodiments, the above step 2022 can be implemented by the following steps 2022a to 2022d:

[0072] Step 2022a, in the first cycle, the first function is used to perform format conversion on the first floating-point number and the second floating-point number, respectively, to obtain a first sub-floating-point number and a second sub-floating-point number.

[0073] The first sub-floating-point number refers to the floating-point number obtained by format conversion of the first floating-point number using the first function, i.e., the simplified first floating-point number based on the first function. The second sub-floating-point number refers to the floating-point number obtained by format conversion of the second floating-point number using the first function, i.e., the simplified second floating-point number based on the first function. The number of cycles can be represented by loop, and the first cycle can be represented by loop id 0.

[0074] In some embodiments, the first function can refer to the format conversion function of big_TF32, i.e., big_TF32=FP32&0xffffe000. At this time, the first sub-floating-point number can be represented as big_TF32_A, and the second sub-floating-point number can be represented as big_TF32_B.

[0075] Step 2022b, in the second cycle, the first function is used to perform format conversion on the first floating-point number to obtain a first sub-floating-point number, and a second function is used to perform format conversion on the second floating-point number to obtain a third sub-floating-point number.

[0076] The third sub-float number refers to a float number obtained by format conversion of the second float number using the second function, i.e., a simplified second float number based on the second function. The number of loops can be represented by loop, and the second loop can be represented by loop id 1.

[0077] In some embodiments, the second function can refer to a format conversion function of small_TF32, i.e., small_TF32=RNE((FP32&0x1fff)+0x1000), and the third sub-float number can be represented as small_TF32_B.

[0078] Step 2022c, the third loop, format conversion of the first float number using the second function to obtain the fourth sub-float number, and format conversion of the second float number using the first function to obtain the second sub-float number.

[0079] The number of loops can be represented by loop, and the third loop can be represented by loop id 2. The fourth sub-float number refers to a float number obtained by format conversion of the first float number using the second function, i.e., a simplified first float number based on the second function.

[0080] In some embodiments, the first function can refer to a format conversion function of big_TF32, i.e., big_TF32=FP32&0xffffe000, and the fourth sub-float number can be represented as small_TF32_A.

[0081] Step 2022d, the fourth loop, format conversion of the first float number and the second float number using the second function respectively to obtain the fourth sub-float number and the third sub-float number; the simplified first float number includes the first sub-float number and the fourth sub-float number, and the simplified second float number includes the second sub-float number and the third sub-float number.

[0082] The number of loops can be represented by loop, and the fourth loop can be represented by loop id 3.

[0083] Based on the above technical solution, in the first cycle, the first function is used to respectively convert the format of the first floating-point number and the second floating-point number to obtain the first sub-floating-point number and the second sub-floating-point number; in the second cycle, the first function is used to convert the format of the first floating-point number to obtain the first sub-floating-point number, and the second function is used to convert the format of the second floating-point number to obtain the third sub-floating-point number; in the third cycle, the second function is used to convert the format of the first floating-point number to obtain the fourth sub-floating-point number, and the first function is used to convert the format of the second floating-point number to obtain the second sub-floating-point number; in the fourth cycle, the second function is used to respectively convert the format of the first floating-point number and the second floating-point number to obtain the fourth sub-floating-point number and the third sub-floating-point number; in this way, according to the number of cycles, different processing modes of format conversion can be used to extract different parts of the first floating-point number and the second floating-point number, to obtain the simplified first floating-point number and the simplified second floating-point number corresponding to each cycle, and the structure of the tensor calculation engine does not need to be changed, so that the floating-point number can be converted while the cycle is performed, and the performance is doubled.

[0084] In some embodiments, the above step 2023 can be implemented by the following steps 2023a to 2023b:

[0085] Step 2023a, performing multiplication operation on the simplified first floating-point number and the simplified second floating-point number in each cycle to obtain a second number of second operation results.

[0086] Since the number of floating-point number multiplication operations that can be processed by the tensor calculation engine at a time is the first number, that is, the number of first floating-point numbers and the number of second floating-point numbers obtained at a time is the first number, the first number of simplified first floating-point numbers and the first number of simplified second floating-point numbers are required in each cycle, and then the number of second operation results after multiplication operation is 1 / 2 of the first number, that is, the second number is 1 / 2 of the first number. For example, if the first number is 8, the second number is 4.

[0087] Step 2023b, performing addition operation on the second number of second operation results to obtain the first operation result.

[0088] Based on the above technical solution, by performing multiplication operation on the simplified first floating-point number and the simplified second floating-point number in each cycle, and performing addition operation on the second number of second operation results after multiplication operation, the multiplication operation result of the first floating-point number and the second floating-point number can be obtained.

[0089] In some embodiments, the above step 202 can also be implemented by the following steps 2024 to 2027:

[0090] Step 2024, extracting different parts of the first floating-point number and the second floating-point number by using different format conversion processing modes to obtain at least two different simplified first floating-point numbers and at least two different simplified second floating-point numbers; the converted first floating-point number includes the simplified first floating-point number, and the converted second floating-point number includes the simplified second floating-point number.

[0091] Step 2025, based on the number of cycles, respectively calling different simplified first floating-point numbers and different simplified second floating-point numbers in different cycles; the number of cycles is determined based on the cycle indication.

[0092] In some embodiments, the specific implementation of step 2025 can be: in the first cycle, calling the first sub-floating-point number and the second sub-floating-point number; in the second cycle, calling the first sub-floating-point number and the third sub-floating-point number; in the third cycle, calling the fourth sub-floating-point number and the second sub-floating-point number; in the fourth cycle, calling the fourth sub-floating-point number and the third sub-floating-point number; the simplified first floating-point number includes the first sub-floating-point number and the fourth sub-floating-point number, and the simplified second floating-point number includes the second sub-floating-point number and the third sub-floating-point number.

[0093] Step 2026, performing multiplication operation on the simplified first floating-point number and the simplified second floating-point number in each cycle to obtain a second number of second operation results.

[0094] Step 2027, performing addition operation on the second number of second operation results to obtain the first operation result.

[0095] Based on the above technical solutions, before the cycle, different parts of the first floating-point number and the second floating-point number are extracted by using different format conversion processing modes, and then different simplified first floating-point numbers and different simplified second floating-point numbers are respectively called in different cycles based on the number of cycles, and then multiplication operation is performed on the simplified first floating-point number and the simplified second floating-point number in each cycle, and addition operation is performed on the second number of second operation results after the multiplication operation, and the multiplication operation result of the first floating-point number and the second floating-point number can also be obtained. In this way, the conversion and then the calling mode can reduce the calculation complexity of each cycle and improve the processing efficiency of the cycle.

[0096] Based on the foregoing embodiments, the floating-point number processing method provided by the embodiments of the present disclosure further includes steps 203 to 204:

[0097] Step 203, in response to a configuration instruction, configuring metadata information of a tensor, and block information of the tensor; each block of the tensor includes a plurality of floating-point numbers in a first data format.

[0098] The tensor is stored in a continuous memory block in the memory, and a single element in the tensor can be a single floating point number. The size of the tensor is directly related to the required memory, and the number of bits of the floating point number (such as 32 bits or 16 bits) affects the storage efficiency of the tensor. The metadata information of the tensor can include but is not limited to: tensor size (dimension), data type, storage offset, step, content attribute and the like.

[0099] In some embodiments, the specific implementation of "configuring the metadata information of the tensor" in step 203 can be: using an application programming interface (API) function to configure the metadata information (TensorInfo) of the tensor.

[0100] In some embodiments, the specific implementation of "configuring the tile information of the tensor" in step 203 can be: defining a bit field in the instruction of the tensor storage engine to represent the information of the data tile. The information of the data tile can include but is not limited to: the coordinates and size of the data tile.

[0101] Step 204: based on the metadata information and the tile information, transferring the data in any tile of the tensor from the global memory to the local memory.

[0102] The global memory refers to a memory region that can be shared among all threads. The local memory can be a memory region in a computer device that is located near the processor, has high speed and small capacity. The design purpose of the local memory is to reduce the delay of the processor accessing the main memory and improve the data access speed. The local memory can be part of the cache or a dedicated high-speed storage area, such as a register or a certain type of cache memory.

[0103] In some embodiments, in order to avoid bank conflict when the tensor computing engine reads the floating point number from the local memory, the global memory and the local memory store the floating point number in a target storage manner; the target storage manner can uniformly distribute multiple floating point numbers for each calculation in different banks.

[0104] In a feasible implementation, the target storage manner can be a swizzle manner. Swizzle is a technology for optimizing memory access by changing the storage manner of data in the memory, which involves address remapping of data before writing to the memory or after reading from the memory.

[0105] As Figure 3As shown, the origin data line includes 16 banks, and the numbers in each bank represent the data stored in each bank; each bank is 16 bytes. Since the same column of different cache lines cannot be read and written at the same time, only different columns can be read and written. Therefore, when calculating, the data in a column cannot be read and written at one time. Based on this, the embodiment of the disclosure adopts the swizzle method to place 0 in different columns of different rows, 1 in different columns of different rows, and 15 in different columns of different rows, so that bank conflict does not occur, and reading and writing can be performed through one clock cycle. A clock cycle refers to the shortest time unit required for a GPU to perform a single operation.

[0106] Based on the above technical solution, by configuring the metadata information and the block information of the tensor, the data in any block of the tensor can be transmitted from the global memory to the local memory based on the metadata information and the block information, facilitating subsequent operations on the floating-point numbers in any block.

[0107] Based on the above embodiment, the floating-point number processing method provided by the embodiment of the disclosure can further include steps 205 to 206:

[0108] Step 205, based on the bit width of the first floating-point number, the data reading delay, and the preset first storage bit, determining a first storage area from the idle memory of the tensor computing engine.

[0109] The bit width of the first floating-point number is 32 bits. The data reading delay refers to the time required to read the first floating-point number. The first storage bit refers to the bit width required during floating-point number operation; for example, not only the floating-point number itself needs to be stored, but also the sign bit, the exponent bit, the mantissa bit, the extension bit, and the sticky bit, etc. need to be stored, which can be set according to business requirements.

[0110] Step 206, based on the bit width of the second floating-point number, the data reading delay, and the preset second storage bit, determining a second storage area from the idle memory of the tensor computing engine.

[0111] The bit width of the second floating-point number is 32 bits. The data reading delay refers to the time required to read the second floating-point number. The second storage bit refers to the bit width required during floating-point number operation; the first storage bit and the second storage bit can be the same or different.

[0112] In some embodiments, after reading the first floating point number and the second floating point number, the first floating point number can be stored in the first storage area, and the second floating point number can be stored in the second storage area. It should be noted that in the prior art, when the FP32 floating point number is calculated at the software level, since the specific space required is unknown, a larger storage space is allocated when the memory is allocated. In the embodiment of the present disclosure, the original FP32 floating point number is stored in the first storage area and the second storage area. Since only the FP32 floating point number to be processed currently read from the local memory needs to be stored, only a storage space matching the FP32 floating point number needs to be allocated. Compared with the prior art, the storage space allocated is smaller, and the storage space is saved to the maximum extent.

[0113] In some embodiments, when the floating point number operation is performed in the manner of conversion first and then retrieval, the converted first floating point number (simplified first floating point number) can be stored in the first storage area, and the converted second floating point number (simplified second floating point number) can be stored in the second storage area, so as to be retrieved at any time.

[0114] Based on the above scheme, based on the bit width of the first floating point number, the data reading delay, and the preset first storage bit, the first storage area is determined from the idle memory of the tensor calculation engine, and based on the bit width of the second floating point number, the data reading delay, and the preset second storage bit, the second storage area is determined from the idle memory of the tensor calculation engine. The first floating point number and the second floating point number can be retrieved at any time, or the simplified first floating point number and the simplified second floating point number can be retrieved at any time, so as to further improve the processing efficiency of the floating point number operation.

[0115] It should be noted that each DP in the dot product calculation unit (DOT8 calculation unit) can process 8 floating point multiplication operations at a time. For the sake of clarity, the multiplication operation of a single first floating point number and a single second floating point number is taken as an example for description.

[0116] An embodiment of the present disclosure provides a floating point number processing method, as shown in Figure 4 The method comprises the following steps 401 to 406:

[0117] Step 401, determining a first number of floating point multiplication operations that can be processed by a tensor calculation engine at a time.

[0118] Step 402, reading the first number of floating point numbers from a first address of a local memory as first floating point numbers, and reading the first number of floating point numbers from a second address of the local memory as second floating point numbers.

[0119] Step 403, based on a loop indication, determining a loop number; the loop number is used to indicate a processing manner of corresponding format conversion.

[0120] Step 404, based on the number of cycles, in different cycles, different format conversion processing modes are respectively adopted to extract different parts of the first floating point number and the second floating point number, to obtain at least two different simplified first floating point numbers and at least two different simplified second floating point numbers.

[0121] The converted first floating point number includes the simplified first floating point number, and the converted second floating point number includes the simplified second floating point number.

[0122] In some embodiments, the specific implementation of step 404 can be: in the first cycle, the first function is adopted to respectively convert the format of the first floating point number and the second floating point number to obtain the first sub-floating point number and the second sub-floating point number; in the second cycle, the first function is adopted to convert the format of the first floating point number to obtain the first sub-floating point number, and the second function is adopted to convert the format of the second floating point number to obtain the third sub-floating point number; in the third cycle, the second function is adopted to convert the format of the first floating point number to obtain the fourth sub-floating point number, and the first function is adopted to convert the format of the second floating point number to obtain the second sub-floating point number; in the fourth cycle, the second function is adopted to respectively convert the format of the first floating point number and the second floating point number to obtain the fourth sub-floating point number and the third sub-floating point number; the simplified first floating point number includes the first sub-floating point number and the fourth sub-floating point number, and the simplified second floating point number includes the second sub-floating point number and the third sub-floating point number.

[0123] Step 405, multiplication operation is performed on the simplified first floating point number and the simplified second floating point number in each cycle to obtain a second number of second operation results.

[0124] Step 406, addition operation is performed on the second number of second operation results to obtain the first operation result.

[0125] The embodiments of the present disclosure provide a floating point number processing method, as shown in the following formula (1): Figure 5 The method includes the following steps 501 to 506:

[0126] Step 501, determining a first number of floating point number multiplication operations that can be processed by a tensor computing engine at a time.

[0127] Step 502, reading the first number of floating point numbers from a first address of a local memory as first floating point numbers, and reading the first number of floating point numbers from a second address of the local memory as second floating point numbers.

[0128] Step 503, adopting different format conversion processing modes to extract different parts of the first floating point number and the second floating point number to obtain at least two different simplified first floating point numbers and at least two different simplified second floating point numbers.

[0129] Step 504, based on the number of cycles, respectively call different simplified first floating-point numbers and different simplified second floating-point numbers in different cycles.

[0130] In some embodiments, the specific implementation of step 504 can be: in the first cycle, call the first sub-floating-point number and the second sub-floating-point number; in the second cycle, call the first sub-floating-point number and the third sub-floating-point number; in the third cycle, call the fourth sub-floating-point number and the second sub-floating-point number; in the fourth cycle, call the fourth sub-floating-point number and the third sub-floating-point number; the simplified first floating-point number includes the first sub-floating-point number and the fourth sub-floating-point number, and the simplified second floating-point number includes the second sub-floating-point number and the third sub-floating-point number.

[0131] Step 505, multiply the simplified first floating-point number and the simplified second floating-point number in each cycle to obtain a second number of second operation results.

[0132] Step 506, add the second number of second operation results to obtain the first operation result.

[0133] The embodiments of the present disclosure provide a tensor computing engine, which comprises: a format conversion unit, configured to respectively convert a first floating-point number and a second floating-point number based on a cycle indication; the cycle indication is used to indicate a processing manner of format conversion performed by the tensor computing engine and a number of cycles; the data format of the first floating-point number and the second floating-point number is a first data format; and a dot product calculation unit, configured to determine a first operation result obtained by multiplying the first floating-point number and the second floating-point number based on the converted first floating-point number and the converted second floating-point number; the converted first floating-point number and the converted second floating-point number are in a second data format, and the second data format is a simplification of the first data format.

[0134] The tensor memory engine (TME) is used to move tensor data from the global memory to the local memory.

[0135] In some embodiments, the dot product calculation unit can be a DOT8 calculation unit. The DOT8 calculation unit can include eight dot product calculation units for performing dot product operations in vector or matrix operations. For example, the eight dot products performed by the DOT8 calculation unit can be represented as: A0*B0 + A1*B1+…+ A7*B7.

[0136] In some embodiments, the format conversion unit for converting the floating-point numbers can be arranged in at least one of the following three positions: 1. The format conversion unit is arranged on the path after reading the first floating-point number and the second floating-point number and before storing the converted first floating-point number and the converted second floating-point number; 2. The format conversion unit is arranged on the path between the first storage area and the second storage area and the DOT8 calculation unit; 3. The format conversion unit is integrated into the DOT8 calculation unit.

[0137] In some embodiments, the format conversion unit is specifically configured to convert the first floating-point number by using a first function to obtain a first sub-floating-point number; convert the second floating-point number by using the first function to obtain a second sub-floating-point number; convert the first floating-point number by using a second function to obtain a fourth sub-floating-point number; and convert the second floating-point number by using the second function to obtain a third sub-floating-point number.

[0138] In some embodiments, the dot product calculation unit is specifically configured to determine a number of cycles based on the cycle indication; the number of cycles is used to indicate a processing mode of corresponding format conversion; based on the number of cycles, in different cycles, different format conversion processing modes are used to extract different parts of the first floating-point number and the second floating-point number to obtain at least two simplified first floating-point numbers and at least two simplified second floating-point numbers; the converted first floating-point number includes the simplified first floating-point number, and the converted second floating-point number includes the simplified second floating-point number; and determine the first operation result based on the simplified first floating-point number and the simplified second floating-point number.

[0139] In some embodiments, the dot product calculation unit is specifically configured to based on the number of cycles, in different cycles, call different simplified first floating-point numbers and different simplified second floating-point numbers; perform multiplication operation on the simplified first floating-point number and the simplified second floating-point number in each cycle to obtain a second number of second operation results; and perform addition operation on the second number of second operation results to obtain the first operation result.

[0140] In some embodiments, the tensor calculation engine further comprises: a first storage area and a second storage area; the first storage area is configured to store the first floating-point number or the converted first floating-point number; and the second storage area is configured to store the second floating-point number or the converted second floating-point number.

[0141] In some embodiments, the dot product calculation unit further comprises: a multiplier configured to multiply the simplified first floating-point number and the simplified second floating-point number in each loop to obtain a second number of second operation results; an exponent alignment unit configured to perform exponent alignment on the second number of second operation results; and an adder configured to add the second number of second operation results after the exponent alignment to obtain the first operation result.

[0142] The application of the floating-point number processing method provided by the embodiments of the present disclosure in an actual scenario is described below.

[0143] The embodiments of the present disclosure adopt a hardware implementation scheme and simple logic control, increase format conversion logic and loop control (loop indication) in a tensor computing engine (TCE), implement the function of implementing FP32 calculation by using TF32 calculation logic, reduce the complexity of software development and the demand of a software implementation scheme on storage resources. The scheme only needs a small amount of hardware resources, can obtain a performance improvement of 2.66 times, and has higher accuracy compared with direct conversion into TF32 for calculation.

[0144] As shown in Figure 6 The tensor memory engine (TME) is responsible for moving tensor data from global memory to local memory. The asynchronous barrier (Async Barrier) synchronization mechanism is used to control the execution order of multiple asynchronous tasks to ensure that they can be executed synchronously at a certain point. The register file (Register File) is a storage device including a group of registers, which can be used to quickly access and store temporary data; for example, used to store local variables and intermediate calculation results. The constant memory (constant memory) is a read-only memory shared by all threads, which can be cached to improve performance and is suitable for storing data that will not change, such as lookup tables or configuration parameters.

[0145] As shown in Figure 7As shown, one implementation is to double the size of the original A buffer and B buffer; in this way, the tensor computing engine stores the big_TF32_A and small_TF32_A converted from the original FP32 floating point A in the A buffer, and stores the big_TF32_B and small_TF32_B converted from the original FP32 floating point B in the B buffer. Then, the tensor computing engine performs four loops, the first loop performs big_TF32_A*big_TF32_B, and the result is stored in the C buffer, and then performs big_TF32_A*small_TF32_B while accumulating the result of the last calculation, and in this way, the remaining small_TF32_A*big_TF32_B and small_TF32_A*small_TF32_B are calculated. At the same time, a control BIT (loop indication) is added in the instruction of the tensor computing engine to control whether small_TF32_A*small_TF32_B is needed. Another implementation can not increase the size of the A / B buffer, and each time the loop is read from the A buffer and the B buffer to perform format conversion.

[0146] The format conversion function is as follows:

[0147] big_TF32 = FP32&0xffffe000;

[0148] small_TF32 = RNE((FP32&0x1fff) + 0x1000).

[0149] The specific implementation scheme of the embodiment of the present disclosure is as follows:

[0150] By configuring the metadata information of the tensor (TensorInfo), such as: configuring the dimensions, strides, and data type of the tensor, and the like. At the same time, the information of the data block is configured in the instruction of the tensor storage engine, such as: configuring the coordinates, size, and the like of the data block. The tensor data is moved from the global memory to the local memory by the tensor storage engine. In order to avoid bankconflict when the tensor computing engine reads data from the local memory, the tensor storage engine and the tensor computing engine adopt the same swizzle mode, such as Figure 3 As shown.

[0151] As shown Figure 7As shown, the DOT8 computation unit in the tensor computation engine has an 8x8 structure. The tensor computation engine reads at least 8x8 Matrix A elements and at least 8x8 Matrix B elements from local memory or registers at a time. The sizes of buffer A and buffer B need to be configured based on the data read latency and the bandwidth of local memory / registers. To hide the data read latency, buffer A and buffer B are usually set relatively large, but this largeness is within a controllable range.

[0152] In the first method, buffer A and buffer B store the converted TP32 data. In this case, the data format conversion unit can be set on the path after reading the original FP32 data and before storing the converted TP32 data.

[0153] In the second approach, buffers A and B still store the original FP32 data, but a data format conversion unit is added between buffers A and B and the DOT8 computation logic.

[0154] The third approach is to integrate the format conversion unit with the DOT8 calculation unit.

[0155] This disclosure uses a third method as an example for illustration. For example... Figure 8 As shown, the DOT8 computation pipeline: Stage 0 conv units are used to convert raw data formats such as FP16, BF16, and TF32 into an internal computation format that uses more mantissa bits than FP32. This embodiment integrates the FP32 to big TF32 and small TF32 format conversion logic with Stage 0. The tensor computation engine can determine whether the output of Stage 0 is big TF32 or small TF32 through peripheral control logic.

[0156] For example, the peripheral control logic adds 3 or 4 loops, the number of which is controlled by instructions from the tensor computation engine. The loop ID (0,1,2,3) is passed to the level 0 conv unit, which then outputs based on the loop ID. Specifically,

[0157] When loop id 0, the conv unit performs format conversion on FP32 floating-point number A and FP32 floating-point number B respectively, and outputs big_TF32_A and big_TF32_B at the same time;

[0158] When loop id 1, the conv unit converts the FP32 floating point number A into big TF32_A and converts the FP32 floating point number B into small TF32_B;

[0159] When loop id 2, the conv unit converts the FP32 floating point number A into small TF32_A and converts the FP32 floating point number B into big TF32_B.

[0160] When loop id 3, the conv unit converts the FP32 floating point number A and the FP32 floating point number B into small TF32_A and small TF32_B simultaneously.

[0161] The calculation result of each loop is taken as the input c of the next calculation. After the loop is completed, a final 8x8 calculation result matrix can be obtained.

[0162] The mul unit is used to perform multiplication operation on the converted two TF floating point numbers. The max.exp unit is used to perform exponential alignment on the operation result of the first stage. The align unit is used to perform addition on the operation result of the second stage.

[0163] The scheme of the embodiments of the present disclosure realizes the matrix calculation of the higher precision FP32 without changing the structure of the existing tensor computing engine (TCE), has higher precision compared with the scheme of directly truncating the FP32 mantissa, reduces the complexity of software development compared with the software implementation scheme, and reduces the demand for storage resources.

[0164] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present disclosure. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily mean the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present disclosure, the size of the serial number of each step / process does not mean the order of execution, and the execution order of each step / process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure. The serial number of the above embodiments of the present disclosure is only for description, not representing the advantages and disadvantages of the embodiments.

[0165] It should be noted that, in the present document, the terms "comprising", "containing" or any other similar term are intended to encompass non-exclusive inclusion, such that processes, methods, articles, or apparatuses that comprise a list of elements are not limited to those elements, but can also include other elements not expressly listed, or also include elements inherent in such processes, methods, articles, or apparatuses. Without further limitation, an element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0166] In several embodiments provided by the present disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the various components shown or discussed can be through some interface, indirect coupling or communication connection between devices or units, which can be electrical, mechanical or other forms.

[0167] The units described above as separate components can or can not be physically separate, and the components shown as units can or can not be physical units; they can be located in one place or distributed on multiple network units; and part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0168] In addition, each functional unit in each embodiment of the present disclosure can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be realized in the form of hardware or in the form of hardware plus software functional unit.

[0169] Those of ordinary skill in the art can understand that all or part of the steps of the above method embodiments can be completed by program instruction related hardware, and the aforementioned program can be stored in a computer readable storage medium, and the program executes the steps of the above method embodiments when executed; and the aforementioned storage medium includes: mobile storage device, read only memory (Read Only Memory, ROM), magnetic disc or optical disc, and various storage program codes.

[0170] Alternatively, the above-mentioned integrated units of the present disclosure, if implemented in the form of software function modules and sold or used as independent products, can also be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present disclosure can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the methods described in the various embodiments of the present disclosure. The aforementioned storage medium includes: mobile storage devices, ROM, magnetic disks or optical disks, and various media that can store program codes.

[0171] The above is only an embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present disclosure, which should be covered within the protection scope of the present disclosure.

Claims

1. A method of floating-point number processing, characterized by, The floating point processing method is applied to a tensor calculation engine, and includes the following steps: obtaining a first floating point number and a second floating point number to be processed currently; the data format of the first floating point number and the second floating point number is a first data format; based on a cycle indication, respectively converting the first floating point number and the second floating point number, and determining a first operation result of the first floating point number multiplied by the second floating point number based on the converted first floating point number and the converted second floating point number; the cycle indication is used to indicate a processing mode of format conversion and a cycle number; the tensor calculation engine is used to convert the first floating point number and the second floating point number by using different processing modes of format conversion in different cycles based on the cycle indication; the converted first floating point number and the converted second floating point number are in a second data format, and the second data format is a simplification of the first data format; wherein the tensor calculation engine further includes a first storage area and a second storage area; the first storage area is used to store the first floating point number or the converted first floating point number; the second storage area is used to store the second floating point number or the converted second floating point number.

2. The floating-point number processing method according to claim 1, wherein The obtaining of the first floating point number and the second floating point number to be processed currently includes: determining a first number of floating point number multiplication operations that can be processed by the tensor calculation engine at a time; reading the first number of floating point numbers from a first address of a local memory as the first floating point number, and reading the first number of floating point numbers from a second address of the local memory as the second floating point number.

3. The floating-point number processing method according to claim 1, wherein The conversion of the first floating point number and the second floating point number based on the cycle indication, and the determination of the first operation result of the first floating point number multiplied by the second floating point number based on the converted first floating point number and the converted second floating point number, include: determining a cycle number based on the cycle indication; the cycle number is used to indicate the corresponding processing mode of format conversion; based on the cycle number, extracting different parts of the first floating point number and the second floating point number by using different processing modes of format conversion in different cycles, to obtain at least two different simplified first floating point numbers and at least two different simplified second floating point numbers; the converted first floating point number includes the simplified first floating point number, and the converted second floating point number includes the simplified second floating point number; determining the first operation result based on the simplified first floating point number and the simplified second floating point number.

4. The floating-point number processing method according to claim 3, wherein The extraction of different parts of the first floating point number and the second floating point number by using different processing modes of format conversion in different cycles based on the cycle number, to obtain at least two different simplified first floating point numbers and at least two different simplified second floating point numbers, includes: In the first cycle, the first function is used to format the first floating-point number and the second floating-point number respectively to obtain a first sub-floating-point number and a second sub-floating-point number; In the second cycle, the first function is used to format the first floating-point number to obtain a first sub-floating-point number, and the second function is used to format the second floating-point number to obtain a third sub-floating-point number; In the third cycle, the second function is used to format the first floating-point number to obtain a fourth sub-floating-point number, and the first function is used to format the second floating-point number to obtain a second sub-floating-point number; In the fourth cycle, the second function is used to format the first floating-point number and the second floating-point number respectively to obtain a fourth sub-floating-point number and a third sub-floating-point number; the simplified first floating-point number includes the first sub-floating-point number and the fourth sub-floating-point number, and the simplified second floating-point number includes the second sub-floating-point number and the third sub-floating-point number.

5. The floating-point number processing method according to claim 4, wherein The first operation result is determined based on the simplified first floating-point number and the simplified second floating-point number, including: In each cycle, the simplified first floating-point number and the simplified second floating-point number are multiplied to obtain a second number of second operation results; The second number of second operation results are added to obtain the first operation result.

6. The floating-point number processing method according to claim 1, wherein The first floating-point number and the second floating-point number are formatted based on the cycle indication, and the first operation result after multiplication of the first floating-point number and the second floating-point number is determined based on the formatted first floating-point number and the formatted second floating-point number, including: Different parts of the first floating-point number and the second floating-point number are extracted using different format conversion processing methods to obtain at least two different simplified first floating-point numbers and at least two different simplified second floating-point numbers; the formatted first floating-point number includes the simplified first floating-point number, and the formatted second floating-point number includes the simplified second floating-point number; In different cycles, different simplified first floating-point numbers and different simplified second floating-point numbers are called based on the cycle number; the cycle number is determined based on the cycle indication; In each cycle, the simplified first floating-point number and the simplified second floating-point number are multiplied to obtain a second number of second operation results; The second number of second operation results are added to obtain the first operation result.

7. The floating-point number processing method according to claim 6, wherein In different cycles, different simplified first floating-point numbers and different simplified second floating-point numbers are called based on the cycle number, including: In the first cycle, a first sub-floating-point number and a second sub-floating-point number are called; In the second cycle, the first sub-floating-point number and a third sub-floating-point number are called; In the third cycle, a fourth sub-floating-point number and the second sub-floating-point number are called; In the fourth cycle, the fourth sub-floating-point number and the third sub-floating-point number are called; the simplified first floating-point number includes the first sub-floating-point number and the fourth sub-floating-point number, and the simplified second floating-point number includes the second sub-floating-point number and the third sub-floating-point number.

8. The floating-point number processing method according to any one of claims 1 to 7, characterized by, The floating-point number processing method further comprises: determining the first storage area from the free memory of the tensor computing engine based on the bit width of the first floating-point number, a data reading delay, and a preset first storage bit; determining the second storage area from the free memory of the tensor computing engine based on the bit width of the second floating-point number, the data reading delay, and a preset second storage bit.

9. The floating-point number processing method according to any one of claims 1 to 7, characterized by, The floating-point number processing method further comprises: in response to a configuration instruction, configuring metadata information of a tensor, and block information of the tensor; each block of the tensor comprises a plurality of floating-point numbers in a first data format; transferring data in any block of the tensor from a global memory to a local memory based on the metadata information and the block information.

10. The floating-point number processing method according to claim 9, wherein The global memory and the local memory store floating-point numbers in a target storage mode; the target storage mode can uniformly distribute a plurality of floating-point numbers calculated each time in different storage areas.

11. A tensor computing engine, comprising: The tensor computing engine comprises: a format conversion unit configured to convert a first floating-point number and a second floating-point number based on a cycle instruction; the cycle instruction is used to indicate a processing mode of format conversion and a cycle number of the tensor computing engine; the tensor computing engine is configured to convert the first floating-point number and the second floating-point number in different cycles based on the cycle instruction and different processing modes of format conversion; data formats of the first floating-point number and the second floating-point number are a first data format; a dot product calculation unit configured to determine a first operation result of the first floating-point number multiplied by the second floating-point number based on the converted first floating-point number and the converted second floating-point number; the converted first floating-point number and the converted second floating-point number are a second data format, which is a simplification of the first data format; The tensor computing engine further comprises a first storage area and a second storage area. The first storage area is configured to store the first floating-point number or the converted first floating-point number. The second storage area is configured to store the second floating-point number or the converted second floating-point number.

12. The tensor computing engine of claim 11, wherein, The dot product calculation unit further comprises: a multiplier configured to perform multiplication operation on the simplified first floating-point number and the simplified second floating-point number in each cycle to obtain a second number of second operation results; the converted first floating-point number comprises the simplified first floating-point number, and the converted second floating-point number comprises the simplified second floating-point number; an exponent alignment unit configured to perform exponent alignment on the second number of second operation results; an adder configured to perform addition operation on the second number of second operation results after the exponent alignment to obtain the first operation result.

Images