Data processing method and device, chip and computer equipment

Abstract

The application relates to the technical field of artificial intelligence chips, and discloses a data processing method, device, chip and computer equipment, in which, first, whether the address of a low-precision data block in memory is continuous is judged according to the memory access granularity, data transmission mode and data arrangement mode of source tensor data; if the address is not continuous, the low-precision data block is described as a high-precision data block through a view conversion mechanism, so that the address continuity advantage of the high-precision data type is utilized, and a more efficient memory access instruction is adapted; subsequently, data processing related operations are performed based on the low-precision data block, and a data processing result of the high-precision data type is obtained; finally, the data processing result of the high-precision data type is described as low-precision result data again, so that the problem of the memory access efficiency bottleneck of low-precision tensor data in a non-continuous memory address scene is solved without changing the original data storage mode, and the memory access efficiency and computing performance are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence chips, and in particular to a data processing method and device, a chip, and a computer device. BACKGROUND

[0002] In artificial intelligence network computing, low-precision data types (such as BF16 and INT8) are widely used due to their high computing efficiency and small memory occupation. In the hardware architecture of related technologies, the data granularity of high-precision data types matches the hardware memory arrangement, so that the special memory access instruction in related technologies can ensure the memory access efficiency of high-precision data types (such as FP32).

[0003] However, the data access mode for low-precision data types affects the performance upper limit, and therefore, an effective data processing method is needed to optimize the memory access mode for low-precision data. SUMMARY

[0004] The present application provides a data processing method and device, a chip, and a computer device, which solve the technical problem of related technologies that the data access mode for low-precision data types affects the performance upper limit, and can improve the memory access efficiency for low-precision data, achieving a balance between memory access efficiency and versatility.

[0005] To achieve the above-mentioned purposes, the main technical solutions adopted by the present application include:

[0006] In a first aspect, the present application provides a data processing method, which includes:

[0007] In the case where the address range corresponding to the low-precision data block is determined to be discontinuous in the memory according to the memory access granularity, data transmission mode, and data arrangement mode of the source tensor data, the low-precision data block described in a first data type is described as a high-precision data block in a second data type; wherein the precision of the second data type is higher than that of the first data type.

[0008] Data processing related operations are performed based on the low-precision data block in a manner corresponding to the second data type, and a data processing result in the second data type is obtained; wherein the data processing related operations include at least one of data transfer and data calculation.

[0009] The data processing result is described as low-precision result data in the first data type.

[0010] Optionally, the data processing related operations are performed based on the low-precision data block in a manner corresponding to the second data type, and a data processing result in the second data type is obtained, including:

[0011] read the low-precision data block from the memory to a register by using a loading mode corresponding to the second data type, to obtain high-precision equivalent data in the second data type;

[0012] perform a data processing related operation based on the high-precision equivalent data in the register, to obtain a data processing result in the second data type.

[0013] Optionally, the performing of the data processing related operation based on the high-precision equivalent data in the register, to obtain the data processing result in the second data type, comprises:

[0014] storing the high-precision equivalent data from the register to the memory by using a storage mode corresponding to the second data type, as the data processing result in the second data type.

[0015] Optionally, the performing of the data processing related operation based on the high-precision equivalent data in the register, to obtain the data processing result in the second data type, comprises:

[0016] describing the high-precision equivalent data as a combination of N low-precision equivalent data in the first data type; wherein N is an integer greater than or equal to 2;

[0017] performing a first type calculation by using the N low-precision equivalent data, to obtain N first intermediate calculation results in the first data type;

[0018] performing a re-description and storage based on the N first intermediate calculation results in the first data type, to obtain the data processing result in the second data type.

[0019] Optionally, the performing of the re-description and storage based on the N first intermediate calculation results in the first data type, to obtain the data processing result in the second data type, comprises:

[0020] describing the N first intermediate calculation results as a whole as a first calculation result in the second data type;

[0021] storing the first calculation result to the memory by using a storage mode corresponding to the second data type, to obtain the data processing result in the second data type.

[0022] Optionally, the performing of the data processing related operation based on the low-precision data block by using the mode corresponding to the second data type, to obtain the data processing result in the second data type, comprises:

[0023] read the low-precision data block from the memory to the cache by using a loading mode corresponding to the second data type, to obtain a high-precision data block in the second data type;

[0024] describe the high-precision data block in the cache as a low-precision data block in the first data type;

[0025] read the low-precision data block from the cache to a register and perform a first type conversion by using a loading mode corresponding to the first data type, to obtain high-precision data in a third data type;

[0026] perform a second type calculation and a second type conversion based on the high-precision data in the register, to obtain a second intermediate calculation result in the first data type;

[0027] re-describe and store the second intermediate calculation result based on the cache, to obtain a data processing result in the second data type.

[0028] Optionally, the precision of the third data type is higher than the precision of the first data type, and the precision of the third data type is lower than the precision of the second data type.

[0029] Optionally, the second type calculation is a high-precision calculation with a precision requirement; and the performing of the second type calculation and the second type conversion based on the high-precision data in the register to obtain the second intermediate calculation result in the first data type comprises:

[0030] perform a high-precision calculation in the register by using high-precision data in the third data type, to obtain a high-precision calculation result in the third data type;

[0031] convert the high-precision calculation result from the third data type to the first data type, to obtain the second intermediate calculation result in the first data type.

[0032] Optionally, the re-describing and storing the second intermediate calculation result based on the cache to obtain the data processing result in the second data type comprises:

[0033] store the second intermediate calculation result from the register to the cache by using a storage mode corresponding to the first data type;

[0034] re-describe the second intermediate calculation result as a second calculation result in the second data type;

[0035] The second calculation result is stored from the cache to the memory using a storage method corresponding to the second data type, thereby obtaining a data processing result using the second data type.

[0036] In a second aspect, an embodiment of the present application provides a data processing device, the device comprising:

[0037] a data block description module, configured to, when it is determined based on the memory access granularity, data transmission mode, and data arrangement mode of the source tensor data that an address range corresponding to the low-precision data block is discontinuous in the memory, describe the low-precision data block of a first data type as a high-precision data block of a second data type; wherein the second data type has a higher precision than the first data type;

[0038] a data processing operation module, configured to perform data processing-related operations based on the low-precision data block in a manner corresponding to the second data type, to obtain a data processing result using the second data type; wherein the data processing-related operations include at least one of data movement and data calculation;

[0039] A result redescription module is configured to redescribe the data processing result as low-precision result data using the first data type.

[0040] In a third aspect, an embodiment of the present application provides a chip, including:

[0041] processor;

[0042] a memory for storing processor-executable instructions;

[0043] The processor is configured to perform any of the above-mentioned data processing methods when executing the instructions stored in the memory.

[0044] In a fourth aspect, an embodiment of the present application provides a computer device comprising a memory, an artificial intelligence chip, and a computer program stored in the memory and executable on the artificial intelligence chip, wherein the artificial intelligence chip implements any of the above-described data processing methods when executing the computer program.

[0045] In the embodiment of the present application, first, it is judged whether the address of the low-precision data block in the memory is continuous according to the access granularity of the source tensor data, the data transmission mode and the data arrangement mode; if the address is not continuous, the low-precision data block is described as a high-precision data block through a view conversion mechanism, so as to utilize the address continuity advantage of the high-precision data type and adapt more efficient access instructions; then, the data processing related operation is performed based on the low-precision data block, and the data processing result of the high-precision data type is obtained; finally, the data processing result of the high-precision data type is described as a low-precision result data, so as to solve the access efficiency bottleneck problem of the low-precision tensor data in the non-continuous memory address scene without changing the original data storage mode, and improve the access efficiency and the computing performance. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the specific embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0047] Figure 1 The schematic structural diagram of a general-purpose graphics processor provided by an embodiment of the present application is shown in the figure;

[0048] Figure 2 The data arrangement schematic diagram of the tensor data provided by an embodiment of the present application is shown in the figure;

[0049] Figure 3 The flowchart of the data processing method provided by an embodiment of the present application is shown in the figure;

[0050] Figure 4 The flowchart of the data processing method provided by an embodiment of the present application is shown in the figure;

[0051] Figure 5 The data processing process in the access scene provided by an embodiment of the present application is shown in the figure;

[0052] Figure 6 The flowchart of the data processing method provided by another embodiment of the present application is shown in the figure;

[0053] Figure 7 The flowchart of the data processing method provided by another embodiment of the present application is shown in the figure;

[0054] Figure 8 The data processing process in the low-precision computing scene provided by an embodiment of the present application is shown in the figure;

[0055] Figure 9 The flowchart of the data computing process provided by an embodiment of the present application is shown in the figure;

[0056] Figure 10 a data processing process in a high-precision computing scenario provided by an embodiment of the present application;

[0057] Figure 11 a schematic diagram of a framework of a data processing apparatus provided by an embodiment of the present application;

[0058] Figure 12 a schematic structural diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will be combined with the accompanying drawings for the embodiments of the present application to clearly and completely describe the technical solutions of the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0060] The data processing method of the present application is based on artificial intelligence (AI). Artificial intelligence is to use a digital computer or a machine controlled by a digital computer to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence.

[0061] The data processing method of the present application can be applied to an artificial intelligence processor, which can be any one of a GPU (Graphics Processing Unit), a TPU (Tensor Processing Unit), and a GPGPU (General-Purpose Graphics Processing Unit). Taking a general-purpose graphics processing unit (GPGPU) as an example for illustration.

[0062] Figure 1 The schematic structural diagram of a general-purpose graphics processing unit (GPGPU) is shown. Please refer to Figure 1 The general-purpose graphics processing unit is actually an array of programmable multi-processors, for example, the programmable multi-processors can be a streaming processor cluster (SPC), which includes Figure 1The illustrated stream processor cluster 1 … stream processor cluster M-1, stream processor cluster M, M is a positive integer greater than 1. In a general-purpose graphics processor, one stream processor cluster processes a computing task, or multiple stream processor clusters process a computing task. Taking stream processor cluster 1 as an example, one stream processor cluster includes multiple processing units (Excute Unit, referred to as EU for short, also known as execution unit). Each processing unit includes an arithmetic logic unit (ALU), a floating point calculation unit, etc., and the processing unit is used to execute a specific computing task. The processing unit includes a register (Thread Local Register, TLR) for storing source data and destination data related to the computing task, and a global memory buffer (Global Memory Buffer, GMB) can connect the EU and the high bandwidth memory (High Bandwidth Memory, HBM) for data transfer and improve the transmission efficiency between the EU and the HBM. The HBM provides large-capacity, low-latency storage to support the high-performance computing requirements of the GPU. Instructions enter the SPC from the outside and are allocated to each EU for execution. Input data is read from the HBM, transmitted to the SPC through the GMB, and distributed to the EU for processing. The results (output data) of the EU processing are returned to the GMB through the SPC and finally written to the HBM.

[0063] Please refer to Figure 2 , which exemplarily takes tensor data in a column-major layout (matrix colmajor layout) as an example for relevant description. Figure 2 The data arrangement of FP32, BF16, S8 / U8 data types in the colmajor layout is shown in Figure 2 . It should be noted that the dark part in Figure 2 is the data block operated once when 256B memory access is performed. For example, for data type FP32, two 128B are arranged continuously; for data type BF16, the front 128B (64Bx2) and the rear 128B (64Bx2) are arranged at intervals; for data type S8 / U8, the front 128B (32Bx4) and the rear 128B (32Bx4) are arranged at intervals.

[0064] In some cases, it is necessary to access with a higher granularity, such as at least 256 bytes, which will be more friendly to the performance of the hardware. This is because there are cachelines on the bus, L2 cache and HBM, with the minimum granularity of reading. It is worth noting that for the memory access operation of the matrix colmajor layout, the burst mode of different data types and the data arrangement characteristics directly affect the performance. The hardware requires a data access granularity of at least 256 bytes to improve efficiency, but the non-continuous address interval will limit the actual performance upper limit. Specifically, please refer to Table 1.

[0065] Table 1

[0066]

[0067] Regarding the data type FP32, the data-intensive arrangement is fully adapted to the hardware instruction, that is, without crossing 2048 bytes, the memory access efficiency is high.

[0068] Regarding the data type BF16, the first 128B (64Bx2) and the last 128B (64Bx2) are separated by 2048 bytes, and the whole data needs to cross the interval, so the use scenario is limited.

[0069] Regarding the data type S8 / U8, the first 128B (32Bx4) and the last 128B (32Bx4) are separated by 2048 bytes, and the whole data needs to cross the interval, which limits the universality.

[0070] It is found through analysis that, under the colmajor layout, the 256Byte burst reading of low-precision data (BF16 / S8 / U8) needs to cross the 2048Byte interval, which causes the hardware to be unable to efficiently load data through continuous memory access instructions (such as burst2 / burst4). For example, taking BF16 as an example, 256Byte reading needs to access four 64B data blocks, the addresses are (0, 0), (1, 0), (32, 0), and (33, 0), the first two segments (0, 0) and (1, 0) are separated from the last two segments ((32, 0), (33, 0)) by 2048 bytes. For example, taking S8 / U8 as an example, 256Byte reading needs to access eight 32B data blocks, the addresses are (0, 0), (1, 0), (2, 0), (3, 0), (64, 0), (65, 0), (66, 0), and (67, 0), the first four segments (0, 0), (1, 0), (2, 0), and (3, 0) are separated from the last four segments (64, 0), (65, 0), (66, 0), and (67, 0) by 2048 bytes.

[0071] It should be noted that the memory layout and arrangement mode of data of different data types stored on the chip are different in different chips, which limits the transmission mode of input and output of data of different data types. The burst mode is a data transmission mode. In the burst mode, after the start address and burst lengths are specified, the same number of storage units starting from the start address are automatically subjected to continuous read / write operations in the data transmission process. In the burst mode, burstm can represent loading m data subblocks at a time, for example, burst4 represents loading 4 data subblocks at a time, and burst8 represents loading 8 data subblocks at a time. The burst mode that can be used by different data types is also limited accordingly.

[0072] In the presence of the above interval for data type BF16 or data type S8 / U8, the overall data size must also cross 2048 bytes in the splitting process, and the use scenario is limited. In addition, the address interval limit affects the upper limit of performance in actual data access.

[0073] Based on this, the embodiment of the present application provides a data processing method suitable for the non-continuous address scenario of low-precision data under some data layout modes (such as column-first arrangement). Specifically, the low-precision data is described as high-precision data through the view operation, the address continuity advantage of the high-precision data is utilized, the special access instruction of the related hardware architecture is compatible, and the performance loss is reduced. It should be noted that the view operation in the embodiment of the present application can be regarded as typecast view, and the data carrying operation or calculation operation can be performed with a higher-precision data type by updating the interpretation mode or description of the data, without involving actual data copying or physical data conversion; or the view operation in the embodiment of the present application can be understood as logically describing the low-precision data as high-precision data, without changing the actual physical arrangement and content of the low-precision data.

[0074] The embodiment of the present application processes the low-precision data as another data type (a higher-precision data type), and the underlying data does not change at all, so that the data is carried by using the memory access instruction more suitable for the high-precision data type in logic, the address continuity advantage of the high-precision data is utilized to complete efficient memory access, and after the calculation is completed, the calculation result is re-described or re-interpreted, the actual description of the low-precision data is restored, and the low-precision data is stored back into the memory, thereby realizing the dual improvement of memory efficiency and universality.

[0075] According to an embodiment of the present application, a data processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown. Please refer to Figure 3 , Figure 3 The flowchart shown in FIG. 1 shows the flow of the data processing method according to an embodiment of the present application. The data processing method comprises the following steps:

[0076] S110, in the case where the address range corresponding to the low-precision data block is discontinuous in the memory according to the memory access granularity, data transmission mode and data arrangement mode of the source tensor data, the low-precision data block of the first data type is described as a high-precision data block of the second data type.

[0077] Wherein, the precision of the second data type is higher than that of the first data type. The first data type refers to the data type used to represent the low-precision data block. For example, the first data type can be BF16, FP16, INT8 or UINT8, etc. The second data type refers to the data type used to logically re-interpret the low-precision data block, and its precision is higher than that of the first data type. For example, the second data type can be FP32, which has higher numerical precision than BF16 or FP16. By describing the low-precision data block as a high-precision data type, the address continuity advantage of the high-precision data type can be utilized, thereby adapting to more efficient memory instructions.

[0078] Wherein, the source tensor data can be a set of tensor-form data input into the data processing flow, which is usually used in deep learning, image processing, matrix operation, etc. For example, the source tensor data can be the weight parameters, activation value tensor in the convolutional neural network, or the pixel data tensor in image processing. The source tensor data can be stored in the memory in different data arrangement modes.

[0079] Wherein, the data arrangement mode refers to the way in which the tensor data is organized and stored in the memory. For example, row-major or column-major arrangement, etc. Different data arrangement modes will affect the continuity of data in the memory, and thus affect the memory efficiency. In the column-major arrangement mode, when the memory access granularity is large, cross-column access is likely to occur, resulting in discontinuous data addresses.

[0080] The low-precision data block refers to a local block of source tensor data stored by using a low-precision data type. For example, the low-precision data block can be a data block of BF16, FP16, INT8 (signed 8-bit integer), or the like. The low-precision data block is usually used to save memory bandwidth and improve computing throughput, but it can cause an address discontinuity problem under a specific data arrangement.

[0081] Specifically, whether the address of the low-precision data block in the memory is continuous is determined according to the memory access granularity, the data transmission manner, and the data arrangement manner of the source tensor data. If the address is discontinuous, the low-precision data block is described as a high-precision data block through a view operation, so as to take advantage of the address continuity of the high-precision data type.

[0082] It can be understood that the described operation (view) in the embodiment can be understood as logical re-interpretation or description of the low-precision data block through view conversion, so as to adapt to the memory access instruction of the high-precision data type, rather than physical data operation. Or, the data type with discontinuous address during memory access is described as the data type with continuous address during memory access. For example, the low-precision data block of the first data type is described as the high-precision data block of the second data type, the data block of BF16 is viewed as the data block of FP32, the data block of FP16 is viewed as the data block of FP32, the data block of INT8 is viewed as the data block of FP32, the data block of Uint8 is viewed as the data block of FP32, and the data block of FP8 is viewed as the data block of FP32.

[0083] It should be noted that the view operation in the embodiment of the present application and the view operator in the related art are both modifying the description of data without changing the actual content of the data, but they are different. The view operator in the related art is an operator in a calculation graph, while the view operation in the embodiment of the present application uses a typecast view that does not originally exist for performance optimization, that is, a process of converting a view or representation form of a certain data type into a view or representation form of another data type. The view operation in the embodiment of the present application can be essentially understood as creating a new descriptor (descriptor) to describe or interpret the low-precision data as high-precision data. The type conversion is realized logically through the DATA_TYPE field of the descriptor, and the access rule of the low-precision data block is redefined.

[0084] It can be understood that in the deep learning and hardware acceleration scene, the Tensor Desc is a specific implementation of the descriptor, which is used to define the key information of the Tensor, such as: data type (such as BF16, FP32, INT8), arrangement layout (such as row-major arrangement, column-major arrangement), dimension information (channel number, width, height, depth, batch number), stride, padding (padValue) and the like.

[0085] S120, in a manner corresponding to the second data type, performing a data processing related operation based on the low-precision data block to obtain a data processing result in the second data type.

[0086] The data processing related operation includes at least one of data carrying and data calculation, and can include but is not limited to matrix operation, data carrying and the like.

[0087] The data processing result refers to a processing result obtained after the data processing related operation, and the data type thereof is the second data type. For example, after carrying or calculating using FP32 precision, the data processing result obtained is also data in FP32 precision, and the numerical precision is higher than that of the original BF16 or INT8 data.

[0088] Specifically, after logically describing the low-precision data block as a high-precision data block, the data is operated or processed using a data processing manner corresponding to the high-precision data type to obtain a data processing result in the high-precision data type.

[0089] S130, re-describing the data processing result as low-precision result data in the first data type.

[0090] The low-precision result data can be low-precision output data in the first data type. Specifically, the data processing result in the high-precision data type is re-interpreted as low-precision data type through view conversion. For example, the output data in FP32 precision is re-interpreted as data in BF16 or INT8 precision through view operation, and is written back to the memory using a processing manner corresponding to the first data type, thereby ensuring compatibility with subsequent processes.

[0091] In the above embodiments, first, it is judged whether the address of the low-precision data block in the memory is continuous according to the access granularity of the source tensor data, the data transmission mode and the data arrangement mode; if the address is not continuous, the low-precision data block is described as a high-precision data block through a view conversion mechanism, so as to utilize the address continuity advantage of the high-precision data type and adapt more efficient access instructions; then, data processing related operations are performed based on the low-precision data block to obtain a data processing result of the high-precision data type; finally, the data processing result of the high-precision data type is described as a low-precision result data, so as to solve the access efficiency bottleneck problem of the low-precision tensor data in the non-continuous memory address scene without changing the original data storage mode, and improve the access efficiency and the computing performance.

[0092] In some embodiments, referring to Figure 4 , the data processing related operations are performed based on the low-precision data block in a manner corresponding to the second data type to obtain a data processing result in the second data type, including:

[0093] S310, the low-precision data block is read from the memory to the register in a loading manner corresponding to the second data type to obtain high-precision equivalent data in the second data type.

[0094] S320, data processing related operations are performed based on the high-precision equivalent data in the register to obtain a data processing result in the second data type.

[0095] The loading manner corresponding to the second data type can be to select an access granularity and a data transmission instruction suitable for the second data type according to the data access characteristics of the second data type, and to use the access granularity and the data transmission instruction to continuously load the low-precision data block from the memory to the register.

[0096] The high-precision equivalent data can be a data form presented in the register after the low-precision data block is logically interpreted or described as a high-precision data type through view conversion logic. It should be noted that although the underlying data does not change, it is treated as high-precision data in the register, so that it can be read using a high-precision data type read instruction.

[0097] Specifically, first, the low-precision data block is read from the memory to the register by using the loading mode corresponding to the second data type, and the high-precision equivalent data in the second data type is obtained; then, the data processing related operation is performed based on the high-precision equivalent data in the register, and finally the data processing result in the second data type is obtained. By introducing the view conversion mechanism in the data loading stage, the low-precision data block with discontinuous addresses caused by data arrangement can be efficiently loaded in the equivalent form of high-precision data type, thereby improving the overall data memory efficiency. For example, when the first data type is BF16 and the second data type is FP32, the memory granularity of burst2 (reading 256 bytes each time) and the loading instruction dedicated to FP32 can be selected, so that the data block with discontinuous addresses stored in BF16 format can be loaded into the register in FP32 format.

[0098] Further, the data processing related operation is performed based on the high-precision equivalent data in the register, and the data processing result in the second data type is obtained, including: storing the high-precision equivalent data from the register to the memory by using the storage mode corresponding to the second data type, and taking the data processing result in the second data type.

[0099] Among them, the storage mode corresponding to the second data type refers to selecting the memory granularity and data transmission instruction suitable for the data type according to the data storage characteristics of the second data type, for writing the high-precision equivalent data in the register to the memory. Specifically, the high-precision equivalent data is transmitted from the register to the memory by using the memory access instruction. For example, in the GPU or AI acceleration chip, the high-precision equivalent data is written back from the register (Register) to the memory of the chip. In this embodiment, the data write-back process is performed based on the storage mode of the second data type, so as to ensure that the data can adapt to the optimized instruction set of the hardware platform when written back to the memory.

[0100] In the above embodiment, by introducing the storage mode matching the second data type after the data processing is completed, and by using the storage mode corresponding to the second data type, the problem that efficient storage instructions cannot be used when the address is discontinuous due to data arrangement is solved; by storing the result in the memory in the high-precision data type, it is ensured that the high-precision equivalent data can be written back to the memory in an efficient and compatible manner, thereby completing the entire data processing process.

[0101] Please refer to Figure 5, exemplary data processing process in memory access scenarios is illustrated. When data only needs to be moved, taking BF16 as an example, 1) view the low-precision data of BF16 type as high-precision data of FP32 type; 2) use the FP32 corresponding load method to read the high-precision data of FP32 type into the register; 3) use the FP32 corresponding store method to write to HBM; 4) view the high-precision data in HBM back to the original data type (i.e. BF16), thus completing the data movement.

[0102] In some embodiments, referring to Figure 6 , based on the high-precision equivalent data in the register, a data processing related operation is performed to obtain a data processing result in the second data type, including:

[0103] S610, the high-precision equivalent data is described as a combination of N low-precision equivalent data in the first data type.

[0104] Wherein, N is an integer greater than or equal to 2. N can be the number of high-precision equivalent data split into low-precision equivalent data, and the value of N depends on the byte relationship between the high-precision data type (second data type) and the low-precision data type (first data type). For example, when the second data type is FP32 (4Byte) and the first data type is BF16 (2Byte), N=2; when the second data type is FP32 (4Byte) and the first data type is INT8 (1Byte), N=4. The selection of N needs to ensure that the combination of low-precision equivalent data after splitting can completely cover the numerical range of high-precision equivalent data, and adapt to the characteristics of the calculation unit of the hardware platform.

[0105] Specifically, through the view conversion mechanism, the high-precision equivalent data in the register is logically split into a combination of multiple low-precision data blocks. For example, when the high-precision equivalent data is of FP32 type and the first data type is of BF16 type, each FP32 data can be split into a combination of two BF16 data (i.e. N=2). Although the underlying data has not changed physically, it is interpreted as a combination of low-precision data in the register, so that low-precision calculation instructions can be called for processing.

[0106] S620, using N low-precision equivalent data to perform first type calculation to obtain N first intermediate calculation results in the first data type.

[0107] The first type of calculation can be a calculation of the split low-precision equivalent data by a calculation instruction of a low-precision data type. For example, parallel calculation of the split low-precision equivalent data by calling an addition instruction of BF16 or INT8. The first type of calculation can be understood as low-precision calculation without precision requirement. The first intermediate calculation result can be a calculation result stored in the first data type after processing by a low-precision calculation instruction.

[0108] Specifically, the split low-precision equivalent data is calculated by a calculation instruction corresponding to the first data type, and the obtained first intermediate result is still of the first data type. For example, if the split BF16 equivalent data is calculated by a BF16 calculation instruction, the obtained intermediate calculation result is still of the BF16 type. This process ensures the low-precision characteristics of the calculation stage, thereby fully exploiting the optimization capability of the hardware platform for low-precision calculation.

[0109] S630, based on the N first intermediate calculation results in the first data type, re-describe and store to obtain a data processing result in the second data type.

[0110] Specifically, the low-precision first intermediate calculation result is logically re-interpreted as a high-precision data type by a view conversion mechanism, and is written into the memory according to the storage mode of the second data type to obtain a data processing result in the second data type. For example, two BF16 intermediate calculation results are combined into one FP32 data processing result, and are written into the memory by an FP32 storage instruction. This process realizes seamless connection between low-precision calculation and high-precision result through the view conversion mechanism, while maintaining the compatibility of the data processing flow.

[0111] Further, please refer to Figure 7 , based on the N first intermediate calculation results in the first data type, re-describe and store to obtain a data processing result in the second data type, comprising:

[0112] S710, the N first intermediate calculation results are described as a first calculation result in the second data type as a whole.

[0113] S720, the first calculation result is stored into the memory by a storage mode corresponding to the second data type to obtain a data processing result in the second data type.

[0114] Specifically, first, the N low-precision first intermediate calculation results are logically combined into a high-precision first calculation result by a view conversion; second, a storage mode corresponding to the second data type is selected to write the high-precision first calculation result in the register into the memory to obtain a high-precision data type data processing result.

[0115] In the above embodiments, by logically merging the low-precision first intermediate calculation result into the high-precision first calculation result of the high-precision data type through view conversion, and adapting the storage mode of the high-precision data type, the problem of being unable to use efficient storage instructions when the address is discontinuous due to data arrangement is solved, and the calculation efficiency of low-precision and the data processing result output of high-precision are realized.

[0116] Please refer to Figure 8 , which exemplarily illustrates the data processing process in the low-precision calculation scenario. When the data needs to be calculated in low precision, taking BF16 as an example, 1) view the low-precision data of BF16 type in HBM into high-precision data of FP32 type; 2) use the load mode corresponding to FP32 to read the high-precision data of FP32 type from HBM into the register; 3) view the high-precision data of FP32 into the combination of two low-precision data of BF16 type, and perform low-precision calculation to obtain two low-precision calculation results of BF16 type; 4) view the two low-precision calculation results of BF16 type into the high-precision calculation result of FP32; 5) use the store mode corresponding to FP32 to write the high-precision calculation result of FP32 back to HBM; 6) view the high-precision calculation result of FP32 back to the original data type (i.e. BF16), and thus the low-precision calculation is completed.

[0117] In some embodiments, please refer to Figure 9 , the data processing related operation is performed based on the low-precision data block in a manner corresponding to the second data type, and a data processing result in the second data type is obtained, including:

[0118] S910, the low-precision data block is read from the memory to the cache in a loading manner corresponding to the second data type, and a high-precision data block in the second data type is obtained.

[0119] Among them, the cache refers to an intermediate storage unit between the memory and the register, which is usually used to temporarily store high-frequency access data to reduce memory access delay, and the cache can be GMB.

[0120] In some cases, since the data precision of the register for processing the low-precision data block is higher than the precision of the first data type, directly loading the low-precision data block into the register will lose the calculation precision, and therefore the cache is used for the transfer operation. Specifically, according to the data access characteristics of the second data type, a memory access granularity and a data transfer instruction suitable for the second data type are selected to load the low-precision data block from the memory to the cache. For example, when the second data type is FP32, the memory access granularity of burst2 (reading 256 bytes at a time) and the FP32-specific loading instruction can be selected, so that the low-precision data block originally stored in BF16 format and discontinuous address is loaded into the cache in FP32 format. This process logically interprets the low-precision data as high-precision data through the view conversion mechanism, so as to adapt to the loading instruction of the high-precision data type. S920, the high-precision data block in the cache is described as a low-precision data block in the first data type.

[0121] Specifically, the high-precision data block in the cache is logically reinterpreted as a low-precision data type through the view conversion mechanism. For example, the FP32 data block in the cache is logically interpreted as a BF16 data block, although the underlying data does not change, but it is treated as a low-precision data block in the cache, so as to adapt to the loading instruction of the low-precision data type. This process provides a basis for subsequent loading of data from the cache to the register and type conversion.

[0122] S930, the low-precision data block is read from the cache to the register in a loading manner corresponding to the first data type and is subjected to a first type conversion, to obtain high-precision data in a third data type.

[0123] The precision of the third data type is higher than the precision of the first data type, and the precision of the third data type is lower than the precision of the second data type. The first type conversion refers to a conversion operation of converting the low-precision data block to high-precision data in the third data type through a hardware mechanism after being loaded into the register.

[0124] In some cases, in order to ensure the calculation precision, the cache is used for the transfer operation, and further, the first type conversion is performed in the process of loading the low-precision data block from the cache to the register after the transfer operation. Specifically, according to the data access characteristics of the first data type, a memory access granularity and a data transfer instruction suitable for the first data type are selected to load the low-precision data block in the cache into the register. In the register, the low-precision data is subjected to type conversion to obtain high-precision data in the third data type. Exemplarily, the third data type can be BF20, which has a higher precision than BF16 or INT8, and a lower precision than the precision of the second data type (such as FP32).

[0125] S940, performing a second type calculation and a second type conversion based on the high-precision data in the register to obtain a second intermediate calculation result in the first data type.

[0126] The second type calculation can be a high-precision calculation. Specifically, in the register, a high-precision calculation operation is performed on the high-precision data in the third data type and is converted into the first data type to obtain the second intermediate calculation result in the first data type. For example, an accumulation is performed on the BF20 data in the register, and the BF20 data is converted into BF16 or INT8 precision data through a hardware mechanism.

[0127] Further, the second type calculation is a high-precision calculation with precision requirements. The second type calculation and the second type conversion based on the high-precision data in the register to obtain the second intermediate calculation result in the first data type include: in the register, performing a high-precision calculation on the high-precision data in the third data type to obtain a high-precision calculation result in the third data type; and converting the high-precision calculation result from the third data type to the first data type to obtain the second intermediate calculation result in the first data type.

[0128] S950, with the cache as a transit, performing a re-description and storage based on the second intermediate calculation result to obtain a data processing result in the second data type.

[0129] Specifically, through a view conversion mechanism, the low-precision second intermediate calculation result is logically re-interpreted as data in the second data type, and is written into the memory through the cache transit to obtain the data processing result in the second data type.

[0130] Further, with the cache as a transit, performing a re-description and storage based on the second intermediate calculation result to obtain a data processing result in the second data type includes: storing the second intermediate calculation result from the register to the cache in a storage mode corresponding to the first data type; re-describing the second intermediate calculation result as a second calculation result in the second data type; and storing the second calculation result from the cache to the memory in a storage mode corresponding to the second data type to obtain the data processing result in the second data type.

[0131] In the above embodiment, first, the low-precision data block is loaded into the cache in the high-precision data type; then, the high-precision data block in the cache is logically interpreted as a low-precision data block and loaded into the register for the first type conversion; then, the second type calculation and the second type conversion are performed in the register to generate a low-precision second intermediate calculation result; finally, the second intermediate calculation result is re-described as a high-precision data type through the cache transfer and written into the memory. Through the multi-stage type conversion and the cache transfer, the above process realizes efficient loading of low-precision data, high-precision calculation and result reorganization, while maintaining the compatibility of the data processing process.

[0132] Please refer to Figure 10 , which exemplarily illustrates the data processing process in the high-precision calculation scenario. When data needs to be calculated with high precision, taking BF16 as an example, 1) the low-precision data of the BF16 type in the HBM is viewed as high-precision data of the FP32 type; 2) the high-precision data of the FP32 type is loaded from the HBM into the GMB using the load mode corresponding to the FP32; 3) in the GMB, the high-precision data of the FP32 type is viewed as a combination of two low-precision data of the BF16 type; 4) the low-precision data of the BF16 type is loaded into the register using the load mode corresponding to the BF16, and the data is converted from the BF16 type to the BF20 type; 5) in the register, high-precision calculation is performed using the data of the BF20 type to obtain a high-precision calculation result; 6) the high-precision calculation result is converted to a calculation result of the BF16 type, and the calculation result is stored in the GMB using the store mode corresponding to the BF16 type; 7) the calculation result of the BF16 type in the GMB is described as a calculation result of the FP32 type; 8) the calculation result of the FP32 type is stored in the HBM using the store mode corresponding to the FP32; 9) the calculation result of the FP32 type is viewed back to the original data type (i.e., the BF16), and thus the high-precision calculation is completed. It should be noted that since the data precision of the low-precision data of the BF16 type processed in the register is BF20, the precision will be lost if the data is directly loaded into the register, and therefore the GMB is needed for transfer. After the transfer, the data completes the type conversion from BF16 to BF20 in the process of being loaded into the register.

[0133] The embodiment of the present application provides a data processing device, Figure 11 is a schematic diagram of the framework of the data processing device provided by the embodiment of the present application. The data processing device 1200 can include a data block description module 1210, a data processing operation module 1220 and a result re-description module 1230.

[0134] The data block description module 1210 is configured to, in a case where it is determined that an address range corresponding to a low-precision data block is discontinuous in the memory according to a memory access granularity, a data transmission manner, and a data arrangement manner of source tensor data, describe the low-precision data block in a first data type as a high-precision data block in a second data type, where the precision of the second data type is higher than that of the first data type.

[0135] The data processing operation module 1220 is configured to perform a data processing related operation based on the low-precision data block in a manner corresponding to the second data type to obtain a data processing result in the second data type, where the data processing related operation includes at least one of data transfer and data calculation.

[0136] The result re-description module 1230 is configured to re-describe the data processing result as low-precision result data in the first data type.

[0137] For the convenience of description, the above apparatus is described in various modules in terms of functions. Of course, in the implementation of the present application, the functions of the modules can be implemented in one or more software and / or hardware.

[0138] The chip provided in the embodiments of the present application includes a processor, a memory for storing processor-executable instructions, and the processor is configured to implement the above method when executing the instructions stored in the memory.

[0139] The processor involved in the embodiments of the present application can be any one of a CPU (Central Processing Unit, central processor), a GPU (Graphics Processing Unit, graphics processor), a TPU (Tensor Processing Unit, tensor processor), a NPU (Neural network Processing Unit, neural network processor), a DPU (Deeplearning Processing Unit, deep learning processor), an APU (Accelerated Processing Unit, accelerated processor), and a GPGPU (General-Purpose computing on Graphics Processing Unit, general-purpose graphics processor), as determined when the embodiments of the present application are applied to specific products or technologies.

[0140] The embodiment of the present application provides a kind of non-transient computer readable storage medium, storage medium can be non-transient computer readable storage medium, one or more computer readable instructions can be stored on storage medium non-transiently.For example, when computer readable instruction is executed by processor, one or more steps in the above data processing method can be executed.The storage medium can be applied in electronic equipment, for example, the storage medium can include storage device in electronic equipment.

[0141] Storage device can include one or more computer program products of any combination, computer program product can include various forms of computer readable storage medium, such as volatile memory and / or non-volatile memory.For example, volatile memory can include random access memory (RAM) and / or cache memory (cache) and the like.Non-volatile memory can include read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory and the like.One or more computer readable instructions can be stored on computer readable storage medium, and processor can run computer readable instructions to realize the various functions of processor.In storage medium, various application programs and various data and the like can also be stored.

[0142] Storage medium can include the memory card of smart phone, the cache component of tablet computer, the hard disk of personal computer, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), flash memory, or any combination of the above storage medium, and can also be other applicable storage medium.

[0143] The embodiment of the present application provides a kind of computer equipment, as shown in Figure 12 The embodiment of the present application provides a kind of computer equipment, as shown in Figure 12 The embodiment of the present application provides a kind of computer equipment, as shown in

[0144] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, a system or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer program code thereon for use by or in connection with an instruction execution system. For the purposes of this description, a computer-usable or computer readable storage medium can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

[0145] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other processing device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other processing device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. Figure 1 The flowchart and / or block diagram in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to the present application. In this regard, each block in the flowchart and / or block diagrams can represent a module, segment, or portion of code, which comprises one or more executable Figure 1 The means for performing the function specified by one or more of the flowchart or block diagram blocks can comprise a processor configured to perform the function in response to computer program instructions, a device or circuitry configured to perform the function, or a combination thereof.

[0146] It should be noted that the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0147] Various embodiments of the present application are described in the specification in a progressive manner, and the same or similar parts between embodiments can be mutually referred to. Each embodiment focuses on the difference from other embodiments. In particular, the system embodiments are described simply because they are basically similar to the method embodiments, and the relevant parts can be referred to the description of the method embodiments.

[0148] The above merely provides embodiments of the present application, but does not serve to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of the claims of the present application.

[0149] While embodiments of the application have been described in connection with the preferred embodiments of the various figures, those of ordinary skill in the art will appreciate that various modifications and changes can be made without departing from the spirit and scope of the application, and that such modifications and changes fall within the scope of the appended claims.

Claims

1. A data processing method, characterized by, The method comprises: In the case that the address range corresponding to the low-precision data block is discontinuous in the memory according to the access granularity of the source tensor data, the data transmission mode and the data arrangement mode, the low-precision data block in the first data type is described as a high-precision data block in a second data type; wherein the precision of the second data type is higher than that of the first data type; Performing data processing related operations based on the low-precision data block in the manner corresponding to the second data type to obtain a data processing result in the second data type; wherein the data processing related operations include at least one of data carrying and data calculation; Re-describe the data processing result as low-precision result data in the first data type.

2. The method of claim 1, wherein, The performing data processing related operations based on the low-precision data block in the manner corresponding to the second data type to obtain a data processing result in the second data type comprises: Reading the low-precision data block from the memory into a register in the loading manner corresponding to the second data type to obtain high-precision equivalent data in the second data type; Performing data processing related operations based on the high-precision equivalent data in the register to obtain a data processing result in the second data type.

3. The method of claim 2, wherein, The performing data processing related operations based on the high-precision equivalent data in the register to obtain a data processing result in the second data type comprises: Storing the high-precision equivalent data from the register into the memory in the storage manner corresponding to the second data type as a data processing result in the second data type.

4. The method of claim 2, wherein, The performing data processing related operations based on the high-precision equivalent data in the register to obtain a data processing result in the second data type comprises: Describing the high-precision equivalent data as a combination of N low-precision equivalent data in the first data type; wherein N is an integer greater than or equal to 2; Performing first type calculation using N low-precision equivalent data to obtain N first intermediate calculation results in the first data type; Re-describing and storing N first intermediate calculation results in the first data type to obtain a data processing result in the second data type.

5. The method of claim 4, wherein, The re-describing and storing N first intermediate calculation results in the first data type to obtain a data processing result in the second data type comprises: Describing N first intermediate calculation results as a whole as a first calculation result in the second data type; Storing the first calculation result into the memory in the storage manner corresponding to the second data type to obtain a data processing result in the second data type.

6. The method of claim 1, wherein, The performing data processing related operations based on the low-precision data block in the manner corresponding to the second data type to obtain a data processing result in the second data type comprises: read the low-precision data block from the memory to the cache by using a loading mode corresponding to the second data type, to obtain a high-precision data block in the second data type; describe the high-precision data block in the cache as a low-precision data block in the first data type; read the low-precision data block from the cache to a register by using a loading mode corresponding to the first data type and perform a first type conversion, to obtain high-precision data in a third data type; perform a second type calculation and a second type conversion based on the high-precision data in the register, to obtain a second intermediate calculation result in the first data type; based on the second intermediate calculation result, perform a re-description and storage by taking the cache as a transit, to obtain a data processing result in the second data type.

7. The method of claim 6, wherein, The precision of the third data type is higher than the precision of the first data type, and the precision of the third data type is lower than the precision of the second data type.

8. The method of claim 6, wherein, The second type calculation is a high-precision calculation with a precision requirement; and the second type calculation and the second type conversion based on the high-precision data in the register to obtain a second intermediate calculation result in the first data type include: perform a high-precision calculation by using high-precision data in the third data type in the register, to obtain a high-precision calculation result in the third data type; convert the high-precision calculation result from the third data type to the first data type, to obtain a second intermediate calculation result in the first data type.

9. The method of claim 6, wherein, The re-description and storage of the second intermediate calculation result by taking the cache as a transit to obtain a data processing result in the second data type include: store the second intermediate calculation result from the register to the cache by using a storage mode corresponding to the first data type; re-describe the second intermediate calculation result as a second calculation result in the second data type; store the second calculation result from the cache to the memory by using a storage mode corresponding to the second data type, to obtain a data processing result in the second data type.

10. A data processing apparatus, characterized by, The apparatus includes: a data block description module configured to describe a low-precision data block in a first data type as a high-precision data block in a second data type in a case where an address range corresponding to the low-precision data block is discontinuous in a memory according to a memory access granularity, a data transmission mode, and a data arrangement mode; the precision of the second data type is higher than the precision of the first data type; a data processing operation module configured to perform a data processing related operation based on the low-precision data block by using a mode corresponding to the second data type, to obtain a data processing result in the second data type; the data processing related operation includes at least one of data transfer and data calculation; a result re-description module configured to re-describe the data processing result as low-precision result data in the first data type.

11. A chip, characterized by include: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the data processing method of any one of claims 1 to 9 when executing the instructions stored by the memory.

12. A computer device, comprising: An apparatus comprising a memory, an artificial intelligence chip, and a computer program stored on the memory and executable on the artificial intelligence chip, the artificial intelligence chip implementing the data processing method according to any one of claims 1 to 9 when executing the computer program.

Images