Data processing method and device, processor, electronic device and storage medium

Abstract

A data processing method and device, processor, electronic device and storage medium. The data processing method includes: performing a first operation multiple times to obtain an output tensor; wherein each time the first operation is performed, the following steps are performed: using a first thread bundle to load multiple sub-convolution kernel tensors from a memory to a cache component; using the first thread bundle to load multiple sub-activation tensors corresponding to the multiple sub-convolution kernel tensors from a memory to a cache component; using a second thread bundle to sequentially read each sub-convolution kernel tensor and the sub-activation tensor corresponding to the sub-convolution kernel tensor from the cache component to a tensor calculation unit, so that the tensor calculation unit performs a matrix multiplication and accumulation operation between each sub-convolution kernel tensor read and the corresponding sub-activation tensor; using the second thread bundle to accumulate the calculation results of multiple matrix multiplication and accumulation operations to obtain a sub-output tensor.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to a data processing method, a data processing apparatus, a processor, an electronic device, and a non-transitory computer-readable storage medium. Background Art

[0002] Convolution operation is one of the most widely used operators in the field of artificial intelligence, and it has good feature extraction and abstraction capabilities. The convolution operation formula can be expressed as Output = Conv(Weight, Activation), where Activation represents the activation tensor, and its tensor shape can be expressed as [N,H,W,C], where the N dimension is used to represent the number of batches, the W dimension is used to represent the width of the activation tensor, the H dimension is used to represent the height of the activation tensor, and the C dimension is used to represent the number of input channels. Weight represents the convolution kernel tensor, and its tensor shape can be expressed as [R,S,K,C], where the R dimension is used to represent the height of the convolution kernel, the S dimension is used to represent the width of the convolution kernel, the C dimension is used to represent the number of input channels, and the K dimension is used to represent the number of output channels. Output represents the output tensor, and its tensor shape can be expressed as [N,P,Q,K], where the N dimension is used to represent the number of batches, the P dimension is used to represent the height of the output tensor, the Q dimension is used to represent the width of the output tensor, and the K dimension is used to represent the number of output channels. Summary of the invention

[0003] At least one embodiment of the present disclosure provides a data processing method for loading an activation tensor and a convolution kernel tensor from a memory into a cache component of a processor, and using a thread warp to perform a convolution operation on the activation tensor and the convolution kernel tensor to obtain an output tensor. The data processing method comprises: performing a first operation multiple times to obtain the output tensor, wherein the output tensor is represented by [NPQ, K], P represents the height, Q represents the width, N represents the batch number, and K represents the number of output channels. The output tensor is divided into multiple sub-output tensors with a granularity of [m, n]. Each first operation is used to obtain a sub-output tensor, m and n are positive integers, and the output tensor is obtained by sequentially splicing the multiple sub-output tensors obtained by the multiple first operations; wherein each execution of the first operation comprises the following steps: using a first thread warp to load multiple sub-convolution kernel tensors from the memory into the cache component, wherein the convolution kernel tensor is represented by [R, S, K, C ], R represents the height of the convolution kernel, S represents the width of the convolution kernel, and C represents the number of input channels. The multiple sub-convolution kernel tensors are obtained by splitting the convolution kernel tensor according to the 1×a granularity on the RS plane composed of the convolution kernel height and the convolution kernel width. Each sub-convolution kernel tensor is represented by [1, a, m, C1], where a is a positive integer greater than 1 and less than or equal to S, and S is an integer multiple of a; using the first thread warp to load multiple sub-activation tensors corresponding to the multiple sub-convolution kernel tensors from the memory to the cache component , wherein each sub-activation tensor is a portion of the activation tensor required by the corresponding sub-convolution kernel tensor when performing the convolution operation; using the second thread bundle to sequentially read each sub-convolution kernel tensor and the sub-activation tensor corresponding to the sub-convolution kernel tensor from the cache component to the tensor calculation unit, so that the tensor calculation unit performs a matrix multiplication and accumulation operation between each sub-convolution kernel tensor read and the corresponding sub-activation tensor; using the second thread bundle to accumulate the calculation results of multiple matrix multiplication and accumulation operations to obtain the sub-output tensor.

[0004] For example, in the data processing method provided by at least one embodiment of the present disclosure, each convolution kernel element included in the sub-convolution kernel tensor reuses the corresponding sub-activation tensor loaded into the cache component when performing the matrix multiplication and accumulation operation.

[0005] For example, in the data processing method provided by at least one embodiment of the present disclosure, the activation tensor is represented as [N, H, W, C], W represents width, and H represents height, and the first thread warp is used to load multiple sub-activation tensors corresponding to the multiple sub-convolution kernel tensors from the memory to the cache component, including: for each sub-activation tensor, according to the sub-convolution kernel tensor corresponding to the sub-activation tensor, determining the first starting position of the sub-activation tensor in the first sequence, wherein the first sequence is obtained by expanding the activation tensor into one dimension in height dimension, width dimension and batch quantity dimension, and the first sequence is represented as [NHW, C]; starting from the first starting position, loading data with a shape size of [n+a-1, C1] as the sub-activation tensor to the cache component.

[0006] For example, in the data processing method provided by at least one embodiment of the present disclosure, a second thread warp is used to read each sub-convolution kernel tensor and the sub-activation tensor corresponding to the sub-convolution kernel tensor from the cache component in sequence to a tensor calculation unit, so that the tensor calculation unit performs a matrix multiplication and accumulation operation between each sub-convolution kernel tensor and the corresponding sub-activation tensor read, including: for each sub-convolution kernel tensor and the corresponding sub-activation tensor: use the second thread warp to load the sub-convolution kernel tensor to the tensor calculation unit; use the second thread warp to perform a second operation to obtain the calculation result of the matrix multiplication and accumulation operation between the sub-convolution kernel tensor and the corresponding sub-activation tensor; wherein each second operation is used to perform a convolution operation on the convolution kernel data with a shape size of [1,1,m,C1] in the sub-convolution kernel tensor, so as to obtain the calculation result through the a second operation.

[0007] For example, in the data processing method provided by at least one embodiment of the present disclosure, the second operation includes executing the following steps: using the second thread warp to load activation tensor data with a shape size of [n, C1] from the corresponding sub-activation tensor to the tensor computing unit; the tensor computing unit performs a matrix multiplication and accumulation operation between the convolution kernel data and the activation tensor data to obtain an operation result; adding the operation result to the current calculation result to update the calculation result, wherein the current calculation result is obtained by accumulating the operation results obtained by the second operations previously performed in the a second operations.

[0008] For example, in the data processing method provided by at least one embodiment of the present disclosure, the corresponding sub-activation tensor includes n+a-1 sequence units, each sequence unit includes C1 elements, and the second thread warp is used to load data with a shape size of [n, C1] from the corresponding sub-activation tensor to the tensor calculation unit, including: determining a second starting position according to the position of the [1, 1, m, C1] data used for the second operation in the sub-convolution kernel tensor; starting from the second starting position, sequentially reading n sequence units in the corresponding sub-activation tensor to the tensor calculation unit; wherein, during the reading process, the element value of the sequence unit at the filling position is modified to a filling default value and loaded into the tensor calculation unit according to the sequence unit index information and the filling information, and the filling position is indicated by the sequence unit index information and the filling information.

[0009] For example, the data processing method provided by at least one embodiment of the present disclosure further includes: enabling two second thread warps to use the tensor computing unit using a ping-pong strategy through synchronization operations, wherein the two second thread warps are used to obtain different sub-output tensors.

[0010] For example, the data processing method provided by at least one embodiment of the present disclosure also includes: calling a vector computing unit to perform further computing processing on the sub-output tensor; enabling two second thread warps to use the tensor computing unit using a ping-pong strategy through synchronization operations, including: in response to one of the two second thread warps completing the first operation, sending a synchronization signal to the other second thread warp to notify the other second thread warp to perform the first operation, and at the same time, the second thread warp calls the vector computing unit to perform the computing processing; in response to the other second thread warp completing the first operation, sending a synchronization signal to the second thread warp to notify the second thread warp to perform the first operation, and at the same time, the other second thread warp calls the vector computing unit to perform the computing processing.

[0011] For example, in the data processing method provided by at least one embodiment of the present disclosure, a first thread warp is used to load multiple sub-convolution kernel tensors from the memory to the cache component, including: using a tensor memory access acceleration instruction to load each sub-convolution kernel tensor to the cache component.

[0012] For example, the data processing method provided by at least one embodiment of the present disclosure also includes: copying the sub-output tensor to the cache component, and using a tensor memory access acceleration instruction to write the sub-output tensor to the memory, wherein, in the process of writing to the memory, the sub-output tensor is written to the corresponding position of the output tensor stored in the memory according to the positional relationship between the multiple sub-output tensors.

[0013] For example, in the data processing method provided in at least one embodiment of the present disclosure, a=3.

[0014] At least one embodiment of the present disclosure provides a processor, comprising an instruction parsing unit, an execution unit, a cache component and a memory, wherein the instruction parsing unit is used to receive and parse data processing instructions, wherein the data processing instructions are used to load activation tensors and convolution kernel tensors from the memory to the cache component, and use thread warps to perform convolution operations on the activation tensors and the convolution kernel tensors to obtain output tensors, and the execution unit executes the data processing method described in any embodiment of the present disclosure after the instruction parsing unit parses the data processing instructions.

[0015] For example, in the processor provided in at least one embodiment of the present disclosure, the execution unit includes a computing unit, and different computing units are configured to perform different first operations.

[0016] At least one embodiment of the present disclosure provides a data processing device, which is used to load an activation tensor and a convolution kernel tensor from a memory into a cache component of a processor, and use a thread warp to perform a convolution operation on the activation tensor and the convolution kernel tensor to obtain an output tensor. The data processing device includes an execution module, which is configured to execute multiple first operations to obtain the output tensor, wherein the output tensor is represented by [NPQ, K], P represents the height, Q represents the width, N represents the batch number, and K represents the number of output channels. The output tensor is divided into multiple sub-output tensors with a granularity of [m, n]. Each first operation is used to obtain a sub-output tensor, m, n are positive integers, and the output tensor is obtained by sequentially splicing the multiple sub-output tensors obtained by the multiple first operations; wherein the execution module includes a first loading module, a second loading module, a third loading module, a tensor calculation unit and an accumulation module, wherein the first loading module is configured to use a first thread warp to load multiple sub-convolution kernel tensors from the memory to the cache component, wherein the convolution kernel tensor is represented by [R, S, K, C ], R represents the height of the convolution kernel, S represents the width of the convolution kernel, and C represents the number of input channels. The multiple sub-convolution kernel tensors are obtained by splitting the convolution kernel tensor on the RS plane composed of the convolution kernel height and the convolution kernel width according to the 1×a granularity. Each sub-convolution kernel tensor is represented by [1, a, m, C1], a is a positive integer greater than 1 and less than or equal to S, and S is an integer multiple of a; the second loading module is configured to use the first thread warp to load multiple sub-activation tensors corresponding to the multiple sub-convolution kernel tensors from the memory to the cache component, wherein each The sub-activation tensor is the part of the activation tensor required by the corresponding sub-convolution kernel tensor when performing the convolution operation; the third loading module is configured to use the second thread beam to read each sub-convolution kernel tensor and the sub-activation tensor corresponding to the sub-convolution kernel tensor from the cache component in sequence to the tensor calculation unit; the tensor calculation unit is configured to perform a matrix multiplication and accumulation operation between each sub-convolution kernel tensor read and the corresponding sub-activation tensor; the accumulation module is configured to use the second thread beam to accumulate the calculation results of multiple matrix multiplication and accumulation operations to obtain the sub-output tensor.

[0017] At least one embodiment of the present disclosure provides an electronic device, comprising: a memory, which non-transitorily stores computer-executable instructions; and a processor, which is configured to execute the computer-executable instructions, wherein the computer-executable instructions, when executed by the processor, implement the data processing method according to any embodiment of the present disclosure.

[0018] At least one embodiment of the present disclosure provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the data processing method according to any embodiment of the present disclosure is implemented.

[0019] In the data processing method provided by at least one embodiment of the present disclosure, the convolution kernel tensor is split into multiple sub-convolution kernel tensors according to the granularity of 1×a, and each sub-convolution kernel tensor and the corresponding sub-activation tensor are loaded from the memory to the cache component. When performing a convolution operation on the sub-convolution kernel and the corresponding sub-activation tensor, the sub-convolution kernel tensor and the corresponding sub-activation tensor are read from the cache component to the tensor calculation unit, so that when the convolution operation is specifically performed, each convolution kernel element in the sub-convolution kernel tensor can reuse the sub-activation tensor, thereby improving the computing intensity, reducing the memory access overhead of the global cache, the secondary cache, and the memory, and reducing and avoiding the repeated loading of the activation tensor. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, but are not intended to limit the present disclosure.

[0021] Figure 1 A schematic structural diagram of a general-purpose graphics processing unit (GPGPU);

[0022] Figure 2 A schematic diagram showing the use of implicit GEMM to implement convolution;

[0023] Figure 3 A schematic flow chart of a data processing method provided for at least one embodiment of the present disclosure;

[0024] Figure 4A is a schematic diagram of an activation tensor;

[0025] Figure 4B A schematic diagram of a convolution kernel tensor;

[0026] Figure 5 A schematic diagram of an output tensor provided for an embodiment of the present disclosure;

[0027] Figure 6 A schematic flow chart of a first operation provided for at least one embodiment of the present disclosure;

[0028] Figure 7 An illustration of a convolution kernel tensor provided for at least one embodiment of the present disclosure;

[0029] Figure 8 A schematic diagram of a process of loading a sub-activation tensor provided in an embodiment of the present disclosure;

[0030] Fig. 9 A schematic diagram of a ping-pong strategy process provided by an embodiment of the present disclosure;

[0031] Fig.10 A schematic diagram of a process for loading activation tensor data provided by an embodiment of the present disclosure;

[0032] Fig.11 A schematic block diagram of a data processing device provided for at least one embodiment of the present disclosure;

[0033] Fig.12 A schematic structural diagram of a processor provided for at least one embodiment of the present disclosure;

[0034] Fig.13 A schematic block diagram of an electronic device provided in accordance with an embodiment of the present disclosure;

[0035] Fig.14 A schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0037] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits the detailed description of some known functions and known components.

[0038] Figure 1A schematic structural diagram of a general-purpose graphics processing unit (GPGPU).

[0039] like Figure 1 As shown, the general-purpose graphics processor is actually an array of programmable multiprocessors. For example, the programmable multiprocessor can be a streaming processor cluster (SPC), for example, including Figure 1 The stream processor clusters 1, ..., and stream processor cluster M are shown, where M is a positive integer greater than 1. In a general-purpose graphics processor, one stream processor cluster processes one computing task, or multiple stream processor clusters process one computing task. Multiple stream processor clusters share data through a global cache or a global memory.

[0040] like Figure 1 As shown, taking stream processor cluster 1 as an example, one stream processor cluster includes multiple computing units, such as Figure 1 In the figure, there are computing units 1, 2, ..., N, where N is a positive integer. Each computing unit (CU) is used to perform arithmetic and logical operations, such as accumulation, reduction, conventional addition, subtraction, multiplication, and division. A computing unit includes multiple cores (also called computing cores or computing cores), each of which includes an arithmetic logic unit (ALU), a floating-point computing unit, etc. The computing core is used to perform specific computing tasks. In addition, the computing unit also includes registers (e.g. Figure 1 The register file in the computing unit and the shared memory are used to hierarchically store source data and destination data related to computing tasks. The shared memory in a computing unit is used to share data between the cores of the computing unit.

[0041] In parallel computing, computing tasks are generally performed by multiple threads. These threads are divided into multiple thread blocks before being executed in a general-purpose graphics processor (or parallel computing processor), and then distributed through the thread block distribution module ( Figure 1 The CPU (not shown) distributes multiple thread blocks to each computing unit. All threads in a thread block must be assigned to the same computing unit for execution. At the same time, the thread block is split into minimum execution thread bundles (or simply thread bundles, warps), each of which contains a fixed number of threads (or less than this fixed number), for example, 32 threads. Multiple thread blocks can be executed in the same computing unit or in different computing units.

[0042] In each compute unit, the warp scheduling / dispatching module ( Figure 1The thread warps are scheduled and allocated (not shown) so that multiple computing cores of the computing unit can run the thread warps. Depending on the number of computing cores in the computing unit, multiple thread warps in a thread block can be executed simultaneously or in time-sharing. Multiple threads in each thread warp will execute the same instruction. The memory execution instruction will be emitted to the shared memory in the computing unit or further emitted to the intermediate level cache or global cache or global memory for read and write operations, etc.

[0043] The current convolution operation is usually implemented using the Implicit GEMM (Generalized Matrix-MatrixMultiply) algorithm. The Implicit GEMM algorithm expresses the convolution operation as a generalized matrix-matrix multiplication (GEMM). During the convolution process, the activation tensor is acted upon by a sliding filter (or convolution kernel) to produce an output tensor. The Implicit GEMM algorithm avoids explicitly converting the activation tensor and convolution kernel into matrix form during the calculation process by means of coordinate mapping, thereby saving storage space.

[0044] Figure 2 Figure 2 shows a schematic diagram of using implicit GEMM to implement convolution. Figure 2 As shown in the figure, the response convolution operation is to perform sliding window dot product calculation on the input N input feature maps of size C×H×W with K convolution kernels of size C×R×S, and obtain N output feature maps of size K×P×Q. The implicit GEMM algorithm regards the convolution kernel as a two-dimensional matrix of size K×(C×R×S), and the input feature map is transformed into a two-dimensional matrix of size (C×R×S)×(N×H×W) according to the position of the convolution kernel sliding window, and then the two matrices are calculated by matrix multiplication to obtain the output feature map.

[0045] However, in this way, during the conversion process, the activation tensor changes from [N, H, W, C] to [NHW, RSC], and the amount of data is expanded by R×S times. In the ImplicitGemm algorithm, these data volume expansions basically occur in the global cache or the second-level cache, and the operator performance is easily limited to the memory access bandwidth of the second-level cache or the global cache. In addition, this algorithm is implemented element by element. For example, when calculating the convolution operation result corresponding to the first element, the input tensor part and the convolution kernel tensor corresponding to the first element are loaded, and the convolution operation of the first element is performed; when calculating the convolution operation result corresponding to the second element, the input tensor part and the convolution kernel tensor corresponding to the second element are loaded, and the convolution operation of the second element is performed, and so on. Due to the characteristics of the convolution operation, this element-by-element loading method will also lead to repeated loading of a lot of data, wasting storage space and memory access bandwidth.

[0046] At least one embodiment of the present disclosure provides a data processing method, a processor, an electronic device, and a non-transitory computer-readable storage medium. The data processing method is used to load an activation tensor and a convolution kernel tensor from a memory into a cache component of a processor, and use a thread warp to perform a convolution operation on the activation tensor and the convolution kernel tensor to obtain an output tensor. The data processing method includes: performing multiple first operations to obtain an output tensor, wherein the output tensor is expressed as [NPQ, K], P represents height, Q represents width, N represents batch number, K represents output channel number, the output tensor is divided into multiple sub-output tensors with [m, n] as granularity, each first operation is used to obtain a sub-output tensor, m, n are positive integers, and the output tensor is obtained by sequentially splicing multiple sub-output tensors obtained by multiple first operations; wherein each execution of the first operation includes performing the following steps: using the first thread warp to load multiple sub-convolution kernel tensors from the memory to the cache component, wherein the convolution kernel tensor is expressed as [R, S, K, C], R represents convolution kernel height, S represents convolution kernel width, C represents input channel number, multiple sub-convolution kernel tensors are obtained by splitting the convolution kernel tensor on the RS plane composed of convolution kernel height and convolution kernel width according to 1×a granularity, and each sub-convolution kernel tensor is expressed as [1, a, m, C1], a is a positive integer greater than 1 and less than S, where S is an integer multiple of a; using the first thread bundle to load multiple sub-activation tensors corresponding to the multiple sub-convolution kernel tensors from the memory to the cache component, wherein each sub-activation tensor is a part of the activation tensor required by the corresponding sub-convolution kernel tensor when performing a convolution operation; using the second thread bundle to sequentially read each sub-convolution kernel tensor and the sub-activation tensor corresponding to the sub-convolution kernel tensor from the cache component to the tensor calculation unit, so that the tensor calculation unit performs a matrix multiplication and accumulation operation between each sub-convolution kernel tensor read and the corresponding sub-activation tensor; using the second thread bundle to accumulate the calculation results of multiple matrix multiplication and accumulation operations to obtain a sub-output tensor.

[0047] In the data processing method provided by at least one embodiment of the present disclosure, the convolution kernel tensor is split into multiple sub-convolution kernel tensors according to the granularity of 1×a, and each sub-convolution kernel tensor and the corresponding sub-activation tensor are loaded from the memory to the cache component. When performing a convolution operation on the sub-convolution kernel and the corresponding sub-activation tensor, the sub-convolution kernel tensor and the corresponding sub-activation tensor are read from the cache component to the tensor calculation unit, so that when the convolution operation is specifically performed, each convolution kernel element in the sub-convolution kernel tensor can reuse the sub-activation tensor, thereby improving the computing intensity, reducing the memory access overhead of the global cache, the secondary cache, and the memory, and reducing and avoiding the repeated loading of the activation tensor.

[0048] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings, but the present disclosure is not limited to these specific embodiments.

[0049] Figure 3A schematic flowchart of a data processing method provided for at least one embodiment of the present disclosure.

[0050] The data processing method provided by at least one embodiment of the present disclosure is used to load activation tensors and convolution kernel tensors from memory into a cache component of a processor, and use a thread bundle to perform a convolution operation on the activation tensor and the convolution kernel tensor to obtain an output tensor.

[0051] For example, the cache component is used to cache the activation tensor and the convolution kernel tensor, and send the activation tensor and the convolution kernel tensor to the tensor calculation unit when the calculation core performs a convolution operation on the activation tensor and the convolution kernel tensor. Figure 1 Shared memory in .

[0052] For example, the activation tensor is expressed as [N,H,W,C], where N represents the batch size, W represents the width, H represents the height, and C represents the number of input channels. The convolution kernel tensor is expressed as [R,S,K,C], where R represents the height of the convolution kernel, S represents the width of the convolution kernel, K represents the number of output channels, and C represents the number of input channels.

[0053] Figure 4A A schematic diagram of an activation tensor.

[0054] like Figure 4A As shown, the input gradient tensor shows a batch, but the input gradient tensor for the actual convolution operation may have multiple batches, and the dimensions of multiple batches are N dimensions. The structure of each batch is the same as Figure 4A Same as shown.

[0055] like Figure 4A As shown, a batch consists of 21×6×8 elements, W=21, H=6, C=8, Figure 4A A small cube in a batch represents an element. For example, an element can have a data width of 8 bits, 16 bits, etc.

[0056] For example, in Figure 4A The H dimension, W dimension, and C dimension are shown in . Assume that the element pointed by the arrow is the origin, C, W, and H represent the three coordinate axes, row is the coordinate value in the height H direction (also the column direction), col is the coordinate value in the width W direction (also the row direction), and c is the coordinate value in the channel C direction (also the depth direction). The coordinates of the element pointed by the arrow in the input data are: c=0, row=0, and col=0. The channel, width, and height values ​​in the coordinates of other elements increase along the direction of the arrow.

[0057] Figure 4B A schematic diagram of a convolution kernel tensor.

[0058] like Figure 4B As shown in the figure, in the back propagation operation of the strided convolution, the convolution kernel tensor shows a batch. Similarly, the convolution kernel tensor for the actual convolution operation may have multiple batches, and the dimensions of multiple batches are C dimensions. The structure of each batch is the same as Figure 4B Same as shown.

[0059] like Figure 4B As shown in the figure, a batch consists of 3×3×7 elements, R=3, S=3, C=7, and the plane formed by the R dimension and the S dimension is a 3×3 convolution kernel. Figure 4B A small cube in a batch represents an element. For example, an element can have a data width of 8 bits, 16 bits, etc.

[0060] For example, in Figure 4B The R dimension, S dimension, and C dimension are shown in . Similarly, assuming that the element pointed by the arrow is the origin, R, S, and C represent the three coordinate axes, row is the coordinate value in the height R direction, col is the coordinate value in the width S direction, and c is the coordinate value in the input channel C direction. The coordinates of the element pointed by the arrow in the input data are: c=0, row=0, and col=0. The channel, width, and height values ​​in the coordinates of other elements increase along the direction of the arrow.

[0061] The output tensor is expressed as [NPQ, K], where P represents the height and Q represents the width. For example, here NPQ means that the output tensor [N, P, Q, K] is expanded into one dimension in N, P and Q dimensions, that is, the output tensor is expanded first in Q dimension, then in P dimension, and finally in N dimension. The expanded output tensor can be expressed as NPQ (pixel) × K, where NPQ (pixel) means that it is expanded into one dimension in P dimension, Q dimension and N dimension.

[0062] like Figure 3 As shown, step S10 may include: performing multiple first operations to obtain an output tensor, wherein the output tensor is divided into multiple sub-output tensors with a granularity of [m, n], each first operation is used to obtain a sub-output tensor, m, n are positive integers, and the output tensor is obtained by sequentially splicing multiple sub-output tensors obtained by multiple first operations.

[0063] For example, when using thread warps to perform convolution operations, if the size of the input data is large, a large image can be divided into multiple small blocks, each of which is called a "tile". Each tile contains a portion of the image data, which can be processed in an independent processing unit.

[0064] For example, the output tensor is divided into multiple sub-output tensors with a granularity of [m, n], and each sub-output tensor is regarded as a "tile". The specific values ​​of m and n can be determined according to actual needs, for example, according to the tile size supported by the processor, the processing speed of the processor, load balancing, data type, etc. For example, in one example, m=32, n=64, of course, the present disclosure is not limited to this.

[0065] Figure 5 A schematic diagram of an output tensor provided for an embodiment of the present disclosure.

[0066] like Figure 5 As shown, batch 0 and batch 1 are shown. Of course, the output tensor can also include more batches. Figure 5 The output tensor is shown to be flat, showing the NPQ dimensions, but in reality has K elements for each box position in the depth direction.

[0067] Figure 5 The white box in the middle represents a sub-output tensor 1, the gray box represents a sub-output tensor 2, and the black box represents a sub-output tensor 3. Sub-output tensor 1, sub-output tensor 2, and sub-output tensor 3 are sequentially concatenated to form the output tensor.

[0068] For example, the convolution operation can be regarded as a general matrix multiplication operation (GEMM), where gemm_M=K, gemm_N=NPQ. The processor can be divided into multiple computing units from the tensor computing unit (such as TensorCore) scheduling level. Using the block idea (dividing a large data set into small blocks, each computing unit can process a piece of data independently), the output tensor is evenly divided into (m,n) granularity to each computing unit for computing tasks. Different computing units execute in parallel the computing tasks for obtaining different sub-output tensors, thereby accelerating the process of convolution operations.

[0069] For example, each first operation is used to obtain a sub-output tensor. The specific execution process of the first operation is described in detail below with reference to the accompanying drawings.

[0070] Figure 6 A schematic flow chart of a first operation provided for at least one embodiment of the present disclosure. Figure 6 As shown, executing the first operation includes executing steps S20-S50.

[0071] For example, steps S20-S50 may be executed sequentially or in other adjusted orders. The embodiments of the present disclosure do not limit the execution order of the steps and may be adjusted according to actual conditions. For example, in some examples, the data processing method provided by at least one embodiment of the present disclosure may selectively execute some steps in steps S20-S50, or may execute some additional steps in addition to steps S20-S50. The embodiments of the present disclosure do not specifically limit this.

[0072] For example, in step S20, a first thread warp is used to load multiple sub-convolution kernel tensors from a memory to a cache component.

[0073] For example, the first warp is a producer warpgroup dedicated to data loading.

[0074] For example, the convolution kernel tensor is expressed as [R, S, K, C], where R represents the height of the convolution kernel, S represents the width of the convolution kernel, C represents the number of input channels, and K represents the number of output channels.

[0075] The multiple sub-convolution kernel tensors include, for example, sub-convolution kernel tensor A0, sub-convolution kernel tensor A1, sub-convolution kernel tensor A2, etc. The multiple sub-convolution kernel tensors are obtained by splitting the convolution kernel tensor on the RS plane composed of the convolution kernel height and the convolution kernel width according to the granularity of 1×a. Each sub-convolution kernel tensor is expressed as [1, a, m, C1], where a is a positive integer greater than 1 and less than or equal to S, and S is an integer multiple of a.

[0076] For example, the value range of a may be greater than or equal to 2 and less than or equal to 8. The specific a may be determined according to S and the hardware conditions of the processor.

[0077] Considering the storage resources and computing power of current processors, setting a=3 can achieve the best balance between storage performance and computing efficiency.

[0078] For example, the sub-convolution kernel tensor is loaded into the cache unit using the Tensor Memory Accelerator (TMA) instruction. TMA is a specialized unit dedicated to tensor type data that speeds up the loading process.

[0079] Figure 7 A schematic diagram of a convolution kernel tensor provided in at least one embodiment of the present disclosure. Figure 7 In the example, R=3, S=3, a=3, but the present disclosure is not limited thereto. Figure 7 This is just an example.

[0080] like Figure 7 The convolution kernel tensor shown is represented as [3, 3, K, C], where K and C are positive integers greater than 1. Figure 7 For example, the convolution kernel tensor is split into three sub-convolution kernel tensors at a 1×3 granularity on the RS plane, which are Figure 7 The sub-convolution kernel tensor composed of black boxes, the sub-convolution kernel tensor composed of gray boxes, and the sub-convolution kernel tensor composed of white boxes.

[0081] For example, each sub-convolution kernel tensor can be expressed as [1, 3, m, C1], where m is the size of the aforementioned sub-output tensor, and C1 can be set according to computing power or data type, for example, C1 can be equal to 64, 32, 128, etc., and the present disclosure does not impose specific restrictions on this. C1 can be less than or equal to C.

[0082] For example, in step S30, a plurality of sub-activation tensors corresponding one-to-one to a plurality of sub-convolution kernel tensors are loaded from a memory to a cache component using a first thread warp.

[0083] For example, each sub-activation tensor is the portion of the activation tensor that the corresponding sub-convolution kernel tensor needs when performing a convolution operation.

[0084] For example, the activation tensor is represented as [N, H, W, C], where W represents width and H represents height. Step S30 may include: for each sub-activation tensor, according to the sub-convolution kernel tensor corresponding to the sub-activation tensor, determining a first starting position of the sub-activation tensor in a first sequence, wherein the first sequence is obtained by expanding the activation tensor into one dimension in height dimension, width dimension and batch number dimension, and the first sequence is represented as [NHW, C]; starting from the first starting position, loading data with a shape size of [n+a-1, C1] as a sub-activation tensor to a cache component.

[0085] For example, each sub-activation tensor is loaded into a cache component using a tensor memory access acceleration instruction.

[0086] Figure 8 A schematic diagram of the process of loading a sub-activation tensor provided in an embodiment of the present disclosure.

[0087] Figure 8 The NHW dimensions of the activation tensor are shown in , but like the output tensor, each box contains C elements in the depth direction. For example, Figure 8 Each block shown in may be referred to as a sequence unit, and each sequence unit includes C elements in a depth direction (C direction).

[0088] like Figure 8 As shown, the first warp uses the TMA instruction to copy the sub-activation tensor B0 required by the sub-convolution kernel tensor A0 to the cache component. The sub-convolution kernel tensor A0, for example, includes Figure 8 The convolution kernel tensors shown in are w00, w01, and w02 on the RS plane.

[0089] After determining the first starting position corresponding to the sub-convolution kernel tensor A0, n+a-1 sequence units in the first sequence are continuously loaded from the first starting position as the sub-activation tensor B0 corresponding to the sub-convolution kernel tensor A0. It should be noted that since each convolution kernel element in the sub-convolution kernel tensor A0 needs to complete the convolution operation, it is necessary to load n+a-1 sequence units to complete the convolution operation of all convolution kernel elements in the sub-convolution kernel tensor A0.

[0090] For example, in step S40, a second thread warp is used to sequentially read each sub-convolution kernel tensor and the sub-activation tensor corresponding to the sub-convolution kernel tensor from the cache component to a tensor computing unit, so that the tensor computing unit performs a matrix multiplication and accumulation operation between each sub-convolution kernel tensor read and the corresponding sub-activation tensor.

[0091] For example, the second warp is a consumer warpgroup, which is dedicated to data calculation.

[0092] For example, when performing matrix multiplication and accumulation operations on the convolution kernel elements included in the sub-convolution kernel tensor, the corresponding sub-activation tensor loaded into the cache component is reused. As a result, the sub-activation tensor cached in the cache component can be reused a times, improving the computational intensity and reducing the memory access overhead of the global cache, the secondary cache, etc.

[0093] For example, step S40 may include: for each sub-convolution kernel tensor and the corresponding sub-activation tensor: using the second thread warp to load the sub-convolution kernel tensor to the tensor computing unit; using the second thread warp to perform a second operation to obtain the calculation result of the matrix multiplication and accumulation operation between the sub-convolution kernel tensor and the corresponding sub-activation tensor.

[0094] For example, each second operation is used to perform a convolution operation on convolution kernel data having a shape size of [1, 1, m, C1] in the sub-convolution kernel tensor to obtain a calculation result through a second operations.

[0095] For example, in one embodiment, a=3, and the second operation is performed three times to obtain the convolution operation between the sub-convolution kernel tensor A0 and the corresponding sub-activation tensor B0. Taking the RS plane as an example, for example, the first second operation is used to perform a convolution operation including the convolution kernel data at the position w00, the second second operation is used to perform a convolution operation including the convolution kernel data at the position w01, and the third second operation is used to perform a convolution operation including the convolution kernel data at the position w02. The results of the three second operations are accumulated to obtain the calculation result A0×B0 between the sub-convolution kernel tensor A0 and the corresponding sub-activation tensor B0.

[0096] For example, the second operation includes performing the following steps: using the second thread warp to load activation tensor data with a shape size of [n, C1] from the corresponding sub-activation tensor to the tensor computing unit; the tensor computing unit performs a matrix multiplication and accumulation operation between the convolution kernel data and the activation tensor data to obtain an operation result; adding the operation result to the current calculation result to update the calculation result, wherein the current calculation result is obtained by accumulating the operation results obtained by the second operation previously performed in a second operation.

[0097] For example, the corresponding sub-activation tensor includes n+a-1 sequence units, each of which includes C1 elements. Using the second thread warp to load data of shape size [n, C1] from the corresponding sub-activation tensor to the tensor calculation unit may include: determining a second starting position according to the position of the [1, 1, m, C1] data in the sub-convolution kernel tensor used for the second operation; starting from the second starting position, sequentially reading n sequence units in the corresponding sub-activation tensor to the tensor calculation unit; wherein, during the reading process, according to the sequence unit index information and the padding information, the element value of the sequence unit at the padding position is modified to the padding default value and then loaded into the tensor calculation unit, and the padding position is indicated by the sequence unit index information and the padding information.

[0098] Therefore, the data at the original convolution filling position is processed by mask operation to the filling default value (default 0), and then loaded into the tensor calculation unit, so that the filling process can be simulated during the loading process, and the sub-activation tensor can still be loaded through the automatic loading process and the filling process is automatically simulated, which speeds up the loading process.

[0099] For the specific process of the second operation, please refer to the relevant description in the specific embodiments below.

[0100] For example, in step S50, the calculation results of multiple matrix multiplication and accumulation operations are accumulated using the second thread warp to obtain a sub-output tensor.

[0101] For example, the convolution kernel tensor can be divided into sub-convolution kernel tensors A0, ..., sub-convolution kernel tensor Ap, and correspond to sub-activation tensors B0, ..., sub-activation tensor Bp respectively. After step S40, calculation results A0×B0, ..., calculation results Ap×Bp are obtained, and then these calculation results are accumulated to obtain sub-output tensors A0×B0+...+Ap×Bp.

[0102] For example, after obtaining the sub-output tensor, the sub-output tensor is copied to the cache component, and the sub-output tensor is written to the memory using the tensor memory access acceleration instruction.

[0103] For example, in the process of writing to memory, the sub-output tensors are written to the corresponding positions of the output tensors stored in memory according to the positional relationship between the multiple sub-output tensors. Figure 5 Example shown.

[0104] For example, the main operation completed by the second thread warp, in addition to calling the vector computing unit to perform convolution calculations, also needs to call the vector computing unit (Vector Engine) to perform further calculations (such as bias, addition, etc.) on the sub-output tensor. In order to fully parallelize the computing resources of the vector computing unit and the tensor computing unit, the data processing method provided by at least one embodiment of the present disclosure also includes: enabling two second thread warps to use the tensor computing unit using a ping-pong strategy through synchronization operations, wherein the two second thread warps are used to obtain different sub-output tensors.

[0105] From a certain point in time, the two second thread warps respectively call the vector computing unit and the tensor computing unit to perform convolution calculations and further computational processing, making full use of the powerful computing power of the tensor computing unit to hide the time consumption of the vector computing unit.

[0106] For example, enabling two second thread warps to use the tensor computing unit using a ping-pong strategy through synchronization operations may include: in response to one of the two second thread warps completing the first operation, sending a synchronization signal to the other second thread warp to notify the other second thread warp to perform the first operation, and at the same time, the second thread warp calls the vector computing unit to perform calculation processing; in response to the other second thread warp completing the first operation, sending a synchronization signal to the second thread warp to notify the second thread warp to perform the first operation, and at the same time, the other second thread warp calls the vector computing unit to perform calculation processing.

[0107] Fig. 9 A schematic diagram of a ping-pong strategy process provided by an embodiment of the present disclosure.

[0108] Fig. 9 Two second warps are shown: second warp 1 and second warp 2. The main operation includes the aforementioned first operation, and the main operation requires the use of a tensor computing unit to obtain a sub-output tensor [m, n]. The computing process is to perform further non-matrix multiplication and accumulation operations such as bias and addition on the sub-output tensor [m, n]. The computing process requires the use of a vector computing unit to perform related operations.

[0109] In order to fully parallelize the computing resources of the tensor computing unit and the vector computing unit, the two second warps are started to use the tensor computing unit and the vector computing unit in a ping-pong strategy through synchronization operation.

[0110] For example, Fig. 9 As shown, the second warp 1 uses the tensor computing unit to perform the main operation, and after obtaining the sub-output tensor, sends a synchronization signal to the second warp 2. Afterwards, the second warp 1 uses the vector computing unit to perform calculation processing, and the second warp 2 starts to execute related main operations after receiving the synchronization signal, such as calling the tensor computing unit to perform matrix multiplication and accumulation operations, and the scheduling right of the tensor computing unit is transferred from the second warp 1 to the second warp 2.

[0111] Afterwards, when the second thread warp 2 obtains the sub-output tensor, it sends a synchronization signal to the second thread warp 1. The second thread warp 2 uses the vector computing unit to calculate the sub-output tensor, and the second thread warp 1 starts to execute the relevant main operations after receiving the synchronization signal. The scheduling right of the tensor computing unit is transferred from the second thread warp 2 to the second thread warp 1.

[0112] The subsequent process is similar. At a certain point in time, the second thread bundle 1 and the second thread 2 respectively call the tensor computing unit and the vector computing unit to perform main operations and calculations. In this way, the powerful computing power of the tensor computing unit can be used to hide the time consumption of the vector computing unit, thereby reducing time consumption and improving computing efficiency.

[0113] The following describes in detail the execution process of the data processing method provided by at least one embodiment of the present disclosure in conjunction with the accompanying drawings.

[0114] For example, in one embodiment, the shape size of the convolution kernel tensor is (3, 3, 256, 512), a=3, the shape size of the activation tensor is (2, 12, 7, 512), and the shape size of the output tensor is (2, 12, 7, 256). The step size parameter is (1, 1), which represents the sliding step size of the H / W dimension convolution kernel, the expansion parameter is (1, 1), which represents the H / W dimension convolution kernel expansion coefficient, and the padding parameter is (1, 1), which represents the H / W dimension padding size.

[0115] Assume that the output tensor is divided into (32, 64) granularity, and each first operation is used to obtain a sub-output tensor with a shape size of (32, 64). Taking a graphics processor as an example, different computing units can perform different first operations to obtain different sub-output tensors.

[0116] Specifically, each execution of the first operation includes the following process.

[0117] Referring to step S20, a TMA instruction is used to load the sub-convolution kernel tensor A0 etc. from a memory (eg, a secondary cache or a global memory) to a cache component (eg, a shared memory).

[0118] For example, the shape size of each sub-convolution kernel tensor is [1,3,32,128]. For example, the elements in the sub-convolution kernel tensor A0 include the first 32 batches in the convolution kernel tensor. Figure 7 The elements at the positions of the black boxes shown in the figure have C-direction coordinates from 0 to 127. The sub-convolution kernel tensor A1 includes, for example, the first 32 batches in the convolution kernel tensor. Figure 7 The elements at the gray box position shown, and the C direction coordinates are 0 to 127. The sub-convolution kernel tensor A2 includes, for example, the first 32 batches in the convolution kernel tensor Figure 7 The elements at the white box position shown, and the C direction coordinates are 0 to 127. The sub-convolution kernel tensor A3 includes, for example, the first 32 batches in the convolution kernel tensor Figure 7 The element at the position of the black box shown, and the C direction coordinate is 128 to 255, and so on.

[0119] Referring to step S30, use the TMA instruction to load the sub-activation tensor B0 corresponding to each sub-convolution kernel tensor from the memory to the cache component.

[0120] For example, each subactivation tensor has shape [66, 128], with size 66 in the NHW dimension and size 128 in the C dimension.

[0121] For the specific process, please refer to the relevant description in the aforementioned step S30, which will not be repeated here.

[0122] Referring to step S40, taking the sub-convolution kernel tensor A0 and the sub-activation tensor B0 as an example, the specific process of obtaining the calculation results of the sub-convolution tensor A0 and the sub-activation tensor B0 is specifically described.

[0123] First, load the subconvolution kernel tensor A0 to the tensor computation unit.

[0124] Then, the second operation is performed three times using the second thread warp to obtain the calculation results A0×B0 of the sub-convolution kernel tensor A0 and the sub-activation tensor B0.

[0125] In each second operation, a convolution operation of convolution kernel data with a shape size of [1, 1, 32, 128] is performed. For example, for the sub-convolution kernel tensor A0, viewed on the RS plane, the first second operation may be a convolution operation of convolution kernel data 1 including w00, the second second operation may be a convolution operation of convolution kernel data 2 including w01, and the third second operation may be a convolution operation of convolution kernel data 3 including w02. Of course, this order is only a schematic, and the present disclosure does not limit the order of a second operation.

[0126] Taking the convolution operation of convolution kernel data 1 as an example, in the second operation, the second thread warp is used to load the activation tensor data 1 with a shape size of [64,128] from the sub-activation tensor B0 to the tensor calculation unit; then, the tensor calculation unit performs the matrix multiplication and accumulation operation of the convolution kernel data 1 and the activation tensor data 1 to obtain the operation result; the operation result is used as the calculation result.

[0127] Afterwards, in the second operation, the second thread warp is used to load the activation tensor data 2 with a shape size of [64,128] from the sub-activation tensor B0 to the tensor calculation unit; then, the tensor calculation unit performs a matrix multiplication and accumulation operation on the convolution kernel data 2 and the activation tensor data 2 to obtain the operation result; the operation result is accumulated with the current data of the calculation result, and the accumulated result is used as the updated calculation result.

[0128] Afterwards, in the second operation, the second thread warp is used to load the activation tensor data 3 with a shape size of [64,128] from the sub-activation tensor B0 to the tensor calculation unit; then, the tensor calculation unit performs a matrix multiplication and accumulation operation of the convolution kernel data 3 and the activation tensor data 3 to obtain the operation result; the operation result is accumulated with the current data of the calculation result, and the accumulated result is used as the updated calculation result. The calculation result at this time is the calculation result A0×B0 of the sub-convolution kernel tensor A0 and the sub-activation tensor B0.

[0129] It should be noted that in the process of loading the activation tensor data, different activation tensor data have different starting loading positions in the sub-activation tensor data, which is related to the position of the convolution kernel data in the sub-convolution kernel tensor.

[0130] Fig.10 A schematic diagram of a process for loading activation tensor data provided by an embodiment of the present disclosure.

[0131] like Fig.10 As shown in the figure, for the sub-activation tensor B0, when loading the activation tensor data 1, the second starting position is p0, and 64 sequence units are loaded to the tensor calculation unit starting from p0. And, during the loading process, according to the mask position indicated by the sequence unit index information and the fill information, Fig.10 The sequence unit marked as mask on the right side is modified to fill the default value (such as 0) through the mask operation and loaded into the tensor calculation unit, which is equivalent to the state after the padding processing is performed, that is, the state with the padding mark on the left side.

[0132] like Fig.10 As shown, for the sub-activation tensor B0, when loading the activation tensor data 2, the second starting position is p1, and 64 sequence units are loaded to the tensor calculation unit starting from p1.

[0133] like Fig.10 As shown in the figure, for the sub-activation tensor B0, when loading the activation tensor data 3, the second starting position is p2, and 64 sequence units are loaded to the tensor calculation unit starting from p2. In addition, during the loading process, according to the mask position indicated by the sequence unit index information and the fill information, Fig.10 The sequence unit marked as mask on the left is modified to fill the default value (such as 0) through the mask operation and loaded into the tensor calculation unit, which is equivalent to the state after the filling process is performed, that is, the state with padding mark on the right.

[0134] In this embodiment, it can be seen that the sub-activation tensors cached in the cache component can be reused multiple times when loaded into the tensor computing unit, without the need to load them once each time a matrix multiplication and accumulation operation is performed, thereby improving computing intensity, reducing memory access requirements for memory, secondary cache, etc., reducing and avoiding operator blocking on memory access bandwidth to memory, secondary cache, etc., improving operator performance, and reducing repeated loading.

[0135] For other sub-activation tensors and corresponding sub-convolution kernel tensors, such as sub-convolution kernel tensor A1 and sub-activation tensor B1, the process of obtaining their calculation results is similar and will not be repeated here.

[0136] For example, the calculation result can be cached in a register unit, such as a thread-local register (TLR), and accumulated in the register unit. Alternatively, an accumulator can be used, which is not specifically limited in the present disclosure.

[0137] For example, the calculation results of multiple sub-convolution kernel tensors and corresponding sub-activation tensors obtained through the above steps are accumulated to obtain a sub-output tensor = A0×B0+A1×B1+...

[0138] The method for obtaining other sub-output tensors is similar to the above process and will not be repeated here.

[0139] After obtaining the sub-output tensor, the vector computing unit can be used to further calculate and process the sub-output tensor. For example, when a second thread warp is performing calculation processing, another second thread warp calls the vector computing unit to perform the first operation of the next block of data, and uses the ping-pong strategy to call the computing power of the vector computing unit and the tensor computing unit at the same time, using the powerful computing power of the tensor computing unit to hide the time consumption of the vector computing unit.

[0140] The sub-output tensors are written into the memory through the TMA instruction and concatenated in order according to the position relationship to obtain the output tensor.

[0141] In the data processing method provided by at least one embodiment of the present disclosure, the convolution kernel tensor is split according to the granularity of 1×a, and the activation tensor can be reused a times in the shared memory, thereby improving the computing intensity, reducing the memory access overhead of the global memory, the secondary cache, and the memory, and reducing and avoiding the repeated loading of the activation tensor. In addition, two second thread warps are used to call the tensor computing unit and the vector computing unit in parallel using a ping-pong strategy, giving full play to the parallel computing resources of the tensor computing unit and the vector computing unit, and using the powerful computing power of the tensor computing unit to hide the time consumption of the vector computing unit, thereby improving the computing efficiency, reducing the computing time consumption, and improving the operator performance.

[0142] At least one embodiment of the present disclosure further provides a data processing device. Fig.11 A schematic block diagram of a data processing device provided for at least one embodiment of the present disclosure.

[0143] The data processing device is used to load the activation tensor and the convolution kernel tensor from the memory into the cache component of the processor, and use the thread bundle to perform a convolution operation on the activation tensor and the convolution kernel tensor to obtain an output tensor.

[0144] like Fig.11 As shown, the data processing device 100 includes an execution module 110 .

[0145] The execution module 110 is configured to perform multiple first operations to obtain the output tensor, wherein the output tensor is expressed as [NPQ, K], P represents the height, Q represents the width, N represents the number of batches, K represents the number of output channels, and the output tensor is divided into multiple sub-output tensors with a granularity of [m, n]. Each first operation is used to obtain a sub-output tensor, m and n are positive integers, and the output tensor is obtained by sequentially splicing at least one sub-output tensor obtained by the first operation.

[0146] For example, the execution module 110 includes codes and programs stored in the memory. The execution module 110 is implemented as a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities. The processing unit can be a general-purpose processor, and can also be a single-chip microcomputer, a microprocessor, a digital signal processor, a dedicated image processing chip, or a field programmable logic array, etc. The execution module 110 executes the code and program to implement some or all of the functions of the execution module 110 described above. For example, the execution module 110 can be a circuit board or a combination of multiple circuit boards, which is used to implement the functions described above. In an embodiment of the present application, the circuit board or the combination of multiple circuit boards may include: (1) one or more processors; (2) one or more non-temporary memories connected to the processor; and (3) firmware stored in the memory that can be executed by the processor.

[0147] For example, the execution module can be implemented as Figure 1In the computing units shown, different computing units perform different first operations on different data blocks to obtain different sub-output tensors.

[0148] For example, the execution module includes a first loading module 1101 , a second loading module 1102 , a third loading module 1103 , a tensor calculation unit 1104 and an accumulation module 1105 .

[0149] The first loading module 1101 is configured to use the first thread warp to load multiple sub-convolution kernel tensors from the memory to the cache component.

[0150] For example, the convolution kernel tensor is expressed as [R, S, K, C], where R represents the convolution kernel height, S represents the convolution kernel width, and C represents the number of input channels. Multiple sub-convolution kernel tensors are obtained by splitting the convolution kernel tensor on the RS plane composed of the convolution kernel height and the convolution kernel width according to the granularity of 1×a. Each sub-convolution kernel tensor is expressed as [1, a, m, C1], where a is a positive integer greater than 1 and less than or equal to S, and S is an integer multiple of a.

[0151] The second loading module 1102 is configured to use the first thread warp to load multiple sub-activation tensors corresponding to the multiple sub-convolution kernel tensors from the memory to the cache component.

[0152] For example, each sub-activation tensor is the portion of the activation tensor that the corresponding sub-convolution kernel tensor needs when performing a convolution operation.

[0153] The third loading module 1103 is configured to use the second thread warp to sequentially read each sub-convolution kernel tensor and the sub-activation tensor corresponding to the sub-convolution kernel tensor from the cache component to the tensor calculation unit.

[0154] The tensor computing unit 1104 is configured to perform a matrix multiplication and accumulation operation between each sub-convolution kernel tensor read and the corresponding sub-activation tensor.

[0155] For example, the tensor computing unit 1104 may be a computing core dedicated to tensor operations, such as a tensor core, to accelerate machine learning training and reasoning tasks.

[0156] For example, the accumulation module 1105 is configured to accumulate the calculation results of multiple matrix multiplication and accumulation operations using the second thread warp to obtain a sub-output tensor. For example, the accumulation module 1105 is implemented by an accumulator (thread private register).

[0157] It should be noted that the execution module 110 can be used to implement Figure 3 Step S10 shown; the first loading module 1101 can be used to implement Figure 6 Step S20 shown; the second loading module 1102 can be used to implement Figure 6Step S30 shown; the third loading module 1103 and the tensor calculation unit 1104 can be used to implement Figure 6 Step S40 shown; the accumulation module 1105 can be used to implement Figure 6 Step S50 shown. Therefore, for the specific description of the functions that can be implemented by the execution module 110, reference can be made to the relevant description of step S10 in the embodiment of the above-mentioned data processing method. For the specific description of the functions that can be implemented by the first loading module 1101, reference can be made to the relevant description of step S20 in the embodiment of the above-mentioned data processing method. For the specific description of the functions that can be implemented by the second loading module 1102, reference can be made to the relevant description of step S30 in the embodiment of the above-mentioned data processing method. For the specific description of the functions that can be implemented by the third loading module 1103 and the tensor calculation unit 1104, reference can be made to the relevant description of step S40 in the embodiment of the above-mentioned data processing method. For the specific description of the functions that can be implemented by the accumulation module 1105, reference can be made to the relevant description of step S50 in the embodiment of the above-mentioned data processing method. The repeated parts will not be repeated here. In addition, the data processing device 100 can achieve technical effects similar to those of the aforementioned data processing method, which will not be repeated here.

[0158] It should be noted that in at least one embodiment of the present disclosure, the data processing device 100 may include more or fewer circuits or units, and the connection relationship between the various circuits or units is not limited and can be determined according to actual needs. The specific configuration of each circuit or unit is not limited and can be composed of analog devices according to circuit principles, or can be composed of digital chips, or can be composed in other applicable ways.

[0159] For example, the data processing device 100 may be implemented in hardware, software, or a combination of hardware and software, and the present disclosure does not impose any specific limitations on this.

[0160] The data processing method provided by at least one embodiment of the present disclosure can achieve a technical effect similar to the data processing method described above, and will not be described in detail here.

[0161] The data processing method and data processing device provided in at least one embodiment of the present disclosure can be applied to different systems or devices, such as Fig.13The electronic device 300 shown. The electronic device 300 can be a terminal, such as a mobile phone terminal, a tablet computer, a laptop computer, an AR device, a VR device, a vehicle terminal, etc., and can also be a server, etc. The data processing method provided in at least one embodiment of the present disclosure can be applied to scenarios involving CPU, high performance computing (HPC) and artificial intelligence (AI) in the electronic device 300, such as tensor computing units, etc. Regarding convolution operations. Of course, the present disclosure is not limited to this, and any scenario, device, apparatus, etc. involving back-propagation calculations can adopt the data processing method or data processing apparatus provided by at least one embodiment of the present disclosure.

[0162] In some embodiments, the data processing device provided by at least one embodiment of the present disclosure may be a chip, for example, the chip is a system-on-a-chip (SoC). The system-on-a-chip includes a processor, which may be a single-core processor or a multi-core processor, a memory, and an I / O interface, etc. After the processor loads the data and application programs in the memory, it processes the data, for example, performs computational processing of back propagation calculations.

[0163] It should be noted that the activation tensor of the data processing method or data processing device provided in at least one embodiment of the present disclosure may have different specific physical meanings according to different application scenarios. For example, the data processing method provided in at least one embodiment of the present disclosure may be applied in the fields of speech processing, image processing, text processing, video processing, etc.

[0164] For example, in the field of speech processing, the activation tensor can be the input and output parameters of convolution operations involved in tasks such as feature extraction, speech enhancement, and speech recognition.

[0165] For example, in the field of image processing, activation tensors can be the input and output parameters of convolution operations involved in tasks such as image recognition, feature extraction, image segmentation, object detection, image classification, and scene reconstruction.

[0166] For example, in the field of text processing, activation tensors can be the input and output parameters of convolution operations involved in tasks such as text classification, sentiment analysis, and text generation.

[0167] For example, in the field of video processing, the activation tensor can be the relevant parameters in the image processing field as mentioned above, or the input and output parameters of the convolution operation that are unique to the video processing field, such as the optical flow operator (used to estimate the motion between video frames) and the target tracking operator (used to track a specific target in the video).

[0168] Of course, the present disclosure is not limited to this, and for other application scenarios or fields, as long as a convolution operation is required, the data processing method described in at least one embodiment of the present disclosure can be applied, and will not be described one by one here.

[0169] Fig.12 A schematic structural diagram of a processor provided in at least one embodiment of the present disclosure. Fig.12 As shown, the processor 200 includes an instruction parsing unit 201 , an execution unit 202 , a memory 203 and a cache unit 204 .

[0170] For example, the instruction parsing unit 201 is used to receive and parse data processing instructions.

[0171] For example, the data processing instructions are used to load activation tensors and convolution kernel tensors from the memory 203 to the cache unit 204 of the processor, and use the thread warp to perform a convolution operation on the activation tensor and the convolution kernel tensor to obtain an output tensor.

[0172] For example, the cache component 204 may include a shared memory. For example, the processor 200 may use the aforementioned Figure 1 The graphics processor or general graphics processor architecture shown.

[0173] For example, the execution unit 202 executes the data processing instruction after the instruction parsing unit parses the data processing instruction.

[0174] For example, after receiving a data processing instruction, the processor parses the data processing instruction, such as decoding the data processing instruction, generating microinstructions and sending the microinstructions to the instruction dispatch unit; the instruction dispatch unit sends the microinstructions to the corresponding scheduling queue according to the microinstruction category; the scheduler in the processor selects appropriate microinstructions from the scheduling queue for execution based on factors such as the priority and dependencies of the microinstructions; in response to the microinstructions, when all or required input data (such as activation tensors and convolution kernel tensors) are prepared, the execution unit reads the data and executes the relevant operations of the data processing instruction.

[0175] For example, the execution unit may include hardware involved in executing data processing instructions, such as a multiplier, a register, a queue, an ALU (Arithmetic Logic Unit), etc.

[0176] For example, when the execution unit 202 executes a data processing instruction, it includes using the first thread warp to load multiple sub-convolution kernel tensors from the memory to the cache component, wherein the convolution kernel tensor is represented as [R, S, K, C], R represents the convolution kernel height, S represents the convolution kernel width, and C represents the number of input channels. The multiple sub-convolution kernel tensors are obtained by splitting the convolution kernel tensor on the RS plane composed of the convolution kernel height and the convolution kernel width according to a granularity of 1×a. Each sub-convolution kernel tensor is represented as [1, a, m, C1], where a is a positive integer greater than 1 and less than S, and S is an integer multiple of a.

[0177] When processor-based upper-level software (e.g., AI applications, HPC applications, and scientific computing applications, etc.) can send data processing instructions for convolution operations to a processor (e.g., a CPU or GPU) through a uniformly packaged function library, the data processing instructions can carry input parameters such as storage addresses; when the processor receives the data processing instructions, the instruction parsing unit 201 parses the data processing instructions, obtains the input parameters, loads the data indicated by the storage address from the memory to the register, and caches the data in the cache component.

[0178] For example, the execution unit includes a computing unit, and different computing units are configured to perform different first operations. For example, each computing unit refers to the aforementioned steps S20-S50 to perform the first operation to obtain a sub-output tensor with a shape size of [m, n], and finally uses the TMA instruction to copy the sub-output tensor from the shared memory to the corresponding position in the memory to sequentially splice into the output tensor.

[0179] Regarding the specific process of using the execution unit 202 to execute the data processing instruction, reference may be made to step S10 and other related contents in the aforementioned data processing method, and the repeated parts will not be repeated here.

[0180] The processor provided in at least one embodiment of the present disclosure can achieve technical effects similar to those of the aforementioned data processing method, and the repeated parts will not be repeated.

[0181] Fig.13 The following is a schematic block diagram of an electronic device provided by an embodiment of the present disclosure. Fig.13 As shown, the electronic device 300 is suitable for implementing the data processing method provided by the embodiment of the present disclosure. It should be noted that Fig.13 The components of the electronic device 300 shown are merely exemplary and non-limiting. The electronic device 300 may also have other components according to actual application requirements.

[0182] like Fig.13As shown, the electronic device 300 may include a processing device (eg, a central processing unit, a graphics processing unit, etc.) 301, which may perform various appropriate actions and processes according to non-transitory computer-readable instructions stored in a memory to implement various functions.

[0183] For example, when the computer readable instructions are executed by the processing device 301, one or more steps in the data processing method described in any of the above embodiments may be executed. It should be noted that for a detailed description of the processing process of the data processing method, reference may be made to the relevant description in the above data processing method embodiments.

[0184] For example, the memory may include any combination of one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) 303 and / or cache memory (cache), etc. For example, computer-readable instructions may be loaded from storage device 308 into random access memory (RAM) 303 to run computer-readable instructions. Non-volatile memory may include, for example, read-only memory (ROM) 302, hard disk, erasable programmable read-only memory (EPROM), portable compact disk read-only memory (CD-ROM), USB memory, flash memory, etc. Various applications and various data, such as style images, and various data used and / or generated by applications, etc., may also be stored in the computer-readable storage medium.

[0185] For example, the processing device 301 , the read-only memory 302 , and the random access memory 303 are connected to each other via a bus 304 . An input / output (I / O) interface 305 is also connected to the bus 304 .

[0186] Typically, the following devices may be connected to the input / output interface 305: an input device 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, a flash memory, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other electronic devices wirelessly or by wire to exchange data. Although Fig.13An electronic device 300 having various devices is shown, but it should be understood that it is not required to implement or have all the devices shown, and the electronic device 300 may alternatively implement or have more or fewer devices. For example, the processing device 301 can control other components in the electronic device 300 to perform the desired functions. The processing device 301 can be a device with data processing capabilities and / or program execution capabilities such as a central processing unit (CPU), a tensor processing unit (TPU) or a graphics processing unit GPU. The central processing unit (CPU) can be an X86, ARM, RISC-V architecture, etc. The GPU can be directly integrated into the SOC, directly integrated into the motherboard, or built into the north bridge chip of the motherboard.

[0187] Fig.14 A schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. Fig.14 As shown, the storage medium 400 may be a non-transitory computer-readable storage medium, and one or more computer-readable instructions 401 may be non-transitory stored on the storage medium 400. For example, when the computer-readable instructions 401 are executed by a processor, one or more steps in the data processing method described above may be executed.

[0188] For example, the storage medium 400 may be applied in the electronic device 300 . For example, the storage medium 400 may include the storage device 308 in the electronic device 300 .

[0189] For example, the storage device may include any combination of one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disk read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer-readable instructions may be stored on the computer-readable storage medium, and the processor may run the computer-readable instructions to implement various functions of the processor. Various applications and various data may also be stored in the storage medium.

[0190] For example, the storage medium may include a memory card of a smart phone, a cache component of a tablet computer, a hard disk of a personal computer, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), flash memory, or any combination of the above storage media, or other applicable storage media.

[0191] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0192] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0193] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0194] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0195] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0196] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

[0197] There are a few points to note about this disclosure:

[0198] (1) The drawings of the embodiments of the present disclosure only relate to the structures related to the embodiments of the present disclosure. Other structures may refer to the general design.

[0199] (2) In the absence of conflict, the embodiments of the present disclosure and the features therein may be combined with each other to obtain new embodiments.

[0200] The above description is only a specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto. The protection scope of the present disclosure shall be based on the protection scope of the claims.

Claims

1. A data processing method for loading an activation tensor and a convolution kernel tensor from a memory into a cache component of a processor, and performing a convolution operation on the activation tensor and the convolution kernel tensor using a thread warp to obtain an output tensor, The data processing method comprises: Perform the first operation multiple times to obtain the output tensor, wherein the output tensor is expressed as [NPQ, K], P represents the height, Q represents the width, N represents the batch number, K represents the number of output channels, and the output tensor is divided into multiple sub-output tensors with [m, n] as the granularity. Each first operation is used to obtain a sub-output tensor, m and n are positive integers, and the output tensor is obtained by sequentially splicing the multiple sub-output tensors obtained by the multiple first operations; Wherein, each time the first operation is performed, the following steps are performed: Use the first thread warp to load multiple sub-convolution kernel tensors from the memory to the cache component, wherein the convolution kernel tensor is represented by [R, S, K, C], R represents the convolution kernel height, S represents the convolution kernel width, and C represents the number of input channels. The multiple sub-convolution kernel tensors are obtained by splitting the convolution kernel tensor on the RS plane composed of the convolution kernel height and the convolution kernel width according to the granularity of 1×a, and each sub-convolution kernel tensor is represented by [1, a, m, C1], where a is a positive integer greater than 1 and less than or equal to S, and S is an integer multiple of a; Using the first thread warp, load a plurality of sub-activation tensors corresponding one-to-one to the plurality of sub-convolution kernel tensors from the memory to the cache component, wherein each sub-activation tensor is a portion of the activation tensor required by the corresponding sub-convolution kernel tensor when performing the convolution operation; Using the second thread warp to sequentially read each sub-convolution kernel tensor and the sub-activation tensor corresponding to the sub-convolution kernel tensor from the cache component to a tensor calculation unit, so that the tensor calculation unit performs a matrix multiplication and accumulation operation between each sub-convolution kernel tensor read and the sub-activation tensor corresponding to the sub-convolution kernel tensor; The calculation results of the plurality of matrix multiplication-accumulation operations are accumulated using the second thread warp to obtain the sub-output tensor.

2. The method according to claim 1, wherein: When performing the matrix multiplication and accumulation operation, each convolution kernel element included in the sub-convolution kernel tensor reuses the corresponding sub-activation tensor loaded into the cache component.

3. The method according to claim 1, wherein: The activation tensor is expressed as [N,H,W,C], where W represents width and H represents height. Using the first thread warp to load a plurality of sub-activation tensors corresponding one-to-one to the plurality of sub-convolution kernel tensors from the memory to the cache component, comprising: For each sub-activation tensor, determine a first starting position of the sub-activation tensor in a first sequence according to a sub-convolution kernel tensor corresponding to the sub-activation tensor, wherein the first sequence is obtained by expanding the activation tensor into one dimension in a height dimension, a width dimension, and a batch number dimension, and the first sequence is represented as [NHW, C]; Starting from the first starting position, data with a shape size of [n+a-1, C1] is loaded as the sub-activation tensor to the cache component.

4. The method according to claim 1, wherein: Using the second thread warp to sequentially read each sub-convolution kernel tensor and the sub-activation tensor corresponding to the sub-convolution kernel tensor from the cache component to the tensor calculation unit, so that the tensor calculation unit performs a matrix multiplication and accumulation operation between each sub-convolution kernel tensor read and the sub-activation tensor corresponding to the sub-convolution kernel tensor, including: For each sub-convolution kernel tensor and the sub-activation tensor corresponding to the sub-convolution kernel tensor: Using the second thread warp to load the sub-convolution kernel tensor into the tensor computing unit; Using the second thread warp to perform a second operation a times to obtain a calculation result of a matrix multiplication-accumulation operation between the sub-convolution kernel tensor and the corresponding sub-activation tensor; Among them, each second operation is used to perform a convolution operation on the convolution kernel data with a shape size of [1, 1, m, C1] in the sub-convolution kernel tensor, so as to obtain the calculation result through the a times of the second operation.

5. The method according to claim 4, wherein: The second operation includes performing the following steps: Using the second thread warp to load activation tensor data with a shape size of [n, C1] from the corresponding sub-activation tensor to the tensor computation unit; The tensor calculation unit performs a matrix multiplication and accumulation operation between the convolution kernel data and the activation tensor data to obtain an operation result; The operation result is accumulated to the current calculation result to update the calculation result, wherein the current calculation result is obtained by accumulating the operation results obtained by the second operations previously performed in the a second operations.

6. The method according to claim 5, wherein: The corresponding sub-activation tensor includes n+a-1 sequence units, each sequence unit includes C1 elements, Using the second warp to load data of shape size [n, C1] from the corresponding sub-activation tensor to the tensor computation unit includes: Determine a second starting position according to the position of the convolution kernel data used by the second operation in the sub-convolution kernel tensor; Starting from the second starting position, sequentially read n sequence units in the corresponding sub-activation tensor to the tensor computation unit; During the reading process, the element value of the sequence unit at the filling position is modified to a filling default value and loaded into the tensor computing unit according to the sequence unit index information and the filling information, and the filling position is indicated by the sequence unit index information and the filling information.

7. The method according to claim 1, further comprising: Two second warps are enabled to use the tensor computing unit in a ping-pong strategy through a synchronization operation, wherein the two second warps are used to obtain different sub-output tensors.

8. The method according to claim 7, further comprising: Calling a vector computing unit to perform further computing processing on the sub-output tensor; Enabling two second thread warps to use the tensor computing unit in a ping-pong strategy through synchronization operation, comprising: In response to one of the two second thread warps completing the first operation, sending a synchronization signal to another second thread warp to notify the other second thread warp to perform the first operation, and at the same time, the one second thread warp calls the vector computing unit to perform the computing process; In response to the other second warp completing the first operation, a synchronization signal is sent to the one second warp to notify the one second warp to execute the first operation. Meanwhile, the other second warp calls the vector computing unit to perform the computing process.

9. The method according to any one of claims 1 to 8, wherein: Using the first thread warp to load a plurality of sub-convolution kernel tensors from the memory to the cache component includes: Use tensor memory access acceleration instructions to load each sub-convolution kernel tensor into the cache component.

10. The method according to any one of claims 1 to 8, further comprising: Copying the sub-output tensor to the cache component, and using a tensor memory access acceleration instruction to write the sub-output tensor to the memory, In the process of writing into the memory, the sub-output tensor is written to the corresponding position of the output tensor stored in the memory according to the positional relationship between the multiple sub-output tensors.

11. A processor comprising an instruction parsing unit, an execution unit, a cache component and a memory, wherein: The instruction parsing unit is used to receive and parse data processing instructions, wherein the data processing instructions are used to load activation tensors and convolution kernel tensors from the memory to the cache component, and use thread warps to perform convolution operations on the activation tensors and the convolution kernel tensors to obtain output tensors. The execution unit executes the data processing method according to any one of claims 1 to 10 after the instruction parsing unit parses the data processing instruction.

12. The processor of claim 11, wherein: The execution unit includes a computing unit, and different computing units are configured to perform different first operations.

13. A data processing device, for loading an activation tensor and a convolution kernel tensor from a memory into a cache component of a processor, and performing a convolution operation on the activation tensor and the convolution kernel tensor using a thread warp to obtain an output tensor, The data processing device comprises an execution module, wherein the execution module is configured to execute a plurality of first operations to obtain the output tensor, wherein: The output tensor is expressed as [NPQ, K], where P represents the height, Q represents the width, N represents the number of batches, and K represents the number of output channels. The output tensor is divided into multiple sub-output tensors with a granularity of [m, n]. Each first operation is used to obtain a sub-output tensor, m and n are positive integers, and the output tensor is obtained by sequentially splicing the multiple sub-output tensors obtained by the multiple first operations; The execution module includes a first loading module, a second loading module, a third loading module, a tensor calculation unit and an accumulation module. The first loading module is configured to use the first thread warp to load multiple sub-convolution kernel tensors from the memory to the cache component, wherein the convolution kernel tensor is expressed as [R, S, K, C], R represents the convolution kernel height, S represents the convolution kernel width, and C represents the number of input channels. The multiple sub-convolution kernel tensors are obtained by splitting the convolution kernel tensor on the RS plane composed of the convolution kernel height and the convolution kernel width according to the granularity of 1×a, and each sub-convolution kernel tensor is expressed as [1, a, m, C1], where a is a positive integer greater than 1 and less than or equal to S, and S is an integer multiple of a; The second loading module is configured to use the first thread warp to load a plurality of sub-activation tensors corresponding to the plurality of sub-convolution kernel tensors from the memory to the cache component, wherein each sub-activation tensor is a portion of the activation tensor required by the corresponding sub-convolution kernel tensor when performing the convolution operation; The third loading module is configured to use the second thread warp to sequentially read each sub-convolution kernel tensor and the sub-activation tensor corresponding to the sub-convolution kernel tensor from the cache component to the tensor calculation unit; The tensor computing unit is configured to perform a matrix multiplication and accumulation operation between each sub-convolution kernel tensor read and a sub-activation tensor corresponding to the sub-convolution kernel tensor; The accumulation module is configured to accumulate calculation results of multiple matrix multiplication and accumulation operations using the second thread warp to obtain the sub-output tensor.

14. An electronic device comprising: A memory non-transitorily stores computer executable instructions; a processor configured to execute the computer executable instructions, Wherein, when the computer executable instructions are executed by the processor, the data processing method according to any one of claims 1-10 is implemented.

15. A non-transitory computer-readable storage medium, wherein: The non-transitory computer-readable storage medium stores computer-executable instructions, When the computer executable instructions are executed by a processor, the data processing method according to any one of claims 1 to 10 is implemented.

Images