Data processing method, electronic device, and storage medium

Abstract

A data processing method, an electronic device and a storage medium. The data processing method comprises: receiving a first original tensor; performing dimension conversion processing on the first original tensor to obtain a second original tensor; performing size expansion on at least one dimension of the second original tensor, and storing a third original tensor obtained by size expansion to a memory; receiving shape information of a to-be-processed tensor and a starting coordinate in a preset coordinate system; combining the third original tensor, the shape information of the to-be-processed tensor and the starting coordinate of the to-be-processed tensor to determine a plurality of requests for loading the to-be-processed tensor to a cache area; sequentially sending the plurality of requests, and sequentially writing sub-data returned by each request to the cache area to load the to-be-processed tensor to the cache area. The method supports dimension conversion, reduces the complexity of dimension conversion on data loading, improves bandwidth utilization, effectively utilizes data bandwidth, and improves loading efficiency.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to a data processing method, an electronic device and a non-transitory computer-readable storage medium. BACKGROUND

[0002] A tensor is a multi-linear mapping defined on the Cartesian product of some vector spaces and some dual spaces, for example, a scalar can be regarded as a 0-dimensional tensor, a vector can be regarded as a 1-dimensional tensor, a matrix can be regarded as a 2-dimensional tensor, and a tensor can have any number of dimensions. Tensor operations are widely used in processors such as parallel processors.

[0003] With the development of artificial intelligence and machine learning, new requirements are put forward for many parallel processor devices represented by parallel processors (for example, multi-core processors, digital signal processors, etc.). In general computing, the computing unit of the parallel processor needs a large amount of data, and these data are generally stored in the storage component of the parallel processor, for example, the storage component can be a memory. Through a data loading instruction, these data can be extracted from the storage component to the cache area for calculation, and through a data storage instruction, the data in the cache area can be stored to the memory.

[0004] The current data storage or loading method wastes data transmission bandwidth, reduces the bandwidth utilization, and cannot effectively utilize the data bandwidth. SUMMARY

[0005] In at least one embodiment of the present disclosure, a data processing method is provided. A first original tensor is received, wherein the first original tensor is a 5-dimensional tensor, and the 5 dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a channel number dimension. Dimension conversion processing is performed on the first original tensor to obtain a second original tensor, wherein the dimension conversion processing includes converting p dimensions in the 5 dimensions, and the p dimensions include at least two of the other four dimensions except the channel number dimension, and p is a positive integer greater than or equal to 2 and less than or equal to 4. At least one dimension of the second original tensor is expanded in size, and a third original tensor obtained by the size expansion is stored in a memory. In the memory, a starting tensor element on each dimension in the third original tensor is located at a starting position of a different storage unit, wherein the starting tensor element is a tensor element corresponding to a starting coordinate of the each dimension in a preset coordinate system, the memory performs read and write operations with the storage unit as a granularity, and the storage unit stores a plurality of tensor elements. Shape information of a to-be-processed tensor and a starting coordinate of the to-be-processed tensor in the preset coordinate system are received. In combination with the third original tensor, the shape information of the to-be-processed tensor, and the starting coordinate of the to-be-processed tensor, a plurality of requests for loading the to-be-processed tensor to a cache area are determined. The plurality of requests are sent in sequence, and sub-data returned by each request is sequentially written to the cache area to load the to-be-processed tensor to the cache area.

[0006] For example, in the data processing method provided in at least one embodiment of the present disclosure, an expanded size of the at least one dimension after the size expansion is determined based on a storage capacity of the storage unit.

[0007] For example, in the data processing method provided in at least one embodiment of the present disclosure, the size expansion is performed on at least one dimension of the second original tensor, including: for any one dimension in the second original tensor, in response to the total storage capacity of all tensor elements with the same coordinates in the target dimension and above the target dimension in the second original tensor not being an integer multiple of the storage capacity of the storage unit, determining to perform size expansion on the any one dimension, and determining the expanded size of the any one dimension according to the total storage capacity; in response to the total storage capacity being an integer multiple of the storage capacity of the storage unit, determining not to perform size expansion on the any one dimension; wherein, in response to the any one dimension being the batch dimension, the target dimension is the any one dimension, and in response to the any one dimension not being the batch dimension, the target dimension is a dimension adjacent to the any one dimension and above the any one dimension indicated by the data storage format of the second original tensor. For example, in the data processing method provided in at least one embodiment of the present disclosure, the expanded size of the any one dimension is such that the total storage capacity of all tensor elements with the same coordinates in the target dimension in the third original tensor is an integer multiple of the storage capacity of the storage unit.

[0008] For example, in the data processing method provided in at least one embodiment of the present disclosure, in response to determining that the to-be-processed tensor is loaded in a first dimension that cannot be continuously loaded but in each dimension below the first dimension that can be continuously loaded, performing the requested division in a second dimension, each requested loading sub-data is located in the memory or is not located in the memory, and in response to the requested loading sub-data being located in the memory, the requested loading sub-data is from the second original tensor and is continuously stored in the memory, wherein the storage format of the to-be-processed tensor indicates that the first dimension is prior to the second dimension in storage or loading, and the first dimension and the second dimension are adjacent.

[0009] For example, in the data processing method provided in at least one embodiment of the present disclosure, when storing the third original tensor into the memory, for the part not belonging to the original content of the second original tensor due to the size expansion, a pre-defined format is used for storage in the memory, wherein the pre-defined format is used to indicate that the corresponding storage content is requested to be divided as data not located in the memory.

[0010] For example, in the data processing method provided by at least one embodiment of the present disclosure, in response to any one of the following conditions being met, it is determined that the to-be-processed tensor cannot be continuously loaded in the first dimension: (1) the size of the to-be-processed tensor in the first dimension is not equal to the size of the second original tensor in the first dimension; (2) the first coordinate value of the starting coordinate of the to-be-processed tensor in the first dimension is not equal to the second coordinate value of the starting coordinate of the third original tensor in the first dimension; (3) the distance interval corresponding to the second dimension is not equal to the predetermined value corresponding to the first dimension, wherein the distance interval corresponding to the second dimension is the distance interval between two target tensor elements stored in the memory, the two target tensor elements being two tensor elements adjacent in the second dimension in the third original tensor, and the predetermined value corresponding to the first dimension being determined based at least on the shape size of the second original tensor.

[0011] For example, in the data processing method provided by at least one embodiment of the present disclosure, the distance interval corresponding to the second dimension is determined at least according to the shape size of the third original tensor, and in response to the storage format of the third original tensor being NDHWC, the product between the shape sizes of each dimension lower than the second dimension in the third original tensor is determined as the distance interval corresponding to the second dimension, wherein N represents the batch dimension, D represents the depth dimension, W represents the width dimension, H represents the height dimension, and C represents the channel number dimension.

[0012] For example, in the data processing method provided by at least one embodiment of the present disclosure, in response to the storage format of the third original tensor being N(C / x)DHW(xC): in the case where the second dimension is not the batch dimension, the product between the shape size of each dimension lower than the second dimension in the third original tensor and the storage capacity of x tensor elements is determined as the distance interval corresponding to the second dimension; in the case where the second dimension is the batch dimension, the product between the shape sizes of each dimension lower than the second dimension in the third original tensor is determined as the distance interval corresponding to the second dimension; wherein N represents the batch dimension, D represents the depth dimension, W represents the width dimension, H represents the height dimension, C represents the channel number dimension, and x is a positive integer greater than 1.

[0013] For example, in the data processing method provided by at least one embodiment of the present disclosure, in response to the data storage format of the second original tensor being NDHWC, in the case that the first dimension is the channel number dimension, the shape size of the second original tensor in the first dimension is determined as the predetermined value corresponding to the first dimension; in the case that the first dimension is not the channel number dimension, the product between the shape size of the second original tensor in the first dimension and the distance interval corresponding to the first dimension is determined as the predetermined value corresponding to the first dimension.

[0014] For example, in the data processing method provided by at least one embodiment of the present disclosure, in response to the data storage format of the second original tensor being N(C / x)DHW(xC), in the case that the second dimension is the height dimension, the product between the shape size of the second original tensor in the first dimension and the storage capacity of x tensor elements is determined as the predetermined value corresponding to the first dimension; in the case that the second dimension is the depth dimension or the channel number dimension, the product between the shape size of the second original tensor in the first dimension and the distance interval corresponding to the first dimension is determined as the predetermined value corresponding to the first dimension; in the case that the second dimension is the batch dimension, the quotient between the shape size of the second original tensor in the first dimension and the storage capacity of the x tensor elements is determined, and the product between the quotient and the distance interval corresponding to the first dimension is determined as the predetermined value corresponding to the first dimension.

[0015] For example, in the data processing method provided by at least one embodiment of the present disclosure, in combination with the third original tensor, the shape information of the to-be-processed tensor and the starting coordinates of the to-be-processed tensor, a plurality of requests for loading the to-be-processed tensor to a cache area are determined, including: based on the starting coordinates of the to-be-processed tensor and the shape information of the to-be-processed tensor, determining a first request sent in the plurality of requests and an initial state of the first request entering a state machine; based on the initial state, in combination with the third original tensor, determining each request in the plurality of requests except the first request by using the state machine.

[0016] For example, in the data processing method provided by at least one embodiment of the present disclosure, each request includes a data read address for indicating a starting position of reading data from the memory, a data write address for indicating a starting position of writing data to the cache area, and a data length loaded by the request. Based on the starting coordinates of the to-be-processed tensor and the shape information of the to-be-processed tensor, the first request in the plurality of requests and the initial state of the first request entering the state machine are determined, including: determining that the first request enters the initial state in the state machine based on the starting coordinates of the to-be-processed tensor; taking the starting coordinates of the to-be-processed tensor as the request initial coordinates corresponding to the first request; determining the data read address of the first request according to the distance interval corresponding to each dimension in the third original tensor and the request initial coordinates corresponding to the first request; determining the starting address of writing the to-be-processed tensor in the cache area as the data write address of the first request; and determining the data length loaded by the first request according to the starting coordinates of the to-be-processed tensor and the shape information of the to-be-processed tensor.

[0017] For example, in the data processing method provided by at least one embodiment of the present disclosure, based on the starting coordinates of the to-be-processed tensor, the initial state of the first request entering the state machine is determined, including: in response to the first coordinate value of the starting coordinates of the to-be-processed tensor in the first dimension being less than the second coordinate value of the starting coordinates of the third original tensor in the first dimension, determining that the initial state is a first state, in response to the first coordinate value being greater than or equal to the second coordinate value and less than a third coordinate value, determining that the initial state is a second state, wherein the difference between the second coordinate value and the third coordinate value is equal to the size of the second original tensor in the first dimension, and in response to the first coordinate value being greater than or equal to the third coordinate value, determining that the initial state is a third state.

[0018] For example, in the data processing method provided by at least one embodiment of the present disclosure, based on the initial state, the state machine is used to determine each request in the plurality of requests except the first request in combination with the third original tensor, including: based on the initial state, the state machine is used to determine the request initial coordinates corresponding to each request and the data length loaded by each request; the data read address of each request is determined according to the distance interval corresponding to each dimension in the third original tensor and the request initial coordinates corresponding to each request; and the data write address of each request is determined according to the data length loaded by each request.

[0019] For example, in the data processing method provided in at least one embodiment of the present disclosure, the dimension conversion processing is performed on the first original tensor to obtain a second original tensor, including: performing a transpose operation on the p dimensions of the first original tensor to obtain the second original tensor.

[0020] In at least one embodiment of the present disclosure, an electronic device is provided, including: a memory, which is configured to store computer executable instructions; and a processor, which is configured to execute the computer executable instructions, and when the computer executable instructions are executed by the processor, the data processing method according to at least one embodiment of the present disclosure is implemented.

[0021] In at least one embodiment of the present disclosure, a non-transitory computer readable storage medium is provided, which stores computer executable instructions, and when the computer executable instructions are executed by a processor, the data processing method according to at least one embodiment of the present disclosure is implemented.

[0022] In the data processing method provided in at least one embodiment of the present disclosure, the dimension conversion is supported, the complexity of the dimension conversion on the data loading is reduced, the size expansion is set, the address alignment is ensured, the starting tensor element of each dimension is located at the starting position of the storage unit, and therefore the bandwidth utilization rate is improved, the data bandwidth is effectively utilized, and the loading efficiency is improved. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings described in the following are only related to some embodiments of the present disclosure, but not limited to the present disclosure.

[0024] Figure 1 FIG. 1 is a structural schematic diagram of a general-purpose graphics processor (GPGPU);

[0025] Figure 2A FIG. 2 is a schematic structure of a tensor;

[0026] Figure 2B FIG. 3 is a storage format schematic diagram of NDHWC;

[0027] Figure 2C FIG. 4 is a storage format schematic diagram of N(C / 32)DHW(32C);

[0028] Figure 3 FIG. 5 is a schematic flowchart of the data processing method provided in at least one embodiment of the present disclosure;

[0029] Figure 4 FIG. 6 is a schematic structure diagram of the second original tensor provided in at least one embodiment of the present disclosure;

[0030] Figure 5AA schematic diagram of a tensor to be processed is provided for an embodiment of the present disclosure.

[0031] Figure 5B A schematic diagram of a tensor to be processed is provided for another embodiment of the present disclosure.

[0032] Figure 6 A schematic diagram of a state machine is provided for an embodiment of the present disclosure.

[0033] Figure 7 A schematic structural diagram of a data processing apparatus is provided for at least one embodiment of the present disclosure.

[0034] Figure 8 A schematic diagram of a non-transitory computer readable storage medium is provided for at least one embodiment of the present disclosure.

[0035] Figure 9 A schematic block diagram of an electronic device is provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] In order to make the objectives, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions of the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are some, but not all, of the embodiments of the present disclosure. Based on the described embodiments of the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative effort belong to the scope of the present disclosure.

[0037] Unless otherwise defined, technical terms or scientific terms used in the present disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which the present disclosure pertains. The terms “first”, “second” and similar terms used in the present disclosure do not denote any order, quantity or importance, but are used to distinguish different components. The terms “include” or “contain” and similar terms mean that the elements or objects before the terms encompass the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects. The terms “connect” or “connected” and similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “up”, “down”, “left”, “right” and the like only represent relative positional relationships, which can change when the absolute positions of the described objects change. In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits the detailed description of some known functions and known components.

[0038] Figure 1 A schematic structural diagram of a general-purpose graphics processing unit (GPGPU) is provided.

[0039] AsFigure 1 As shown, the general-purpose graphics processor is actually an array of programmable multi-processors, for example, the programmable multi-processors can be Streaming Processor Clusters (SPCs), for example, including Figure 1 Streaming Processor Cluster 1,..., Streaming Processor Cluster M, M being a positive integer greater than 1. In the general-purpose graphics processor, one Streaming Processor Cluster processes a computing task, or multiple Streaming Processor Clusters process a computing task. The multiple Streaming Processor Clusters share data through a global cache or a global memory.

[0040] As shown, taking Streaming Processor Cluster 1 as an example, one Streaming Processor Cluster includes multiple computing units, for example Figure 1 Computing Unit 1, Computing Unit 2,..., Computing Unit N in Figure 1 , N being a positive integer. Each computing unit (CU) is used to perform arithmetic logic operations, for example, accumulation, reduction, general addition, subtraction, multiplication, division, etc. One computing unit includes multiple cores (also referred to as computing cores or computing cores), each computing core including an arithmetic logic unit (ALU), a floating point computing unit, etc., and the computing core is used to perform a specific computing task. In addition, the computing unit also includes a register (for example, a register file in Figure 1 ) and a shared memory, which are used to store source data and destination data related to the computing task in layers, and the shared memory in one computing unit is used to share data among the cores in the computing unit.

[0041] As shown, each Streaming Processor Cluster is also provided with a buffer (Buffer) for data caching of the N computing units in the Streaming Processor Cluster. Figure 1

[0042] In parallel computing, a computing task is generally executed through multiple threads. These threads are divided into multiple thread blocks before being executed in the general-purpose graphics processor (or parallel computing processor), and then the multiple thread blocks are distributed to the computing units via a thread block distribution module (not shown in Figure 1 ). All threads in one thread block must be allocated to the same computing unit for execution. At the same time, the thread block is split into the smallest execution thread bundle (or simply referred to as thread bundle, warp), each thread bundle contains a fixed number (or less than the fixed number) of threads, for example, 32 threads. Multiple thread blocks can be executed in the same computing unit, or in different computing units.

[0043] In each computing unit, a thread bundle scheduling / distribution module (not shown in Figure 1 ​The memory execution instruction is transmitted to the shared memory in the computing unit or further transmitted to the L2 cache or the global cache or the global memory (for example, High Bandwidth Memory, HBM) to perform read / write operations, etc. Figure 1

[0044] As shown in Figure 1 , a general computing operation, such as a computing operation on a matrix or a convolution computing operation, usually requires a large amount of data, which is usually stored in a memory, such as an HBM. When performing a general computing operation, data needs to be loaded from the memory (Load operation), and when obtaining a computing result, data needs to be stored to the memory (Store operation). The storage manner of data in the memory will affect the memory access bandwidth, and further affect the hardware utilization of the computing unit.

[0045] For example, the shape size of a tensor can be represented by a1, a2, a3, a4, a5, a1, a2, a3, a4, a5 respectively indicate the size of the tensor data in 5 dimensions, and a1, a2, a3, a4, a5 are positive integers. For example, the 5 dimensions include [N, D, H, W, C], the N dimension represents the batch size, that is, the number of data samples grabbed at one time, the D dimension represents the depth, the H dimension represents the height, the W dimension represents the width, and the C dimension represents the number of channels. For example, taking the first tensor as an example, a1 can be the N dimension size, a2 can be the D dimension size, a3 can be the H dimension size, a4 can be the W dimension size, and a5 can be the C dimension size. Of course, the present disclosure does not make specific limitations thereto.

[0046] For example, the general computing operation includes a general matrix multiplication (GEMM). The data required for the general matrix multiplication includes a first tensor and a second tensor.

[0047] For example, the general computing operation can include a convolution operation, and the data required for the convolution operation includes two types of convolution kernels (kernal) and input data (activation), for example, the input data can be an input picture required for the convolution operation, etc. The convolution operation can include, for example, convolution multiplication calculation in neural network processing (for example, convolution neural network) frequently used in the field of artificial intelligence, etc., such as multi-channel convolution, multi-dimensional convolution.

[0048] The elements of the tensor are stored in the memory (for example,​Figure 1 There are many formats for placing tensors in memory, called data storage formats (layouts). The data storage format is used to indicate the storage order and dimension arrangement of tensors in the storage component.

[0049] For example, Figure 2A is a schematic structure of a tensor. Figure 2A In the tensor shown, a1 is the N dimension and is equal to 1, a2 is the D dimension and is equal to 1, a3 is the H dimension and is equal to 5, a4 is the W dimension and is equal to 4, and a5 is the C dimension and is equal to 64.

[0050] For example, Figure 2A The pixel elements of the tensor in are represented as 0, 1, 2, 3, ..., and so on. Figure 2A The tensors shown describe different data storage formats.

[0051] For example, the data storage format may include NDHWC, also known as Linear mode. Figure 2B This is a schematic diagram of the storage format of NDHWC.

[0052] For example, for NDHWC, Figure 2B As shown, from the first channel (a5=0, Figure 2B The first element of c0) in ( Figure 2B 0 in the , and then stores the second channel (a5=1, Figure 2B The first element of c1 in ( Figure 2B 20), and so on, until the first elements of all channels are laid out, for example, to the 64th channel (a5=63, Figure 2B The first element of c63) ( Figure 2B After the element 1260 in the , select the first channel (a5=0, Figure 2B The second element of c0 in ( Figure 2B 1 in the array), and then store the second channel (a5=1, Figure 2B The second element of c1 in ( Figure 2B ), and so on, until the second elements of all channels are laid out, and so on.

[0053] For example, the data storage format may further include N(C / x)DHW(xC), also known as an interleave mode, where x may be 8, 16, 32, etc. as needed.

[0054] N(C / x)DHW(xC) is similar to NDHWC, but with one key difference that in the memory layout of N(C / x)DHW(xC), a5 channels are divided into a5 / x groups, each group of x channels: the first group consists of channels a5=0 to a5=x-1, the second group consists of channels a5=x to a5=2x-1, each group is arranged in the NDHWC format.

[0055] Figure 2C A storage format diagram for N(C / 32)DHW(32C).

[0056] As shown in Figure 2C , 64 channels are divided into two groups, each group of 32 channels. The first group consists of channels a5=0 (c0 in Figure 2C ) to a5=31 (c31 in Figure 2C ), and the second group consists of channels a5=32 to a5=63. Then each group is arranged in the NDHWC format.

[0057] When extracting or storing a partial tensor in the original tensor, it is impossible to extract each dimension continuously when extracting, and it is impossible to accurately obtain the requested data amount and data location when loading or storing data to be able to optimally implement data loading or storage, greatly reducing the data bandwidth when loading data, and reducing the hardware computing efficiency.

[0058] The storage capacity of a storage unit corresponding to a storage address in the data memory is usually greater than the bit width of a single tensor element. For example, in HBM, the storage capacity of a storage unit corresponding to a storage address is 512B, that is, a plurality of tensor elements can usually be stored in the storage unit corresponding to 1 address. For example, taking the tensor element as the FP16 data format, up to 256 tensor elements can be stored in a storage unit.

[0059] Currently, tensors are stored continuously in memory. For example, assuming that the tensor has a W dimension size of 7 and a C dimension size of 32, and the data format of the tensor element is BF16, taking the storage format as NDHWC for example, in a storage unit, not only all elements of the tensor with H dimension coordinate h are stored, but also 32 tensor elements with H dimension coordinate h+1, W dimension coordinate 0, and C dimension coordinate 0 to 31 in the tensor. That is, the data stored in a storage unit changes in the H dimension coordinate.

[0060] Assuming that the data to be loaded includes data with an H-dimension coordinate of h+1 and W-dimension coordinates of 0 and 1, when the data is loaded, not only data stored in a storage unit with an H-dimension coordinate of h+1 and a W-dimension coordinate of 0 needs to be read, but also data stored in another storage unit with an H-dimension coordinate of h+1 and a W-dimension coordinate of 1 needs to be read. For the memory, it is more hardware-friendly to read data starting from the address starting position, while reading data starting from the middle of the storage unit reduces the execution efficiency.

[0061] In addition, generally, for the HBM, data is operated in the granularity of the storage unit when the data is loaded from the memory to the cache area (or vice versa), so all the data in the storage unit is sent to the cache area regardless of whether it is needed or not. Therefore, for the above example, the loaded data contains all the data in at least two storage units, so it contains more data that is actually not needed to use, thereby wasting the data transmission bandwidth, reducing the bandwidth utilization, and failing to effectively utilize the data bandwidth.

[0062] In addition, in some scenarios, it is often necessary to perform a dimension conversion operation on the tensor in any dimension, and the current loading manner cannot be compatible with the tensor dimension conversion.

[0063] At least one embodiment of the present disclosure provides a data processing method, an electronic device and a non-transitory computer-readable storage medium.

[0064] The data processing method provided by at least one embodiment of the present disclosure includes: receiving a first original tensor, wherein the first original tensor is a 5-dimensional tensor, and the 5 dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension and a channel number dimension; performing dimension conversion processing on the first original tensor to obtain a second original tensor, wherein the dimension conversion processing includes converting p dimensions in the 5 dimensions, the p dimensions including at least two dimensions in the other four dimensions except the channel number dimension, and p is a positive integer greater than or equal to 2 and less than or equal to 4; performing size expansion on at least one dimension of the second original tensor, and storing a third original tensor obtained by the size expansion to a memory, wherein in the memory, a starting tensor element on each dimension in the third original tensor is located at a starting position of a different storage unit, wherein the starting tensor element is a tensor element corresponding to a starting coordinate of each dimension in a preset coordinate system, the memory performs read and write operations in the granularity of a storage unit, and the storage unit stores a plurality of tensor elements; receiving shape information of a to-be-processed tensor and a starting coordinate of the to-be-processed tensor in the preset coordinate system; determining a plurality of requests for loading the to-be-processed tensor to a cache area in combination with the third original tensor, the shape information of the to-be-processed tensor and the starting coordinate of the to-be-processed tensor; and sequentially sending the plurality of requests, and sequentially writing sub-data returned by each request to the cache area to load the to-be-processed tensor to the cache area.

[0065] In the data processing method provided in at least one embodiment of the present disclosure, dimension conversion is supported, which reduces the complexity of dimension conversion on data loading; size extension is set to ensure address alignment so that the starting tensor element of each dimension is located at the starting position of the storage unit, thereby improving bandwidth utilization, effectively utilizing data bandwidth, and improving loading efficiency.

[0066] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings, but the present disclosure is not limited to these specific embodiments.

[0067] Figure 3 A schematic flowchart of a data processing method provided in at least one embodiment of the present disclosure.

[0068] like Figure 3 As shown, the data processing method provided by at least one embodiment of the present disclosure includes at least steps S10-S60.

[0069] For example, in step S10 , a first original tensor is received.

[0070] The first original tensor is a 5-dimensional tensor, and the 5 dimensions include batch dimension, depth dimension, height dimension, width dimension, and channel number dimension.

[0071] The first original tensor can be obtained from other computing units, streaming processor clusters, etc., or the first original tensor can also be obtained from memory, and the present disclosure does not impose specific limitations on this.

[0072] The shape and size of the first original tensor can be expressed as b1×b2×b3×b4×b5, where b1, b2, b3, b4, and b5 respectively indicate the sizes of the original tensor in the five dimensions and are all positive integers. The present disclosure does not limit the specific shape and size of the first original tensor.

[0073] In step S20, dimension conversion is performed on the first original tensor to obtain a second original tensor.

[0074] The dimensionality conversion process includes performing dimensionality conversion on p dimensions among the five dimensions, where the p dimensions include at least two dimensions among the other four dimensions except the channel number dimension, and p is a positive integer greater than or equal to 2 and less than or equal to 4.

[0075] For example, step S20 may include: performing a transposition operation on the first original tensor in p dimensions to obtain a second original tensor.

[0076] For example, in one embodiment, p = 2, and the transposition operation on the first original tensor in 2 dimensions includes swapping two dimensions in the tensor. For example, the shape size of the first original tensor is represented as b1 x b2 x b3 x b4 x b5, and the second dimension (for b2) and the third dimension (for b3) are swapped, and the shape size of the second original tensor obtained is represented as b1 x b3 x b2 x b4 x b5. The same as the conventional transposition meaning, the swapping here includes swapping the specific content in the dimension. That is, in the dimension conversion process, only the positions of the selected p dimensions are exchanged, and the other dimensions remain in place. In the fields of deep learning and scientific computing, such dimension conversion is very common, for example, for adjusting the convolution input format or tensor rearrangement.

[0077] Figure 4 The schematic structural diagram of the second original tensor provided for at least one embodiment of the present disclosure.

[0078] Figure 4 In the middle, each solid small cube represents an element, and the tensor composed of multiple solid small cubes is the second original tensor. The second original tensor has undergone the dimension conversion process as described above.

[0079] Figure 4 What is shown is a batch and a tensor at a certain depth in the batch, but the second original tensor can have multiple batches, and each batch can have multiple tensors in the depth dimension, and the structure is the same as that shown in Figure 4 , which will not be described again here.

[0080] Suppose Figure 4 The element pointed to by the arrow is the origin of the preset coordinate system of the second original tensor, C, W, and H represent the three coordinate axes, row is the coordinate value in the H dimension direction, col is the coordinate value in the W dimension direction, and c is the coordinate value in the C dimension direction. The coordinates of the element pointed to by the arrow in the preset coordinate system are: c = 0, row = 0, and col = 0. The channel, width, and height values in the coordinates of other elements increase in the direction of the arrow. Of course, the second original tensor can also include D dimension and N dimension coordinates, which are not shown here.

[0081] It should be noted that in the present disclosure, the coordinate system used by the second original tensor, and the third original tensor, the to-be-processed tensor, etc. in the following text is the same and is the preset coordinate system.

[0082] For example, the preset coordinate system is determined with a certain element in the third original tensor as the origin of the coordinate system. For example, referring to the embodiment as follows Figures 5A-5B , all take the top left corner vertex in the third original tensor as the origin of the preset coordinate system. Of course, the present disclosure is not limited thereto.

[0083] For example, the starting coordinates of the tensor to be processed in the preset coordinate system determined by the third original tensor are, for example, the coordinates of the top-left corner of the tensor to be processed in the embodiment shown in Figures 5A-5B The coordinate value of the starting coordinates of the tensor to be processed is the minimum coordinate value among the coordinate values of all elements in the tensor to be processed.

[0084] For example, the shape size of the second original tensor is represented by tensor_c, tensor_w, tensor_h, tensor_d, and tensor_n, where tensor_n is the N-dimensional size, tensor_d is the D-dimensional size, tensor_h is the H-dimensional size, tensor_w is the W-dimensional size, and tensor_c is the C-dimensional size, in Figure 4 In the example shown in FIG. 6, tensor_n = 1, tensor_d = 1, tensor_h = 4, tensor_w = 8, and tensor_c = 8.

[0085] In step S30, at least one dimension of the second original tensor is size-expanded, and the third original tensor obtained by the size expansion is stored in the memory.

[0086] In the memory, the starting tensor element in each dimension of the third original tensor is located at the starting position of a different storage unit, where the starting tensor element is the tensor element corresponding to the starting coordinates of each dimension in the preset coordinate system, the memory performs read and write operations in the granularity of the storage unit, and the storage unit stores a plurality of tensor elements.

[0087] The tensor element corresponding to the starting coordinates has the minimum coordinate value in the dimension.

[0088] As described above, the memory performs read and write operations in the granularity of the storage unit, all data in each storage unit corresponds to the same memory address, and the capacity of the storage unit can be set as needed, for example, 512B. When reading and writing, a storage unit is read and written as a whole, without considering whether all data in the storage unit is necessary.

[0089] For example, the starting tensor element in each dimension of the third original tensor is located at the starting position of a different storage unit, thereby achieving address alignment of each dimension, so that the data stored by each dimension is an integer multiple of the storage unit capacity, and the situation that after request splitting, the data loaded by each request also needs to read the storage unit containing the data that does not need to be loaded is avoided as much as possible. For example, in the foregoing example, that is, the tensor has a size of 7 in the W dimension, a size of 32 in the C dimension, and the data format of the tensor element is BF16, assuming that after the size expansion in the W dimension, the starting tensor element of each row is located at the starting position of a storage unit, then for a request to load data with a coordinate of h+1 in the H dimension, 0 and 1 in the W dimension, only one storage unit needs to be loaded when loading the data, without loading the storage unit containing the data with a coordinate of h in the H dimension as described above, which greatly reduces invalid data loading, improves bandwidth utilization, effectively utilizes data bandwidth, and improves loading efficiency.

[0090] For example, the expanded size of at least one dimension is determined based on the storage capacity of the storage unit when the size expansion is performed.

[0091] For example, in some embodiments, the size expansion of at least one dimension of the second original tensor can include: for any one dimension in the second original tensor, in response to the total storage capacity of all tensor elements with the same coordinates in the target dimension and above the target dimension in the second original tensor not being an integer multiple of the storage capacity of the storage unit, determining to perform size expansion on the any one dimension, and determining the expanded size of the any one dimension according to the total storage capacity; in response to the total storage capacity being an integer multiple of the storage capacity of the storage unit, determining not to perform size expansion on the any one dimension; wherein in response to the any one dimension being a batch dimension, the target dimension is the any one dimension, and in response to the any one dimension not being a batch dimension, the target dimension is a dimension adjacent to and above the any one dimension indicated by the data storage format of the second original tensor.

[0092] For example, the expanded size of the any one dimension is such that the total storage capacity of all tensor elements with the same coordinates in the target dimension in the third original tensor is an integer multiple of the storage capacity of the storage unit.

[0093] It should be noted that in the present disclosure, the data storage format of the second original tensor, the first original tensor, the third original tensor, and the tensor to be processed is the same, for example, they are all NDHWC, or they are all N(C / x)DHW(xC), where x is a positive integer greater than 1. Taking NDHWC as an example, the C dimension is the lowest dimension during storage, followed by the H dimension, then the W dimension, then the D dimension, and finally the N dimension. Taking N(C / x)DHW(xC) as an example, the W dimension is the lowest dimension during storage, followed by the H dimension, then the D dimension, then the C dimension, and finally the N dimension, and, for the W dimension, continuous xC is actually also considered.

[0094] For example, taking the data storage format of the second original tensor as NDHWC, when any dimension is C, the target dimension is W. If the total storage capacity of all tensor elements with the same coordinate in the W dimension of the second original tensor is not an integer multiple of the storage capacity of the storage unit, the C dimension is expanded. Taking the coordinate w in the W dimension as an example, all tensor elements with coordinate w include tensor_c tensor elements with different coordinates in the C dimension but the same coordinates in other dimensions, where tensor_c is the size of the second original tensor in the C dimension.

[0095] If the total memory capacity of tensor_c tensor elements is not an integer multiple of the memory capacity of the storage unit, then the C dimension is expanded. Specifically, if the expanded size of the C dimension is stride_c, then stride_c × bit_width is an integer multiple of the memory capacity of the storage unit, where bit_width is the bit width of the tensor element, which determines stride_c.

[0096] by Figure 4 For example, the expanded size makes the tensor element with c=0, row=0, col=0 at the starting position of one storage unit, and the tensor element with c=0, row=0, col=1 at the starting position of another storage unit.

[0097] When any dimension is W, the target dimension is H. If the total storage capacity of all tensor elements with the same coordinates in the H dimension of the second original tensor is not an integer multiple of the storage capacity of the storage unit, the W dimension is expanded. Taking the H dimension coordinate as an example, all tensor elements with the same H coordinate and the same coordinates in the D and N dimensions above the H dimension include, for example, a row of tensor elements and all elements of a row of tensor elements in the C dimension direction, that is, stride_c×tensor_w tensor elements, where tensor_w is the size of the second original tensor in the W dimension.

[0098] If the total storage capacity of the stride_c tensor_w tensor elements in the memory is not an integer multiple of the storage capacity of the storage unit, the W dimension is expanded. Specifically, if the size of the W dimension after expansion is stride_w, stride_c stride_w bit_width is an integer multiple of the storage capacity of the storage unit, thereby determining stride_w.

[0099] For example, the expanded size is such that the tensor element of c = 0, row = 0, col = 0 is at the start of a storage unit, and the tensor element of c = 0, row = 1, col = 0 is at the start of another storage unit. Figure 4

[0100] The same applies to the H dimension, the D dimension and the N dimension, which will not be repeated here.

[0101] For example, a third original tensor is obtained in this way, and the shape size of the third original tensor can be represented as stride_c, stride_w, stride_h, stride_d, stride_n, where stride_n is the N dimension size, stride_d is the D dimension size, stride_h is the H dimension size, stride_w is the W dimension size, and stride_c is the C dimension size. The starting tensor element of the third original tensor stored in the memory in each dimension is located at the start of the storage unit, thereby achieving storage address alignment.

[0102] In a specific example, for example, the storage capacity of a single storage unit is 512B, the size of the second original tensor in the W dimension is tensor_w = 7, the size of the second original tensor in the C dimension is tensor_c = 32, and the bit width of the tensor element is bit_width = 16bit. For the W dimension, the storage capacity occupied by a row of pixels is 7x32x2B < 512B, so the W dimension can be expanded in size, for example, to 8, so that 8x32x2B = 512B, that is, the size of the third original tensor in the W dimension is stride_w = 8.

[0103] For example, when storing the third original tensor into the memory, the part formed due to the size expansion that does not belong to the original content of the second original tensor is stored in the memory in a predefined format, where the predefined format is used to indicate the division of the corresponding storage content as data not located in the memory. For example, the predefined format is used to indicate the division of the corresponding storage content as data not located in the memory for subsequent requests, for example, can be regarded as Figure 6 ​The black part in the middle. For example, the predefined format can be specified as needed, for example, it can be a certain special agreed data format, or it is directly empty. The present disclosure does not make specific limitations on this.

[0104] In step S40, the shape information of the to-be-processed tensor and the starting coordinates in the preset coordinate system are received.

[0105] For example, the shape information of the to-be-processed tensor is represented by a1, a2, a3, a4, a5, a1, a2, a3, a4, a5 respectively indicate the size of the to-be-processed tensor in five dimensions and are all positive integers, and the five dimensions include batch dimension, depth dimension, height dimension, width dimension and channel number dimension. In the subsequent embodiments in the present disclosure, a1 is taken as an example of N-dimensional size, a2 is taken as an example of D-dimensional size, a3 is taken as an example of H-dimensional size, a4 is taken as an example of W-dimensional size, and a5 is taken as an example of C-dimensional size.

[0106] For example, the third original tensor is stored in the memory, and the to-be-processed tensor can be a partial tensor in the third original tensor, or the to-be-processed tensor can be the third original tensor itself.

[0107] Figure 5A The schematic diagram of the to-be-processed tensor provided by an embodiment of the present disclosure is shown.

[0108] Figure 5A In the middle, each solid line small cube with a white background and a black frame represents an element stored in the memory and belonging to the second original tensor, and the tensor composed of a plurality of solid line small cubes is the second original tensor. The small cube with a gray black frame represents a part formed due to the foregoing size expansion, which constitutes a part of the third original tensor, but does not belong to the second original tensor, and is stored and represented in the memory in a predefined storage format.

[0109] Figure 5A What is shown is one batch and one tensor at a certain depth in the batch, but the original tensor can have multiple batches, and each batch can have multiple tensors in the depth dimension, which has the same structure as Figure 5A shown, which will not be described again here.

[0110] For example, in the example of Figure 5A , the dashed line frame is the to-be-processed tensor, which includes part of the elements in the third original tensor. The size of the to-be-processed tensor in each dimension is represented by a1, a2, a3, a4, a5. For example, assuming that the N-dimensional size a5 and the D-dimensional size a4 are both equal to 1, Figure 5A In the example of

[0111] For example, in some other embodiments, the to-be-processed tensor comprises at least partial elements of the third original tensor and tensor elements not stored in the memory. For example, in a padding mode, the to-be-processed tensor comprises, in addition to at least partial contents of the third original tensor, a plurality of elements with a predetermined value (for example, 0) added at the edges of the third original tensor. Of course, in some other embodiments, the to-be-processed tensor can even not comprise any element in the third original tensor, and can perform a data loading operation by using a preset coordinate system. Details are not described herein.

[0112] Figure 5A The to-be-processed tensor shown can be used to perform a matrix calculation operation, for example. In some scenarios, the data processing method provided by at least one embodiment of the present disclosure can also be used for a convolution calculation operation.

[0113] As an example, Figure 5B A schematic diagram of acquiring a to-be-processed tensor in a continuous manner according to an embodiment of the present disclosure is shown. In Figure 5B each tensor element is shown as a square block, the third original tensor covers all solid square blocks (that is, both white square blocks and gray shaded square blocks), and the third original tensor also covers dashed square blocks obtained by size expansion in step S30.

[0114] The starting coordinates of the third original tensor are represented as (0, 0, 0), that is, c=0, row=0, and col=0. Then, in Figure 5B the gray shaded square blocks represent the to-be-processed tensor, and the starting coordinates are represented as (c=0, row=0, col=2), that is, data is acquired from the second pixel point in the first row of the third original tensor, and then, in Figure 5B a process of acquiring a to-be-processed tensor in a sequential (or also referred to as a continuous) manner according to an embodiment of the present disclosure is shown schematically, that is, data is acquired from the third original tensor, for example, pixel by pixel, starting from the starting point (c=0, row=0, col=2) shown by the dashed arrow, until the number of acquired pixels reaches copy_pixel_num. It should be noted that, in the present disclosure, a pixel is a tensor description manner without considering the C dimension, that is, the C dimension does not affect the number of pixels.

[0115] In Figure 5B the example, the shape information of the to-be-processed tensor comprises the number of pixels included in the to-be-processed tensor (for example, represented as copy_pixel_num). Figure 5B only three dimensions H, W, and C are shown in the figure, it can be understood that if the data of the three dimensions has not reached the number of pixels included in the to-be-processed tensor, data of higher dimensions can be further acquired, which is not limited herein. In addition, as shown inFigure 5B As shown in the third original tensor in the middle, the C dimension does not affect the number of pixels itself. Assuming, Figure 5B The data storage format of the third original tensor in the memory is in the NDHWC format, and the data storage format of the processed tensor in the memory is in the NCHW format. Figure 5B In the example shown in the left, the total number of pixels of the processed tensor is copy_pixel_num = 24.

[0116] In step S50, in combination with the shape information of the processed tensor and the starting coordinates of the processed tensor, a plurality of requests for loading the processed tensor to the cache area are determined.

[0117] As described above, for the scenario of matrix calculation shown in the left, the shape information of the processed tensor may be, for example, the shape size of the processed tensor in 5 dimensions; for the scenario of convolution calculation shown in the right, the shape information of the processed tensor may be, for example, the number of pixels to be loaded. Figure 5A Figure 5B As described above, for the scenario of matrix calculation shown in the left, the shape information of the processed tensor may be, for example, the shape size of the processed tensor in 5 dimensions; for the scenario of convolution calculation shown in the right, the shape information of the processed tensor may be, for example, the number of pixels to be loaded.

[0118] In step S60, the plurality of requests are sent in sequence, and the sub-data returned by each request is sequentially written to the cache area to load the processed tensor to the cache area.

[0119] For example, the cache area is, for example, a buffer in the stream processor cluster.

[0120] The data processing method provided by at least one embodiment of the present disclosure further divides the data request according to whether the tensor can be continuously loaded in the first dimension, and whether the sub-data loaded by each request is all located in the memory or all not located in the memory, so that the division of the data request is more reasonable and more suitable for the loading and storage of the tensor data, greatly improving the bandwidth and efficiency during data access, and further improving the hardware utilization rate of the computing unit and the hardware performance.

[0121] For example, in response to determining that the processed tensor cannot be continuously loaded in the first dimension but can be continuously loaded in each dimension lower than the first dimension, the division of the request is performed on the second dimension, the sub-data loaded by each request is all located in the memory or all not located in the memory, and in response to the sub-data loaded by the request being all located in the memory, the sub-data loaded by the request is from the second original tensor and continuously stored in the memory, wherein the storage format of the processed tensor indicates that the first dimension is prior to the second dimension during storage or loading, and the first dimension and the second dimension are adjacent.

[0122] ​The object to be loaded, i.e. the tensor to be processed, is split into multiple different requests according to a certain rule and sent in turn, and the returned data is sequentially written into the cache area, thereby efficiently loading the tensor to be processed into the memory. One principle in splitting is that the sub-data loaded by each request is either all in the memory (the sub-data loaded by the request all belong to the data range of the second original tensor) or all not in the memory (the sub-data loaded by the request all do not belong to the data range of the second original tensor), because the sub-data of the request is sent to the memory when it is in the memory, and the sub-data of the request can be converted into an operation such as writing a predetermined value to the cache area by hardware when it is not in the memory. This splitting method can more reasonably send corresponding loading requests to different hardware.

[0123] In addition, when splitting, if it is determined that the tensor to be processed cannot be continuously loaded in the first dimension but can be continuously loaded in each dimension lower than the first dimension when loading the tensor to be processed, further division of the request is performed in the second dimension, so that the request can be more reasonably split, the data continuously stored in the memory is retained as much as possible, the number of requests is reduced, the tensor to be processed is efficiently loaded, the bandwidth when loading or storing data from the memory is greatly improved, the performance is improved, and the efficiency of the hardware computing unit is improved.

[0124] For example, taking NDHWC as an example, the second dimension is the W dimension when the first dimension is the C dimension, or the second dimension is the H dimension when the first dimension is the W dimension, or the second dimension is the D dimension when the first dimension is the H dimension, and so on.

[0125] In response to any one of the following conditions being met, it is determined that the tensor to be processed cannot be continuously loaded in the first dimension:

[0126] (1) The size of the tensor to be processed in the first dimension copy_t is not equal to the size of the second original tensor in the first dimension tensor_t

[0127] (2) The first coordinate value t_coord_b of the starting coordinate of the tensor to be processed in the first dimension is not equal to the second coordinate value of the starting coordinate of the third original tensor in the first dimension

[0128] (3) The distance interval corresponding to the second dimension is not equal to the predetermined value corresponding to the first dimension.

[0129] Here, the distance interval corresponding to the second dimension is the distance interval between two target tensor elements stored in the memory, and the two target tensor elements are two tensor elements adjacent in the second dimension in the third original tensor, and the predetermined value corresponding to the first dimension is determined based on at least the shape size of the second original tensor.

[0130] For example, the distance interval corresponding to the second dimension is determined according to at least a shape size of the third original tensor, in response to the storage format of the third original tensor being NDHWC, a product between shape sizes of respective dimensions lower than the second dimension in the third original tensor is determined as the distance interval corresponding to the second dimension, wherein N represents a batch dimension, D represents a depth dimension, W represents a width dimension, H represents a height dimension, and C represents a channel number dimension.

[0131] Specifically, for NDHWC, the distance interval corresponding to the W dimension is global_stride0=stride_c, that is, a size of the third original tensor in the C dimension. Refer to Figure 5A Or Figure 5B For example, the coordinates of the two target tensor elements can be (c=0, row=1, col=3) and (c=0, row=2, col=3).

[0132] The distance interval corresponding to the H dimension is global_stride1=stride_c×stride_w, that is, a product of sizes of the third original tensor in the C dimension and the W dimension. Refer to Figure 5A Or Figure 5B For example, the coordinates of the two target tensor elements can be (c=0, row=1, col=3) and (c=0, row=2, col=3).

[0133] The distance interval corresponding to the D dimension is global_stride2=stride_c×stride_w×stride_h, that is, a product of sizes of the third original tensor in the C dimension, the W dimension, and the H dimension.

[0134] The distance interval corresponding to the N dimension is global_stride2=stride_c×stride_w×stride_h×stride_d, that is, a product of sizes of the third original tensor in the C dimension, the W dimension, the H dimension, and the D dimension.

[0135] For example, in response to the data storage format of the second original tensor being NDHWC: in a case where the first dimension is a channel number dimension, a shape size of the second original tensor in the first dimension is determined as the predetermined value corresponding to the first dimension; in a case where the first dimension is not a channel number dimension, a product between the shape size of the second original tensor in the first dimension and a distance interval corresponding to the first dimension is determined as the predetermined value corresponding to the first dimension.

[0136] For example, the predetermined value corresponding to the C dimension is tensor_c, that is, a shape size of the second original tensor in the C dimension.

[0137] For example, the predetermined value corresponding to the W dimension is tensor_w x global_stride0, and for example, the predetermined value corresponding to the H dimension is tensor_h x global_stride1, that is, the product of the shape size of the second original tensor in the H dimension and the distance interval corresponding to the H dimension.

[0138] For example, the predetermined value corresponding to the D dimension is tensor_d x global_stride2, that is, the product of the shape size of the second original tensor in the D dimension and the distance interval corresponding to the D dimension.

[0139] Therefore, taking the data storage format as NDHWC and the second coordinate value as 0 as an example, assuming that the first dimension is the C dimension and the second dimension is the W dimension, when any of the following conditions is met, it is determined that the C dimension is discontinuous:

[0140] (1) The starting coordinate c_coord_b of the C dimension of the to-be-processed tensor is not equal to 0

[0141] (2) The size copy_c of the C dimension of the to-be-processed tensor is not equal to the size tensor_c of the C dimension of the second original tensor

[0142] (3) The distance interval global_stride0 corresponding to the W dimension is not equal to the predetermined value tensor_c corresponding to the C dimension

[0143] For example, the sub-data loaded by each request has different coordinates in the C dimension, but has the same coordinates in the W, H, D, and N dimensions. At this time, it can be understood that the third original tensor is unfolded into a one-dimensional vector composed of multiple pixels in the order of WHDN without considering the depth direction, and each sub-data loaded by the request is the data in a pixel, that is, the request is divided in the W dimension, and the next request can be to load the data in the adjacent next pixel, that is, the request is divided in the manner of jumping pixels (referred to as perw).

[0144] Of course, it can be understood that when the following three conditions are met at the same time, it is determined that the C dimension is continuous:

[0145] (1) The starting coordinate c_coord_b of the C dimension of the to-be-processed tensor is equal to 0

[0146] (2) The size copy_c of the C dimension of the to-be-processed tensor is equal to the size tensor_c of the C dimension of the second original tensor

[0147] (3) The distance interval global_stride0 corresponding to the W dimension is equal to the predetermined value tensor_c corresponding to the C dimension

[0148] For example, when the C dimension is continuous, the splitting of the request in a perh manner can be considered.

[0149] Still taking the data storage format as NDHWC and the second coordinate value as 0 as an example, assuming that the first dimension is the W dimension and the second dimension is the H dimension, when any of the following conditions is met, it is determined that the W dimension is discontinuous:

[0150] (1) The starting coordinate of the to-be-processed tensor in the W dimension is not equal to 0

[0151] (2) The size copy_w of the to-be-processed tensor in the W dimension is not equal to the size tensor_w of the second original tensor in the W dimension

[0152] (3) The distance interval global_stride1 corresponding to the H dimension is not equal to the predetermined value tensor_w x global_stride0 corresponding to the W dimension

[0153] At this time, when the C dimension is continuous, the sub-data to be loaded by each request is different in the C dimension and the W dimension, but the coordinates of the H dimension, the D dimension, and the N dimension are the same. At this time, it can be understood that the sub-data loaded by each request belongs to the same row, that is, the requests are divided in the H dimension, and the sub-data loaded by different requests is located in different rows (referred to as perh).

[0154] Of course, it can be understood that when the following three conditions are met at the same time, it is determined that the W dimension is continuous:

[0155] (1) The starting coordinate of the to-be-processed tensor in the W dimension is equal to 0

[0156] (2) The size copy_w of the to-be-processed tensor in the W dimension is equal to the size tensor_w of the second original tensor in the W dimension

[0157] (3) The distance interval global_stride1 corresponding to the H dimension is equal to the predetermined value tensor_w x global_stride0 corresponding to the W dimension

[0158] For example, when the W dimension is continuous, the splitting of the request in a perd manner can be considered.

[0159] Assuming that the first dimension is the H dimension and the second dimension is the D dimension, when any of the following conditions is met, it is determined that the H dimension is discontinuous:

[0160] (1) The starting coordinate of the to-be-processed tensor in the H dimension is not equal to 0

[0161] (2) The size copy_h of the to-be-processed tensor in the H dimension is not equal to the size tensor_h of the second original tensor in the H dimension

[0162] (3) the distance interval global_stride2 corresponding to the D dimension of the to-be-processed tensor is not equal to the predetermined value tensor_hxglobal_stride1 corresponding to the H dimension

[0163] At this time, when the C dimension and the W dimension are continuous, the sub-data to be loaded by each request is different in the coordinates of the C dimension, the W dimension, and the H dimension, but the same in the coordinates of the D dimension and the N dimension (referred to as perd).

[0164] Of course, it can be understood that when the following three conditions are met simultaneously, it is determined that the H dimension is continuous:

[0165] (1) the starting coordinate of the to-be-processed tensor in the H dimension is equal to 0

[0166] (2) the size copy_h of the to-be-processed tensor in the H dimension is equal to the size tensor_h of the second original tensor in the H dimension

[0167] (3) the distance interval global_stride2 corresponding to the D dimension of the to-be-processed tensor is equal to the predetermined value tensor_hxglobal_stride1 corresponding to the H dimension

[0168] For example, when the H dimension is continuous, the splitting of the request in the pern manner can be considered.

[0169] Suppose that the first dimension is the D dimension and the second dimension is the N dimension, when any of the following conditions is met, it is determined that the D dimension is discontinuous:

[0170] (1) the starting coordinate of the to-be-processed tensor in the D dimension is not equal to 0

[0171] (2) the size copy_d of the to-be-processed tensor in the D dimension is not equal to the size tensor_d of the second original tensor in the D dimension

[0172] (3) the distance interval global_stride3 corresponding to the N dimension of the to-be-processed tensor is not equal to the predetermined value tensor_dxglobal_stride2 corresponding to the D dimension

[0173] At this time, when the C dimension, the W dimension, and the H dimension are continuous, the sub-data to be loaded by each request is different in the coordinates of the C dimension, the W dimension, the H dimension, and the D dimension, but the same in the coordinate of the N dimension, that is, the request is divided in the D dimension (referred to as pern).

[0174] Of course, it can be understood that when the following three conditions are met simultaneously, it is determined that the N dimension is continuous:

[0175] (1) the starting coordinate of the to-be-processed tensor in the D dimension is equal to 0

[0176] (2) the size of the to-be-processed tensor in the D dimension copy_d is equal to the size of the second original tensor in the D dimension tensor_d

[0177] (3) the distance interval global_stride3 corresponding to the N dimension of the to-be-processed tensor is equal to the predetermined value tensor_d x global_stride2 corresponding to the D dimension

[0178] At this time, when the C dimension, the W dimension, the H dimension and the D dimension are all continuous, the sub-data to be loaded by each request is the same in the CWHDN dimension coordinates, and for the tensor data located in the memory, in essence, at this time, a request can be sent to the memory to load the data in the memory, and other requests can be used to load data not located in the memory (referred to as per1).

[0179] If the low dimension, that is, the C dimension, the W dimension, the H dimension and the D dimension are not continuous, the N dimension must also be not continuous.

[0180] The request division logic of the to-be-processed tensor with the data storage format of N(C / x)DHW(xC) is the same as that of NDHWC, and the difference lies in that the dimension arrangement of N(C / x)DHW(xC) is different from that of NDHWC. For N(C / x)DHW(xC), since its special interleaving structure is necessarily continuous in (xC), it is considered that the lowest dimension is the W dimension, followed by the H dimension, followed by the D dimension, followed by the C dimension, and the highest is the N dimension.

[0181] Due to the difference in storage format, the way of determining the distance interval corresponding to each dimension and the predetermined value corresponding to each dimension is also different.

[0182] In response to the storage format of the third original tensor being N(C / x)DHW(xC): in the case that the second dimension is not the batch dimension, the product between the shape size of each dimension lower than the second dimension in the third original tensor and the storage capacity of x tensor elements is determined as the distance interval corresponding to the second dimension; in the case that the second dimension is the batch dimension, the product between the shape size of each dimension lower than the second dimension in the third original tensor is determined as the distance interval corresponding to the second dimension.

[0183] Specifically, for N(C / x)DHW(xC), the distance interval global_stride0 corresponding to the H dimension is stride_w x x x bit_width, where x x bit_width represents the storage capacity of x tensor elements, and stride_w is the shape size of the third original tensor in the W dimension.

[0184] The distance interval corresponding to the D dimension is global_stride1=stride_h×stride_w×x×bit_width, where stride_h represents the shape size of the third original tensor in the H dimension.

[0185] The distance interval corresponding to the C dimension is global_stride2=stride_d×stride_h×stride_w×x×bit_width, where stride_d represents the shape size of the third original tensor in the D dimension.

[0186] The distance interval corresponding to the N dimension is global_stride3=stride_c×stride_d×stride_h×stride_w, where stride_c represents the shape size of the third original tensor in the C dimension.

[0187] In response to the storage format of the third original tensor being N(C / x)DHW(xC): in the case that the second dimension is the height dimension, the product of the shape size of the second original tensor in the first dimension and the storage capacity of x tensor elements is determined as the predetermined value corresponding to the first dimension; in the case that the second dimension is the depth dimension or the channel number dimension, the product of the shape size of the second original tensor in the first dimension and the distance interval corresponding to the first dimension is determined as the predetermined value corresponding to the first dimension; in the case that the second dimension is the batch dimension, the quotient between the shape size of the second original tensor in the first dimension and the storage capacity of x tensor elements is determined, and the product of the quotient and the distance interval corresponding to the first dimension is determined as the predetermined value corresponding to the first dimension.

[0188] For example, for N(C / x)DHW(xC), the predetermined value corresponding to the W dimension is tensor_w×x×bit_width, that is, the product between the shape size of the second original tensor in the W dimension and the storage capacity of x tensor elements.

[0189] The predetermined value corresponding to the H dimension is tensor_h×global_stride0, that is, the product between the shape size of the second original tensor in the H dimension and the distance interval corresponding to the H dimension.

[0190] The predetermined value corresponding to the D dimension is tensor_d×global_stride1, that is, the product between the shape size of the second original tensor in the D dimension and the distance interval corresponding to the D dimension.

[0191] The predetermined value corresponding to the C dimension is tensor c / (x*bit width)*global stride2, that is, a quotient between the storage capacity of the shape size x tensor elements of the second original tensor in the C dimension is determined, and the product between the quotient and the distance interval corresponding to the C dimension.

[0192] For example, in the data storage format N(C / x)DHW(xC), taking the second coordinate value 0 as an example, assuming that the first dimension is the W dimension and the second dimension is the H dimension, when any of the following conditions is met, it is determined that the W dimension is discontinuous:

[0193] (1) The starting coordinate of the to-be-processed tensor in the W dimension is not equal to 0

[0194] (2) The size copy_w of the to-be-processed tensor in the W dimension is not equal to the size tensor_w of the second original tensor in the W dimension

[0195] (3) The distance interval global_stride0 corresponding to the H dimension is not equal to the predetermined value tensor_w*x*bit_width corresponding to the W dimension

[0196] At this time, it can be understood that each requested load sub-data belongs to the same row, that is, the requests are divided in the H dimension, and the sub-data loaded by different requests are located in different rows (referred to as perh).

[0197] Of course, it can be understood that when the following three conditions are met at the same time, it is determined that the W dimension is continuous:

[0198] (1) The starting coordinate of the to-be-processed tensor in the W dimension is equal to 0

[0199] (2) The size copy_w of the to-be-processed tensor in the W dimension is equal to the size tensor_w of the second original tensor in the W dimension

[0200] (3) The distance interval global_stride0 corresponding to the H dimension is equal to the predetermined value tensor_w*x*bit_width corresponding to the W dimension

[0201] For example, when the W dimension is continuous, the splitting of the request can be considered in the perd manner.

[0202] Assuming that the first dimension is the H dimension and the second dimension is the D dimension, when any of the following conditions is met, it is determined that the H dimension is discontinuous:

[0203] (1) The starting coordinate of the to-be-processed tensor in the H dimension is not equal to 0

[0204] (2) The size copy_h of the to-be-processed tensor in the H dimension is not equal to the size tensor_h of the second original tensor in the H dimension

[0205] (3) The distance interval global_stride1 corresponding to the tensor to be processed in the D dimension is not equal to the predetermined value tensor_h×global_stride0 corresponding to the H dimension

[0206] In this case, the special interleaving structure is by default continuous in (xC). In addition, when the W dimension is also continuous, the sub-data to be loaded in each request has different coordinates in the C dimension, W dimension, and H dimension, but the coordinates in the D dimension and N dimension are the same (abbreviated as perd).

[0207] Of course, it is understandable that continuity in the H dimension is determined when the following three conditions are met at the same time:

[0208] (1) The starting coordinate of the tensor to be processed in the H dimension is equal to 0

[0209] (2) The size of the tensor to be processed in the H dimension copy_h is equal to the size of the second original tensor in the H dimension tensor_h

[0210] (3) The distance interval global_stride1 corresponding to the tensor to be processed in the D dimension is equal to the predetermined value tensor_h×global_stride0 corresponding to the H dimension

[0211] In addition, for N(C / x)DHW(xC), due to the special nature of the interleaving pattern (xC), pixels are continuous, so the first dimension from low to high is W, H, D, C, and N. For the conditions that cannot be loaded continuously when the data storage format is N(C / x)DHW(xC) and the division principles of requests, please refer to the relevant content of NDHWC and will not be listed here in detail.

[0212] Of course, for the data storage format N(C / x)DHW(xC), although the C dimension of this storage format must be continuous, the request can also be split using the perw method in the NDHWC mode. For example, the instruction carries corresponding information indicating that each request contains data in one pixel, that is, a pixel-skipping method. The perw method can also be used to divide the request of the N(C / x)DHW(xC) storage format. The specific division method is as described above and will not be repeated here.

[0213] The following describes in detail how multiple requests for loading tensors to be processed are determined.

[0214] For example, in some embodiments, step S50 can include: determining, based on the starting coordinates of the tensor to be processed and the shape information of the tensor to be processed, a first request sent in the plurality of requests and an initial state into which the first request enters the state machine; and determining, based on the initial state, each request other than the first request in the plurality of requests by using the state machine in combination with the third original tensor.

[0215] Each request includes three parameters: a data read address for indicating a starting position of reading data from the memory, a data write address for indicating a starting position of writing data to the cache area, and a length of loaded data.

[0216] The data read address of each request is obtained by accumulating and summing all previous request sizes.

[0217] For example, taking the data storage format NDHWC as an example, the calculation formulas of the data read address Addr_1 and the data write address Addr_2 of the nth request are as follows:

[0218] Addr_1 = u_addr_base + n_coord × global_stride3 + d_coord × global_stride2 + h_coord × global_stride1 + w_coord × global_stride0 + c_coord × bit_width (Formula 1)

[0219] Addr_2 = b_addr_base + req_size_1 + req_size_2 + … + req_size_n-1 (Formula 2)

[0220] Addr_2 = b_addr_base + req_size_1 + req_size_2 + … + req_size_n-1 (Formula 2)

[0221] Addr_2 = b_addr_base + req_size_1 + req_size_2 + … + req_size_n-1 (Formula 2)

[0222] Addr_2 = b_addr_base + req_size_1 + req_size_2 + … + req_size_n-1 (Formula 2)

[0223] Addr_2 = b_addr_base + req_size_1 + req_size_2 + … + req_size_n-1 (Formula 2)

[0224] Wherein, u_addr_base represents the storage address of the starting coordinate position element of the third original tensor in the memory, n_coord, d_coord, h_coord, w_coord, c_coord represent the request initial coordinates corresponding to the nth request, global_stride3, global_stride2, global_stride1, global_stride0 represent the distance interval corresponding to each dimension, the specific definition is as described above, b_addr_base represents the starting address of the cache area for writing the to-be-processed tensor, req_size_1, req_size_2,..., req_size_n-1 represent the data length loaded by the first n-1 sent requests respectively.

[0225] For example, taking the data storage format N(C / x)DHW(xC) as an example, the calculation formula of the data read address Addr_3 and the data write address Addr_4 of the nth request is as follows:

[0226] Addr_3= n_coord×global_stride3+

[0227] c_coord / (x×bit_width)×global_stride2+

[0228] d_coord×global_stride1+

[0229] h_coord×global_stride0+

[0230] w_coord×bit_width (Formula 3)

[0231] Addr_4=b_addr_base+req_size_1+ req_size_2+…+ req_size_n-1(Formula 4)

[0232] Wherein, x×bit_width represents the size occupied by x tensor elements. The definitions of other parameters are the same as those in formula 1 and formula 2, and the repeated parts will not be described again.

[0233] For example, in some embodiments, based on the starting coordinates of the tensor to be processed, the shape information of the tensor to be processed, and the data storage format of the tensor to be processed, the first request sent in the plurality of requests and the initial state of the first request entering the state machine are determined, including: determining the initial state of the first request entering the state machine based on the starting coordinates of the tensor to be processed; taking the starting coordinates of the tensor to be processed as the request initial coordinates corresponding to the first request; determining the data read address of the first request according to the distance interval corresponding to each dimension in the third original tensor, the request initial coordinates corresponding to the first request; determining the starting address of the cache area for writing the tensor to be processed as the data write address of the first request; and determining the data length loaded by the first request according to the starting coordinates of the tensor to be processed and the shape information of the tensor to be processed.

[0234] For the first request sent, the request initial coordinates corresponding thereto are the starting coordinates of the tensor to be processed, so that the data read address of the first request can be calculated by referring to the above formula. The starting address of the cache area for writing the tensor to be processed is taken as the data write address of the first request.

[0235] The data length loaded by the first request is determined according to the starting coordinates of the tensor to be processed and the shape information of the tensor to be processed. For example, if the tensor to be processed includes the shape size of the tensor to be processed in 5 dimensions, when the sum of the first coordinate value t_coord_b of the starting coordinates of the tensor to be processed in the first dimension and the shape size copy_t of the tensor to be processed in the first dimension is less than the second coordinate value (i.e., the coordinate value of the starting coordinates of the second original tensor / third original tensor in the first dimension, for example, 0), the data length loaded by the first request is the shape size copy_t of the tensor to be processed in the first dimension. For example, if the sum of the first coordinate value t_coord_b of the starting coordinates of the tensor to be processed in the first dimension and the shape size copy_t of the tensor to be processed in the first dimension is greater than or equal to the second coordinate value, the data length loaded by the first request is the absolute value of the first coordinate value t_coord_b.

[0236] Considering the advantages that the state machine has a clear logical structure, is easy to maintain and expand, and is particularly suitable for processing multi-condition, multi-branch logical scenarios, in the present disclosure, the state machine is adopted to automatically update the request initial coordinates corresponding to each request and the data length loaded, so as to avoid complex conditional nesting, and the logic is clear, easy to maintain, and highly scalable.

[0237] Figure 6 A schematic diagram of the state machine provided by an embodiment of the present disclosure is shown.

[0238] For example, the state machine includes a first state s0, a second state s1, and a third state s2, and the initial state of the state machine is determined by the starting coordinates coord_b of the tensor to be processed.

[0239] For example, in response to the starting coordinate of the tensor to be processed being less than the second coordinate value of the starting coordinate of the second original tensor / third original tensor in the first dimension, the initial state is determined to be the first state; in response to the first coordinate value being greater than or equal to the second coordinate value and less than the third coordinate value, the initial state is determined to be the second state, wherein the difference between the second coordinate value and the third coordinate value is equal to the size of the second original tensor in the first dimension; in response to the first coordinate value being greater than or equal to the third coordinate value, the initial state is determined to be the third state.

[0240] Reference is made to Figure 6 , the tensor_t direction represents data in the first dimension, data with a coordinate in the t dimension less than the second coordinate value and greater than the third coordinate value is not located in the memory (the black part in Figure 6 , data with a coordinate in the t dimension between the second coordinate value and the third coordinate value (the white part in Figure 6 ) is stored in the memory, which is the actual size of the second original tensor data in the first dimension, and the black part can be regarded as content not belonging to the original memory data due to padding, etc.

[0241] In addition, as mentioned above, for the part in the third original tensor not belonging to the original second original tensor due to size expansion, it is also regarded as a black part not located in the memory, and if the tensor to be processed contains the data content of this part, the corresponding request is sent according to the request determination mode of the memory data below.

[0242] The state only switches to itself and adjacent states. For example Figure 6 , the first state s0 can jump to the first state s0 or the second state s1, the second state s1 can jump to the first state s0, the second state s1 and the third state s2, and the third state s2 can jump to the first state s0, the second state s1 and the third state s2.

[0243] Figure 6 The six cases (① to ⑥) in indicate the state jump changes experienced by the tensor to be processed with different sizes in the first dimension.

[0244] Figure 6 For example, , case ① in

[0245] is a case where the first coordinate value is less than the second coordinate value, and the sum of the first coordinate value and the size of the tensor to be processed in the first dimension is less than the second coordinate value, and the state machine always loops in the left first state s0. Figure 6Case 2 in the above table is the case that the first coordinate value is less than the second coordinate value, and the sum of the first coordinate value and the dimension of the tensor to be processed in the first dimension is greater than or equal to the second coordinate value and less than the third coordinate value. In this case, the state machine always loops between the first state s0 and the second state s1.

[0246] For example, Figure 6 Case 3 in the above table is the case that the first coordinate value is greater than or equal to the second coordinate value but less than the third coordinate value, and the sum of the first coordinate value and the dimension of the tensor to be processed in the first dimension is less than the third coordinate value. In this case, the state machine always loops in the second state s1.

[0247] For example, Figure 6 Case 4 in the above table is the case that the first coordinate value is less than the second coordinate value, and the sum of the first coordinate value and the dimension of the tensor to be processed in the first dimension is greater than or equal to the third coordinate value. In this case, the state machine loops between the first state s0, the second state s1, and the third state s2.

[0248] For example, Figure 6 Case 5 in the above table is the case that the first coordinate value is greater than or equal to the second coordinate value, and the sum of the first coordinate value and the dimension of the tensor to be processed in the first dimension is greater than or equal to the third coordinate value. In this case, the state machine loops between the second state s1 and the third state s2.

[0249] For example, Figure 6 Case 6 in the above table is the case that the first coordinate value is greater than or equal to the third coordinate value. In this case, the state machine always loops in the third state s2.

[0250] After the initial state is determined, the request initial coordinate corresponding to the second sent request is determined based on the initial state, and the state entered by the second sent request is determined in combination with the trigger condition. Then, the request initial coordinate corresponding to the third sent request and the data length loaded by the second sent request are determined based on the state entered by the second sent request, and the state entered by the third sent request is determined in combination with the trigger condition, and so on.

[0251] For example, in some embodiments, based on the initial state, in combination with the third original tensor, determining each request in the plurality of requests except the first request by using the state machine can include: determining the request initial coordinate corresponding to each request and the data length loaded by each request by using the state machine based on the initial state; determining the data read address of each request according to the distance interval corresponding to each dimension in the third original tensor and the request initial coordinate corresponding to each request; and determining the data write address of each request according to the data length loaded by each request.

[0252] For example, the state machine outputs the request initial coordinates corresponding to the current request and the data length loaded by the current request in each state, and further prepares the request initial coordinates corresponding to the next request for the next state.

[0253] After determining the request initial coordinates corresponding to the current request, it can be judged whether the sub-data loaded by the current request is located in the memory according to the request initial coordinates.

[0254] For example, in response to the coordinate value of the request initial coordinates corresponding to the current request in the first dimension being greater than or equal to the second coordinate value and less than the third coordinate value, it is determined that the sub-data to be loaded by the current request is all located in the memory. At this time, the data read address and the data write address of the current request can be determined by referring to the above formulas 1-4, and then the current request is sent to the memory to load the corresponding sub-data to the cache area.

[0255] For example, in response to the coordinate value of the request initial coordinates corresponding to the current request in the first dimension being less than the second coordinate value or greater than or equal to the third coordinate value, it is determined that the sub-data to be loaded by the current request is all not located in the memory. At this time, in response to the sub-data to be loaded by the current request being all not located in the memory, the current request is converted to, for example, write a plurality of predetermined values to the cache area using hardware, wherein the number of the plurality of predetermined values is determined by the data length specified by the current request. For example, the predetermined value is 0.

[0256] The process of determining the request initial coordinates and the data length loaded by each request by using the state machine will be described below.

[0257] When the first coordinate value of the to-be-processed tensor in the first dimension is less than the second coordinate value, enter the first state s0.

[0258] For example, when the current request is the first request, the state of the current request is the initial state, the state machine outputs the request initial coordinates corresponding to the current request and the data length loaded by the current request, and prepares the request initial coordinates for the next state.

[0259] For example, in response to the state of the current request being the first state and satisfying the first condition, it is determined that the next request of the current request enters the first state. For example, the first condition includes that the sum of the size copy_t of the to-be-processed tensor in the first dimension and the first coordinate value is less than the second coordinate value. Taking the second coordinate value as 0 as an example, the first condition includes that the size copy_t of the to-be-processed tensor in the first dimension is less than the absolute value of the first coordinate value.

[0260] For example, if the current request is the first state s0 and the size copy_t of the to-be-processed tensor in the first dimension is relatively small, for example, the sum of the size of the to-be-processed tensor in the first dimension and the first coordinate value is less than the second coordinate value, the state of the next request is still the first state s0. For example, as Figure 6Case 1 in FIG. 1 1 : s0->s0.

[0261] In this case, the coordinate value of the request initial coordinate corresponding to the next request in the first dimension is determined to be the first coordinate value t_coord_b, and the coordinate values of the request initial coordinate corresponding to the next request in the other dimensions except the first dimension are updated according to the coordinate values of the other dimensions.

[0262] Specifically, for the other dimensions, the updating manner includes three kinds of self-incrementing (1 or x), returning to the initial coordinate, and being unchanged, for example, for the second dimension, the coordinate of the second dimension of each request is self-incremented, the third dimension and higher dimensions remain unchanged, until reaching the boundary of the tensor to be processed in the second dimension, after which the coordinate of the second dimension is updated back to the starting coordinate and the coordinate of the third dimension is self-incremented, the third dimension is adjacent to the second dimension and higher than the second dimension, and each dimension is incremented in this way. The coordinates of the other dimensions lower than the first dimension remain unchanged.

[0263] For example, in this case, the data length req_size loaded by the current request is the size copy_t of the tensor to be processed in the first dimension.

[0264] For example, in response to the state of the current request being the first state and satisfying a second condition, it is determined that the next request of the current request enters the second state s1. For example, the second condition includes that the size copy_t of the tensor to be processed in the first dimension is greater than or equal to the absolute value of the first coordinate value t_coord_b. For example, as shown in FIG. 1 1, the second condition includes that the size copy_t of the tensor to be processed in the first dimension is greater than or equal to the absolute value of the first coordinate value t_coord_b. Figure 6 s0->s1 in case 2 and case 4 in FIG. 1 1.

[0265] For example, if the current request is in the first state s0 and the size copy_t of the tensor to be processed in the first dimension is relatively large, for example, the sum of the size of the tensor to be processed in the first dimension and the first coordinate value is greater than or equal to the second coordinate value, the state of the next request jumps to the second state s1. For example, assuming that the second coordinate value is 0, the second condition includes that the size copy_t of the tensor to be processed in the first dimension is greater than or equal to the absolute value of the first coordinate value t_coord_b.

[0266] In this case, the coordinate value of the request initial coordinate corresponding to the next request in the first dimension is determined to be the second coordinate value, for example, 0, and the coordinate values of the request initial coordinate corresponding to the next request in the other dimensions except the first dimension are unchanged. Since the first state jumps to the second state, the data in the first dimension has not been loaded completely, so the coordinate values of the request initial coordinate corresponding to the next request in the other dimensions except the first dimension are unchanged.

[0267] For example, in this case, the data length req_size loaded by the current request is determined based on the first coordinate value and the second coordinate value, for example, is the difference between the first coordinate value and the second coordinate value.

[0268] For example, in response to the state of the current request being the second state s1 and the third condition being satisfied, it is determined that the next request of the current request enters the first state s0. The third condition includes that the first coordinate value t_coord_b is less than the second coordinate value and the sum of the first coordinate value t_coord_b and the size copy_t of the to-be-processed tensor in the first dimension is less than the third coordinate value (for example, the second coordinate value is 0 and the third coordinate value is the size tensor_t of the second original tensor in the first dimension), for example, as in case ② of s1->s0 in Figure 6 The sum of the first coordinate value t_coord_b and the size copy_t of the to-be-processed tensor in the first dimension is less than the third coordinate value, so it will not enter the third state s2.

[0269] In this case, it is determined that the coordinate value of the request initial coordinate corresponding to the next request in the first dimension is the first coordinate value t_coord_b, and it is determined that the coordinate values of the request initial coordinate corresponding to the next request in the other dimensions except the first dimension are updated according to the lower coordinate values. Specifically, for the other dimensions, the updating manner includes self-increment (1 or x), return to the initial coordinate, and no change, for example, for the second dimension, the coordinate of the second dimension of each request is self-incremented, the third dimension and higher dimensions remain unchanged, until reaching the boundary of the to-be-processed tensor in the second dimension, after which the coordinate of the second dimension is updated back to the starting coordinate and the coordinate of the third dimension is self-incremented, the third dimension is adjacent to and higher than the second dimension, and each dimension is incremented in this way. For the other dimensions lower than the first dimension, the coordinates remain unchanged.

[0270] For example, in this case, the data length req_size loaded by the current request is determined based on the first coordinate value and the size copy_t of the to-be-processed tensor in the first dimension, for example, if the second coordinate value is 0, the data length req_size loaded by the current request is the sum of the first coordinate value and the size copy_t of the to-be-processed tensor in the first dimension.

[0271] For example, the current request is in the second state s1 and satisfies the fourth condition, it is determined that the next request enters the second state s1. The fourth condition includes that the first coordinate value is greater than or equal to the second coordinate value, and the sum of the first coordinate value and the size of the to-be-processed tensor in the first dimension is less than the third coordinate value, and the difference between the second coordinate value and the third coordinate value is equal to the size of the second original tensor in the first dimension. For example, the current request is in the second state, which indicates that the sub-data of the request are all in the memory, that is, all valid data. When the size of the to-be-processed tensor in the first dimension copy_t is small, for example, the sum of the size of the to-be-processed tensor in the first dimension and the first coordinate value is less than the third coordinate value, the state of the next request is still the second state s1 and will not jump to the third state s2. For example, as in case ③ in FIG. 6. Figure 6

[0272] In this case, it is determined that the coordinate value of the request initial coordinate corresponding to the next request in the first dimension is the first coordinate value t_coord_b, and it is determined that the coordinate values of the request initial coordinate corresponding to the next request in other dimensions except the first dimension are updated according to the lower coordinate values in the other dimensions. Specifically, for the other dimensions, the updating manner includes self-increment (1 or x), return to the initial coordinate, and no change, for example, for the second dimension, the second dimension coordinate of each request is self-incremented, the third dimension and higher dimensions remain unchanged, until the second dimension reaches the boundary of the to-be-processed tensor in the second dimension, and then the second dimension coordinate is updated back to the initial coordinate and the third dimension coordinate is self-incremented, the third dimension is adjacent to the second dimension and higher than the second dimension, and each dimension is incremented in this way. The other dimension coordinates lower than the first dimension remain unchanged.

[0273] For example, in this case, the data length req_size loaded by the current request is the size of the to-be-processed tensor in the first dimension copy_t.

[0274] For example, in response to that the current request is in the second state s1 and satisfies the fifth condition, it is determined that the next request enters the third state s2. The fifth condition includes that the sum of the size of the to-be-processed tensor in the first dimension and the first coordinate value is greater than or equal to the third coordinate value, at this time, the size of the to-be-processed tensor in the first dimension copy_t is relatively large, and the state of the next request jumps to the third state s2. For example, as in s1->s2 in case ④ and case ⑤ in FIG. 6. Figure 6

[0275] In this case, it is determined that the coordinate value of the request initial coordinate corresponding to the next request in the first dimension is the third coordinate value, and it is determined that the coordinate values of the request initial coordinate corresponding to the next request in other dimensions except the first dimension remain unchanged.

[0276] ​​For example, in this case, the data length req_size loaded by the current request is determined based on the coordinate value t_coord of the request initial coordinate corresponding to the current request in the first dimension and the third coordinate value, for example, the data length req_size loaded by the current request is the difference between the third coordinate value and the coordinate value t_coord.

[0277] For example, in response to the state of the current request being the third state, the state entered by the next request of the current request is determined according to the first coordinate value.

[0278] For example, when the first coordinate value is less than the second coordinate value, it is determined that the next request enters the first state s0, for example Figure 6 s2->s0 in case ④ in the above table. When the first coordinate value is greater than or equal to the second coordinate value but less than the third coordinate value, it is determined that the next request enters the second state s1, for example Figure 6 s2->s1 in case ⑤ in the above table. When the first coordinate value is greater than or equal to the third coordinate value, it is determined that the next request enters the third state s2, for example Figure 7 s2->s2 in case ⑥ in the above table.

[0279] In this case, the coordinate value t_coord_b of the request initial coordinate corresponding to the next request in the first dimension is determined to be the first coordinate value, and the coordinate values of the request initial coordinate corresponding to the next request in other dimensions except the first dimension are updated according to the lower coordinate values in other dimensions. Specifically, for other dimensions, the update method includes incrementing (1 or x), returning to the initial coordinate, or remaining unchanged, for example, for the second dimension, the coordinate of the second dimension of each request is incremented until it reaches the boundary of the tensor to be processed in the second dimension, and then the starting coordinate is updated and the third dimension coordinate is incremented, the third dimension is adjacent to and higher than the second dimension, and each dimension is incremented in this way. The coordinate of the lower dimension than the first dimension remains unchanged.

[0280] For example, in this case, the data length req_size loaded by the current request is determined based on the coordinate value t_coord of the request initial coordinate corresponding to the current request in the first dimension and the third coordinate value, for example, the data length req_size loaded by the current request is the difference between the third coordinate value and the coordinate value t_coord.

[0281] In other embodiments, the state machine can also update the remaining size remain_copy_t of the remaining 5 dimensions, which represents the amount of data that has not been loaded in each dimension. When judging the state transition condition of the state machine, the relationship between the fourth coordinate value of the request initial coordinate corresponding to the current request in the first dimension and the remaining size is judged.

[0282] For example, taking the first state s0 jumping to the first state s0 as an example, the first condition can include that the sum of the fourth coordinate value t_coord of the request initial coordinate corresponding to the current request in the first dimension and the remaining size remain_copy_t of the to-be-processed tensor in the first dimension is less than the second coordinate value, and the fourth coordinate value t_coord is less than the second coordinate value. The remaining size remain_copy_t is the number of remaining tensor data in the first dimension of the to-be-processed tensor that has not been requested to be loaded.

[0283] For example, taking the first state s0 jumping to the second state s1 as an example, the second condition can include that the sum of the fourth coordinate value t_coord of the request initial coordinate corresponding to the current request in the first dimension and the remaining size remain_copy_t of the to-be-processed tensor in the first dimension is greater than or equal to the second coordinate value.

[0284] For example, taking the second state s1 jumping to the first state s0 as an example, the third condition can include that the sum of the fourth coordinate value t_coord of the request initial coordinate corresponding to the current request in the first dimension and the remaining size remain_copy_t of the to-be-processed tensor in the first dimension is less than or equal to the third coordinate value, and the first coordinate value is less than the second coordinate value.

[0285] For example, taking the second state s1 jumping to the second state s1 as an example, the fourth condition includes that the sum of the fourth coordinate value and the remaining size is less than the third coordinate value, and the first coordinate value is greater than or equal to the second coordinate value.

[0286] For example, taking the second state s1 jumping to the third state s2 as an example, the fifth condition can include that the sum of the request initial coordinate corresponding to the current request and the remaining size remain_copy_t of the to-be-processed tensor in the first dimension is greater than the third coordinate value.

[0287] For example, for the third state s2, the next request of the current request enters the state according to the first coordinate value, which can be specifically referred to the foregoing content.

[0288] Of course, with the continuous increase of the coordinates, when the remaining sizes of the respective dimensions are all equal to 1, the state machine can end the coordinate determination of the respective requests, and the state machine can enter an idle state.

[0289] The state switching conditions and the updated coordinate contents of the specific state machine can be changed and set according to actual needs, and the logic is similar to the logic of the state machine described above. Here, examples are not given one by one.

[0290] It should be noted that the above process of using the state machine to determine the initial coordinates of the request corresponding to the request and the length of the loaded data is basically the same for NDHWC and N(C / x)DHW(xC). The difference is that in response to the data storage format of the tensor to be processed being NDHWC, the coordinate value of the channel number dimension is automatically incremented by 1 when updated, and in response to the data storage format of the tensor to be processed being N(C / x)DHW(xC), the coordinate value of the channel number dimension is automatically incremented by x when updated.

[0291] For example, in the data processing method provided by at least one embodiment of the present disclosure, step S60 may include: for any request, if all sub-data loaded in response to the any request are located in the memory, sending the any request to the memory; if none of the sub-data loaded in response to the any request are located in the memory, converting the any request into writing a plurality of predetermined values ​​to the cache, wherein the number of the plurality of predetermined values ​​is determined by the length of the data loaded specified by the any request. For example, at this time, the current request is converted to, for example, using hardware to write a plurality of predetermined values ​​to the cache, wherein the number of the plurality of predetermined values ​​is determined by the length of the data loaded specified by the current request, for example, the predetermined value is 0.

[0292] For example, the following describes the data loading process when the first dimension is C, using the data storage format as NDHWC and the shape information of the tensor to be processed including the shape and size of the tensor to be processed in five dimensions. In this example, the first coordinate value is the coordinate value c_coord_b of the starting coordinate of the tensor to be processed in the C dimension, the second coordinate value is 0, and the third coordinate value is the size tensor_c of the second original tensor in the C dimension, and the predetermined value is 0.

[0293] For example, if any one of the following three conditions is met, it is determined that the tensor to be processed cannot be loaded continuously in the C dimension: the size copy_c of the tensor to be processed in the C dimension is not equal to the size tensor_c of the third original tensor in the C dimension, the first coordinate value c_coord_b is not equal to 0, and the distance interval global_stride0 corresponding to the W dimension is equal to the predetermined value stride_c corresponding to the C dimension.

[0294] First, in step S10, a first original tensor is received. The shape and size of the first original tensor are expressed as b1×b2×b3×b4×b5, for example. b1, b2, b3, b4, and b5 respectively indicate the sizes of the original tensor in the five dimensions and are all positive integers. Specifically, b1 represents the C dimension, b2 represents the W dimension, b3 represents the H dimension, b4 represents the D dimension, and b5 represents the N dimension.

[0295] Then, in step S20, the first original tensor is dimensionally transformed to obtain a second original tensor. For example, the W dimension and H dimension of the first original tensor are transposed to obtain a second original tensor. For example, the shape of the second original tensor is represented as b1×b3×b2×b4×b5, where the size of the second original tensor in the C dimension is tensor_c=b1, the size of the second original tensor in the W dimension is tensor_w=b3, the size of the second original tensor in the H dimension is tensor_h=b2, the size of the second original tensor in the D dimension is tensor_d=b4, and the size of the second original tensor in the N dimension is tensor_n=b5.

[0296] Then, in step S30, at least one dimension of the second original tensor is size-expanded, and a third original tensor obtained by the size expansion is stored in the memory.

[0297] The specific process of size expansion and storage can be referred to the above content and will not be repeated here.

[0298] Thereafter, in step S40 , the shape information of the tensor to be processed and the starting coordinates of the tensor to be processed in a preset coordinate system are received.

[0299] For example, in this embodiment, the shape information of the tensor to be processed includes the shape dimensions of the tensor to be processed in five dimensions, namely copy_c / h / w / d / n, etc., and the starting coordinates of the tensor to be processed in the preset coordinate system, namely c / h / w / d / n_coord_b.

[0300] Thereafter, in step S50 , multiple requests for loading the tensors to be processed are determined based on the above information.

[0301] Referring to the process described above, the state machine determines the initial coordinates of each request and the length of the data to be loaded, and accordingly determines the data read address and data write address for each request. For example, in this example, the coordinates of the sub-data loaded by each request are different in the C dimension, but the coordinates in the H, W, D, and N dimensions are the same.

[0302] Table 1 shows the updating process of requesting initial coordinates and requesting loading data length provided by an embodiment of the present disclosure.

[0303] Table 1

[0304]

[0305] As shown in Table 1, if the first coordinate value c_coord_b is less than 0, it is determined that the first request enters the first state s0. The data reading address and the data writing address of the first request refer to the above description and are not repeated here.

[0306] If the first condition is satisfied, it is determined that the next request enters the first state sO. The first condition is described above. As shown in Table 1, at this time, the coordinate value c_coord of the request initial coordinate corresponding to the next request in the C dimension is the first coordinate value c_coord_b; the coordinate value w_coord of the request initial coordinate corresponding to the next request in the W dimension is incremented by 1, and returns to w_coord_b when the boundary in the W dimension of the tensor to be processed is reached; the coordinate value h_coord of the request initial coordinate corresponding to the next request in the H dimension is incremented by 1 when w_coord returns to w_coord_b, and remains unchanged if the boundary in the H dimension of the tensor to be processed is not reached, and returns to h_coord_b when the boundary in the H dimension of the tensor to be processed is reached; the coordinate value d_coord of the request initial coordinate corresponding to the next request in the D dimension is incremented by 1 when h_coord returns to h_coord_b, and remains unchanged if the boundary in the D dimension of the tensor to be processed is not reached, and returns to d_coord_b when the boundary in the D dimension of the tensor to be processed is reached; the coordinate value n_coord of the request initial coordinate corresponding to the next request in the N dimension is incremented by 1 when d_coord returns to d_coord_b, and remains unchanged otherwise.

[0307] At this time, the data length loaded by the first request is the size of the tensor to be processed in the C dimension, copy_c.

[0308] If the second condition is satisfied, it is determined that the next request enters the second state s1. The second condition is described above. At this time, the coordinate value c_coord of the request initial coordinate corresponding to the next request in the C dimension is 0; the coordinate value w_coord of the request initial coordinate corresponding to the next request in the W dimension, the coordinate value h_coord of the request initial coordinate corresponding to the next request in the H dimension, the coordinate value d_coord of the request initial coordinate corresponding to the next request in the D dimension, and the coordinate value n_coord of the request initial coordinate corresponding to the next request in the N dimension are all unchanged.

[0309] At this time, the data length loaded by the first request is the absolute value of the first coordinate value.

[0310] Since the coordinate value c_coord of the request initial coordinate corresponding to the first request in the C dimension is less than 0, none of the sub-data loaded by the first request is in the memory, and in step S60, the first request is converted into an operation of writing copy_c or |c_coord_b| 0s into the cache area, |c_coord_b| representing the absolute value of c_coord_b.

[0311] In addition, it should be noted that if it is determined that the coordinate value c_coord of the request initial coordinate corresponding to the request is less than 0, the data read address and the data write address of the request do not need to be calculated, thereby reducing the amount of calculation.

[0312] As shown in Table 1, when the first coordinate value c_coord_b is greater than or equal to 0 and less than tensor_c, it is determined that the first request enters the second state s1, and the data read address and the data write address of the first request are as described above, and thus will not be described again here.

[0313] If the third condition is met, it is determined that the next request enters the first state s0, and the third condition is as described above. As shown in Table 1, at this time, the coordinate value c_coord of the request initial coordinate corresponding to the next request in the C dimension is the first coordinate value c_coord_b; the coordinate value w_coord of the request initial coordinate corresponding to the next request in the W dimension is incremented by 1, and returns to w_coord_b when the boundary of the W dimension of the to-be-processed tensor is reached; the coordinate value h_coord of the request initial coordinate corresponding to the next request in the H dimension is incremented by 1 when the coordinate value w_coord returns to w_coord_b, and remains unchanged if the boundary of the H dimension of the to-be-processed tensor is not reached, and returns to h_coord_b when the boundary of the H dimension of the to-be-processed tensor is reached; the coordinate value d_coord of the request initial coordinate corresponding to the next request in the D dimension is incremented by 1 when the coordinate value h_coord returns to h_coord_b, and remains unchanged if the boundary of the D dimension of the to-be-processed tensor is not reached, and returns to d_coord_b when the boundary of the D dimension of the to-be-processed tensor is reached; the coordinate value n_coord of the request initial coordinate corresponding to the next request in the N dimension is incremented by 1 when the coordinate value d_coord returns to d_coord_b, and remains unchanged in other cases.

[0314] In addition, at this time, the data length loaded by the first request is the sum of the size copy_c of the to-be-processed tensor in the C dimension and the first coordinate value c_coord_b.

[0315] If the fourth condition is satisfied, it is determined that the next request enters the second state sl, and the fourth condition is described above. As shown in Table 1, at this time, the coordinate value c coord of the request initial coordinate corresponding to the next request in the C dimension is the first coordinate value c coord b; the coordinate value w coord of the request initial coordinate corresponding to the next request in the W dimension is increased by 1, and returns to w coord b when the boundary in the W dimension of the to-be-processed tensor is reached; the coordinate value h coord of the request initial coordinate corresponding to the next request in the H dimension is increased by 1 when w coord returns to w coord b, and remains unchanged if the boundary in the H dimension of the to-be-processed tensor is not reached, and returns to h coord b when the boundary in the H dimension of the to-be-processed tensor is reached; the coordinate value d coord of the request initial coordinate corresponding to the next request in the D dimension is increased by 1 when h coord returns to h coord b, and remains unchanged if the boundary in the D dimension of the to-be-processed tensor is not reached, and returns to d coord b when the boundary in the D dimension of the to-be-processed tensor is reached; the coordinate value n coord of the request initial coordinate corresponding to the next request in the N dimension is increased by 1 when d coord returns to d coord b, and remains unchanged in other cases.

[0316] At this time, the data length loaded by the first request is the size copy c of the to-be-processed tensor in the C dimension.

[0317] If the fifth condition is satisfied, it is determined that the next request enters the third state s2, and the fifth condition is described above. As shown in Table 1, at this time, the coordinate value c coord of the request initial coordinate corresponding to the next request in the C dimension is the third coordinate value tensor c; the coordinate value w coord of the request initial coordinate corresponding to the next request in the W dimension, the coordinate value h coord of the request initial coordinate corresponding to the next request in the H dimension, the coordinate value d coord of the request initial coordinate corresponding to the next request in the D dimension, and the coordinate value n coord of the request initial coordinate corresponding to the next request in the N dimension are all unchanged.

[0318] At this time, the data length loaded by the first request is the difference between the size tensor c of the second original tensor in the C dimension and the coordinate value c coord of the request initial coordinate corresponding to the next request in the C dimension.

[0319] Since the state of the first request is the second state sl, the sub-data loaded by the first request is all located in the memory, and in step S60, the first request is sent to the memory to load the sub-data in the second original tensor stored in the memory to the memory.

[0320] As shown in Table 1, when the first coordinate value c_coord_b is greater than or equal to tensor_c, it is determined that the first request enters the third state s2, and the data read address and the data write address of the first request refer to the foregoing description, which will not be repeated here.

[0321] According to the first coordinate value c_coord_b, the state into which the next request enters is determined, for example, when the first coordinate value c_coord_b is less than 0, it is determined that the next request enters the first state s0, when the first coordinate value c_coord_b is greater than or equal to 0 but less than tensor_c, it is determined that the next request enters the second state s1, and when the first coordinate value c_coord_b is greater than or equal to tensor_c, it is determined that the next request enters the third state s2.

[0322] As shown in Table 1, regardless of the state into which the next request enters, the update logic of the request initial coordinate and the data length loaded by the first request are completely the same.

[0323] For example, taking the next request entering the first state s0 as an example, at this time, the coordinate value c_coord of the request initial coordinate corresponding to the next request in the C dimension is the first coordinate value c_coord_b; the coordinate value w_coord of the request initial coordinate corresponding to the next request in the W dimension is incremented by 1, and when the boundary of the W dimension of the to-be-processed tensor is reached, it is returned to w_coord_b; the coordinate value h_coord of the request initial coordinate corresponding to the next request in the H dimension is incremented by 1 when the boundary of the H dimension of the to-be-processed tensor is reached, and is kept unchanged if the boundary of the H dimension of the to-be-processed tensor is not reached; the coordinate value d_coord of the request initial coordinate corresponding to the next request in the D dimension is incremented by 1 when the boundary of the D dimension of the to-be-processed tensor is reached, and is kept unchanged if the boundary of the D dimension of the to-be-processed tensor is not reached; the coordinate value n_coord of the request initial coordinate corresponding to the next request in the N dimension is incremented by 1 when the boundary of the N dimension of the to-be-processed tensor is reached, and is kept unchanged in other cases. The data length loaded by the first request is the difference between the size tensor_c of the C dimension of the second original tensor and the coordinate value c_coord of the request initial coordinate corresponding to the next request in the C dimension.

[0324] Since the coordinate value c_coord of the request initial coordinate corresponding to the first request in the C dimension is greater than tensor_c, none of the sub-data loaded by the first request is in the memory, and in step S60, the first request is converted into an operation of writing c_coord-tensor_c 0s into the cache area.

[0325] Afterwards, the above process is continued to determine the request initial coordinates and loaded data length corresponding to each subsequent request, and step S60 is executed to send multiple requests. The specific process will not be repeated here.

[0326] In the above embodiments, the data processing method provided by at least one embodiment of the present disclosure supports dimension conversion, reduces the complexity of dimension conversion on data loading, and is stored in memory after dimension conversion, after which the data request can be directly split and sent; size extension is set to ensure address alignment so that the starting tensor element of each dimension is located at the starting position of the storage unit, thereby improving bandwidth utilization, effectively utilizing data bandwidth, and improving loading efficiency. In addition, the present disclosure splits the object to be loaded, that is, the tensor to be processed, into multiple different requests according to a certain rule and sends them in sequence, and writes the returned data sequentially into the cache area, thereby efficiently loading the tensor to be processed into the memory, and the division of data requests is more reasonable, more suitable for the loading and storage of tensor data, greatly improving the bandwidth and efficiency of data memory access, thereby improving the hardware utilization of the computing unit and improving hardware performance. One principle for splitting is that the sub-data requested to be loaded in each request are either all in the memory (the sub-data requested to be loaded all belong to the data range of the second original tensor) or not in the memory (the sub-data requested to be loaded do not belong to the data range of the second original tensor). This is because when the requested sub-data is in the memory, a request is sent to the memory. When the requested sub-data is not in the memory, the request can be converted into an operation such as writing a predetermined value to the cache by the hardware. This splitting method can more reasonably send corresponding loading requests to different hardware.

[0327] At least one embodiment of the present disclosure further provides a data processing device. Figure 7 A schematic structural diagram of a data processing device provided for at least one embodiment of the present disclosure.

[0328] like Figure 3 As shown, the data processing device 100 provided by at least one embodiment of the present disclosure includes a first receiving module 101, a dimension conversion module 102, a size expansion module 103, a second receiving module 104, a request splitting module 105, a sending module 106, a memory 107 and a cache area 108.

[0329] The first receiving module 101 is configured to receive a first original tensor. The first original tensor is a 5-dimensional tensor, where the 5 dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a channel number dimension.

[0330] The dimension conversion module 102 is configured to perform dimension conversion processing on the first original tensor to obtain a second original tensor. The dimension conversion processing includes performing dimension conversion on p dimensions of the five dimensions, where the p dimensions include at least two dimensions of the other four dimensions except the channel number dimension, and p is a positive integer greater than or equal to 2 and less than or equal to 4.

[0331] The size expansion module 103 is configured to perform size expansion on at least one dimension of the second original tensor, and store a third original tensor obtained by the size expansion to the memory 107. In the memory 107, a starting tensor element in each dimension of the third original tensor is located at a starting position of a different storage unit, where the starting tensor element is a tensor element corresponding to a starting coordinate of each dimension in a preset coordinate system, the memory 107 performs read and write operations with a storage unit as a granularity, and the storage unit stores a plurality of tensor elements.

[0332] The second receiving module 104 is configured to receive shape information of a to-be-processed tensor and starting coordinates in a preset coordinate system.

[0333] The request splitting module 105 is configured to determine a plurality of requests for loading the to-be-processed tensor to the cache area 108 in combination with the third original tensor, the shape information of the to-be-processed tensor, and the starting coordinates of the to-be-processed tensor.

[0334] The sending module 106 is configured to sequentially send the plurality of requests, sequentially write sub-data returned by each request to the cache area 108, and load the to-be-processed tensor to the cache area 108.

[0335] For example, the first receiving module 101, the dimension conversion module 102, the size expansion module 103, the second receiving module 104, the request splitting module 105, and the sending module 106 include codes and programs stored in the memory, and the first receiving module 101, the dimension conversion module 102, the size expansion module 103, the second receiving module 104, the request splitting module 105, and the sending module 106 are implemented as, for example, a central processing unit (CPU) or other forms of processing units having data processing and / or instruction execution capabilities. The processing unit can be a general-purpose processor, and also a single-chip microcomputer, a microprocessor, a digital signal processor, a dedicated image processing chip, or a field programmable logic array, etc. The first receiving module 101, the dimension conversion module 102, the size expansion module 103, the second receiving module 104, the request splitting module 105, and the sending module 106 execute the codes and programs to implement some or all functions of the first receiving module 101, the dimension conversion module 102, the size expansion module 103, the second receiving module 104, the request splitting module 105, and the sending module 106 as described above. For example, the first receiving module 101, the dimension conversion module 102, the size expansion module 103, the second receiving module 104, the request splitting module 105, and the sending module 106 can be one circuit board or a combination of multiple circuit boards for implementing the functions as described above. In the embodiments of the present application, the one circuit board or the combination of multiple circuit boards can include: (1) one or more processors; (2) one or more non-transitory memories connected to the processors; and (3) a firmware stored in the memory and executable by the processors.

[0336] It should be noted that the first receiving module 101 can be configured to implement the step S10 shown in FIG. 1; the dimension conversion module 102 can be configured to implement the step S20 shown in FIG. 1; the size expansion module 103 can be configured to implement the step S30 shown in FIG. 1; the second receiving module 104 can be configured to implement the step S40 shown in FIG. 1; the request splitting module 105 can be configured to implement the step S50 shown in FIG. 1; and the sending module 106 can be configured to implement the step S60 shown in FIG. 1. Figure 3 Figure 3 It should be noted that the first receiving module 101 can be configured to implement the step S10 shown in FIG. 1; the dimension conversion module 102 can be configured to implement the step S20 shown in FIG. 1; the size expansion module 103 can be configured to implement the step S30 shown in FIG. 1; the second receiving module 104 can be configured to implement the step S40 shown in FIG. 1; the request splitting module 105 can be configured to implement the step S50 shown in FIG. 1; and the sending module 106 can be configured to implement the step S60 shown in FIG. 1. Figure 3 Figure 3 It should be noted that the first receiving module 101 can be configured to implement the step S10 shown in FIG. 1; the dimension conversion module 102 can be configured to implement the step S20 shown in FIG. 1; the size expansion module 103 can be configured to implement the step S30 shown in FIG. 1; the second receiving module 104 can be configured to implement the step S40 shown in FIG. 1; the request splitting module 105 can be configured to implement the step S50 shown in FIG. 1; and the sending module 106 can be configured to implement the step S60 shown in FIG. 1. Figure 3 Figure 8 ​​​The step S60 is shown. Therefore, for specific description of the functions that the first receiving module 101 can implement, reference can be made to the related description of the step S10 in the above-described embodiments of the data processing method, for specific description of the functions that the dimension conversion module 102 can implement, reference can be made to the related description of the step S20 in the above-described embodiments of the data processing method, for specific description of the functions that the dimension expansion module 103 can implement, reference can be made to the related description of the step S30 in the above-described embodiments of the data processing method, for specific description of the functions that the second receiving module 104 can implement, reference can be made to the related description of the step S40 in the above-described embodiments of the data processing method, for specific description of the functions that the request splitting module 105 can implement, reference can be made to the related description of the step S50 in the above-described embodiments of the data processing method, for specific description of the functions that the sending module 106 can implement, reference can be made to the related description of the step S60 in the above-described embodiments of the data processing method, and the repeated parts will not be described herein again. In addition, the data processing apparatus 100 can achieve similar technical effects as the foregoing data processing method, and the repeated parts will not be described herein again.

[0337] It should be noted that, in at least one embodiment of the present disclosure, the data processing apparatus 100 can include more or fewer circuits or units, and the connection relationship between the circuits or units is not limited, and can be determined according to actual needs. The specific constituting mode of each circuit or unit is not limited, and can be constituted by an analog device according to the circuit principle, or can be constituted by a digital chip, or constituted in other applicable modes.

[0338] For example, the data processing apparatus 100 can be realized in a hardware, software or combination of hardware and software manner, and the present disclosure does not make specific limitation thereon.

[0339] In some embodiments, the data processing apparatus provided by at least one embodiment of the present disclosure can be a chip, for example, the chip is a system on chip (SoC). The system on chip includes a processor which can be a single-core processor or a multi-core processor, a memory and an I / O interface, etc.

[0340] Figure 8 A schematic diagram of a non-transitory computer readable storage medium provided by at least one embodiment of the present disclosure is shown. For example, as shown in Figure 9 The storage medium 200 can be a non-transitory computer readable storage medium, and one or more computer readable instructions 201 can be non-transitorily stored on the storage medium 200. For example, when the computer readable instructions 201 are executed by a processor, one or more steps of the above-described data processing method can be performed.

[0341] For example, the storage medium 200 may be applied to an electronic device 300 . For example, the storage medium 200 may include a storage device 308 in the electronic device 300 .

[0342] For example, the storage device may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, a flash memory, etc. One or more computer-readable instructions may be stored on the computer-readable storage medium, and the processor may execute the computer-readable instructions to implement various functions of the processor. The storage medium may also store various application programs and various data.

[0343] For example, the storage medium may include a memory card of a smart phone, a cache component of a tablet computer, a hard disk of a personal computer, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), flash memory, or any combination of the above storage media, or other applicable storage media.

[0344] Figure 9 This is a schematic block diagram of an electronic device provided by an embodiment of the present disclosure. Figure 9 As shown, the electronic device 300 is suitable for implementing the data processing method provided by the embodiment of the present disclosure. It should be noted that ​ The components of the electronic device 300 shown are merely exemplary and non-limiting. The electronic device 300 may also have other components according to actual application requirements.

[0345] like ​ As shown, the electronic device 300 may include a processing device 301 (eg, a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to non-transitory computer-readable instructions stored in a memory to implement various functions.

[0346] For example, when the computer readable instructions are executed by the processing device 301, one or more steps of the data processing method according to any of the above embodiments may be executed. It should be noted that for a detailed description of the processing process of the data processing method, reference may be made to the relevant descriptions in the above embodiments of the data processing method.

[0347] For example, the memory can include any combination of one or more computer program products, which can include various forms of computer-readable storage media, such as volatile and / or non-volatile memory. Volatile memory, for example, can include random access memory (RAM) 303 and / or cache, etc., from which computer-readable instructions can be loaded and executed, for example, from storage 308 into random access memory (RAM) 303. Non-volatile memory, for example, can include read-only memory (ROM) 302, hard disk, erasable programmable read-only memory (EPROM), compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. Various applications and various data, such as style images, and various data used and / or generated by the applications, etc., can also be stored in the computer-readable storage media.

[0348] For example, the processing device 301, the read-only memory (ROM) 302, and the random access memory (RAM) 303 are connected to each other through a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0349] Generally, the following devices can be connected to the input / output (I / O) interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, a flash memory, etc.; and communication devices 309. The communication devices 309 can allow the electronic device 300 to communicate wirelessly or wired with other electronic devices to exchange data. Although ​ The electronic device 300 is shown with various devices, but it should be understood that not all of the shown devices are required to be implemented or provided, and the electronic device 300 can instead be implemented or provided with more or fewer devices. For example, the processing device 301 can control other components in the electronic device 300 to perform desired functions. The processing device 301 can be a central processing unit (CPU), a tensor processing unit (TPU), or a graphics processing unit (GPU), etc., which has data processing capability and / or program execution capability. The central processing unit (CPU) can be of X86, ARM, RISC-V architecture, etc. The GPU can be directly integrated into a SOC, directly integrated onto a motherboard, or built into a north bridge chip of the motherboard.

[0350] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0351] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit does not necessarily limit the unit itself.

[0352] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0353] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the present disclosure is not limited to technical solutions formed by specific combinations of the aforementioned technical features. It also encompasses other technical solutions formed by any combination of the aforementioned technical features or their equivalents, without departing from the scope of the above disclosure. For example, a technical solution formed by replacing the aforementioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0354] Moreover, while operations are depicted in a particular, chronological sequence in this disclosure, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, at all. To the extent that specific steps are described in a particular order, alternate implementations can perform the operations in different orders, or employ the steps in parallel or concurrently. Additionally, certain features can also be implemented as processes - in places other than those that are described, combined with other processes, or distributed among multiple processes. Similarly, in some circumstances, multimedia instructions can be executed by one or more processors, while in other circumstances, multimedia instructions are executed by a digital signal processor. Further, the description sometimes uses terms like "produce" to describe processes. These terms are high-level descriptions of the actual operations that are performed. The actual operations that correspond to these terms will vary depending on the particular implementation and are appropriately chosen by those persons having ordinary skill in the art, based on their best judgment, to implement the specific functionality as described by the terms. There can also be steps not described that are part of conveying the information. The various steps described herein can be implemented over a wired network or a wireless network.

[0355] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

[0356] For the disclosure, there are the following points to be explained:

[0357] (1) The drawings of the embodiments of the disclosure only involve the structures involved in the embodiments of the disclosure, and other structures can refer to the general design.

[0358] (2) In the case of no conflict, the embodiments of the disclosure and the features in the embodiments can be combined to obtain new embodiments.

[0359] The above only describes the specific implementation of the disclosure, but the protection scope of the disclosure is not limited to this. The protection scope of the disclosure should be subject to the protection scope of the claims.

Claims

1. A data processing method, comprising: Receive a first original tensor, wherein the first original tensor is a 5-dimensional tensor, and the 5 dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a channel number dimension; Performing a dimension conversion process on the first original tensor to obtain a second original tensor, wherein the dimension conversion process includes performing a dimension conversion on p dimensions of the five dimensions, the p dimensions including at least two dimensions of the other four dimensions excluding the channel number dimension, and p is a positive integer greater than or equal to 2 and less than or equal to 4; Performing size expansion on at least one dimension of the second original tensor, and storing a third original tensor obtained by the size expansion in a memory, wherein, in the memory, starting tensor elements on each dimension of the third original tensor are located at starting positions of different storage units, wherein the starting tensor elements are tensor elements corresponding to starting coordinates of each dimension in a preset coordinate system, the memory performs read and write operations with the storage unit as a granularity, and the storage unit stores multiple tensor elements; Receiving shape information of a tensor to be processed and a starting coordinate in the preset coordinate system; Determining, based on the third original tensor, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed, a plurality of requests for loading the tensor to be processed into a cache area; The multiple requests are sent in sequence, and the sub-data returned by each request are sequentially written into the cache area to load the tensor to be processed into the cache area.

2. The data processing method according to claim 1, wherein: The expanded size of the at least one dimension after the size expansion is performed is determined based on the storage capacity of the storage unit.

3. The data processing method according to claim 2, wherein: Expanding the size of at least one dimension of the second original tensor includes: For any dimension of the second original tensor, in response to a total storage capacity of all tensor elements of the second original tensor with the same coordinates in a target dimension and higher than the target dimension not being an integer multiple of the storage capacity of the storage unit, determining to expand the size of the any dimension, and determining the expanded size of the any dimension according to the total storage capacity; In response to the total storage capacity being an integer multiple of the storage capacity of the storage unit, determining not to perform size expansion on any one dimension; In which, in response to any one dimension being the batch dimension, the target dimension is the any one dimension; in response to any one dimension being not the batch dimension, the target dimension is a dimension adjacent to and higher than the any one dimension as indicated by the data storage format of the second original tensor.

4. The data processing method according to claim 3, wherein: The expanded size of any one dimension is such that the total storage capacity of all tensor elements of the third original tensor with the same coordinates in the target dimension is an integer multiple of the storage capacity of the storage unit.

5. The data processing method according to claim 1, wherein: In response to determining that, when loading the to-be-processed tensor, the first dimension cannot be loaded continuously but dimensions lower than the first dimension can be loaded continuously, performing the requested partitioning on the second dimension, wherein each sub-data requested to be loaded is located in the memory or none of the sub-data is located in the memory, and in response to the sub-data loaded by the request are all located in the memory, the sub-data requested to be loaded are from the second original tensor and are stored continuously in the memory, The storage format of the tensor to be processed indicates that the first dimension takes precedence over the second dimension when storing or loading, and the first dimension and the second dimension are adjacent. The data processing method according to claim 1 , wherein: When storing the third original tensor to the memory, the part that does not belong to the original content of the second original tensor due to the size expansion is stored in the memory in a predefined format, wherein the predefined format is used to indicate that the corresponding storage content is requested to be divided as data that is not located in the memory.

7. The data processing method according to claim 5, wherein: In response to satisfying any one of the following conditions, it is determined that the tensor to be processed cannot be loaded continuously in the first dimension: (1) The size of the to-be-processed tensor in the first dimension is not equal to the size of the second original tensor in the first dimension; (2) The first coordinate value of the starting coordinate of the tensor to be processed in the first dimension is not equal to the second coordinate value of the starting coordinate of the third original tensor in the first dimension; (3) The distance interval corresponding to the second dimension is not equal to the predetermined value corresponding to the first dimension, wherein the distance interval corresponding to the second dimension is the distance interval between two target tensor elements stored in the memory, and the two target tensor elements are two tensor elements in the third original tensor that are adjacent in the second dimension, and the predetermined value corresponding to the first dimension is determined based on at least the shape and size of the second original tensor.

8. The data processing method according to claim 7, wherein: The distance interval corresponding to the second dimension is determined at least according to the shape and size of the third original tensor, In response to the storage format of the third original tensor being NDHWC, determining the product of shape sizes of dimensions lower than the second dimension in the third original tensor as the distance interval corresponding to the second dimension, Wherein, N represents the batch dimension, D represents the depth dimension, W represents the width dimension, H represents the height dimension, and C represents the channel number dimension.

9. The data processing method according to claim 7, wherein: In response to the storage format of the third original tensor being N(C / x)DHW(xC): When the second dimension is not the batch dimension, determining a product of a shape size of each dimension lower than the second dimension in the third original tensor and a storage capacity of x tensor elements as a distance interval corresponding to the second dimension; In a case where the second dimension is the batch dimension, determining a product of shape sizes of dimensions lower than the second dimension in the third original tensor as a distance interval corresponding to the second dimension; Wherein, N represents the batch dimension, D represents the depth dimension, W represents the width dimension, H represents the height dimension, C represents the channel number dimension, and x is a positive integer greater than 1.

10. The data processing method according to claim 7, wherein: In response to the second raw tensor, the data storage format is NDHWC: In a case where the first dimension is the channel number dimension, determining a shape size of the second original tensor in the first dimension as a predetermined value corresponding to the first dimension; In a case where the first dimension is not the channel number dimension, determining the product of the shape size of the second original tensor in the first dimension and the distance interval corresponding to the first dimension as the predetermined value corresponding to the first dimension, Wherein, N represents the batch dimension, D represents the depth dimension, W represents the width dimension, H represents the height dimension, and C represents the channel number dimension.

11. The data processing method according to claim 7, wherein: In response to the second original tensor, the data storage format is N(C / x)DHW(xC): In a case where the second dimension is the height dimension, determining a product of a shape size of the second original tensor in the first dimension and a storage capacity of x tensor elements as a predetermined value corresponding to the first dimension; In a case where the second dimension is the depth dimension or the channel number dimension, determining a product of a shape size of the second original tensor in the first dimension and a distance interval corresponding to the first dimension as a predetermined value corresponding to the first dimension; In a case where the second dimension is the batch dimension, determining a quotient between a shape size of the second original tensor in the first dimension and a storage capacity of the x tensor elements, and determining a product of the quotient and a distance interval corresponding to the first dimension as a predetermined value corresponding to the first dimension, Wherein, N represents the batch dimension, D represents the depth dimension, W represents the width dimension, H represents the height dimension, C represents the channel number dimension, and x is a positive integer greater than 1.

12. The data processing method according to any one of claims 1 to 11, wherein: Determining, in combination with the third original tensor, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed, a plurality of requests for loading the tensor to be processed into a cache area, including: Determining, based on the starting coordinates of the tensor to be processed and the shape information of the tensor to be processed, a first request sent among the multiple requests and an initial state in the state machine entered by the first request; Based on the initial state and in combination with the third original tensor, the state machine is used to determine each request among the multiple requests except the first request.

13. The data processing method according to claim 12, wherein: Each request includes a data read address for indicating a starting position for reading data from the memory, a data write address for indicating a starting position for writing data to the cache, and the length of the data requested to be loaded. Determining, based on the starting coordinates of the tensor to be processed and the shape information of the tensor to be processed, a first request sent among the multiple requests and an initial state in the state machine entered by the first request, including: Determining, based on the starting coordinates of the tensor to be processed, that the first request enters the initial state in the state machine; Using the starting coordinates of the tensor to be processed as the initial request coordinates corresponding to the first request; determining a data read address for the first request according to distance intervals corresponding to dimensions in the third original tensor and initial request coordinates corresponding to the first request; Determine a starting address in the cache area for writing the to-be-processed tensor as a data write address for the first request; Determine the data length of the first requested load according to the starting coordinates of the tensor to be processed and the shape information of the tensor to be processed.

14. The data processing method according to claim 13, wherein: Determining, based on the starting coordinates of the tensor to be processed, that the first request enters the initial state in the state machine includes: In response to a first coordinate value of the starting coordinate of the to-be-processed tensor in the first dimension being smaller than a second coordinate value of the starting coordinate of the third original tensor in the first dimension, determining that the initial state is the first state, In response to the first coordinate value being greater than or equal to the second coordinate value and less than a third coordinate value, determining the initial state to be a second state, wherein a difference between the second coordinate value and the third coordinate value is equal to a size of the second original tensor in the first dimension, In response to the first coordinate value being greater than or equal to the third coordinate value, the initial state is determined to be the third state.

15. The data processing method according to claim 12, wherein: Determining, based on the initial state and in combination with the third original tensor, each request other than the first request in the plurality of requests using the state machine, includes: Based on the initial state, using the state machine to determine the request initial coordinates corresponding to each request and the data length loaded by each request; Determining a data read address for each request according to a distance interval corresponding to each dimension in the third original tensor and an initial request coordinate corresponding to each request; The data write address of each request is determined according to the length of the data loaded by each request.

16. The data processing method according to any one of claims 1 to 11, wherein: Performing dimension conversion on the first original tensor to obtain a second original tensor includes: Perform a transposition operation on the first original tensor in the p dimensions to obtain the second original tensor.

17. An electronic device comprising: a memory that non-transitorily stores computer-executable instructions; a processor configured to execute the computer-executable instructions, Wherein, when the computer executable instructions are executed by the processor, the data processing method according to any one of claims 1 to 16 is implemented.

18. A non-transitory computer-readable storage medium, wherein: The non-transitory computer-readable storage medium stores computer-executable instructions, When the computer-executable instructions are executed by a processor, the data processing method according to any one of claims 1 to 16 is implemented.

Images