Data processing method, electronic equipment and storage medium

Abstract

The invention discloses a data processing method, electronic equipment and a storage medium. The data processing method comprises the following steps: receiving a first original tensor; performing dimension conversion processing on the first original tensor to obtain a second original tensor; carrying out size expansion on at least one dimension of the second original tensor, and storing a third original tensor obtained by size expansion into a memory; receiving shape information of the tensor to be processed and an initial coordinate in a preset coordinate system; in combination with the third original tensor, the shape information of the tensor to be processed and the initial coordinate of the tensor to be processed, determining a plurality of requests for loading the tensor to be processed to the cache region; and sending a plurality of requests in sequence, and writing the sub-data returned by each request into the cache region in sequence to load the tensor to be processed to the cache region. According to the method, dimension conversion is supported, the data loading complexity of dimension conversion is reduced, the bandwidth utilization rate is improved, the data bandwidth is effectively utilized, and the loading efficiency is improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to a data processing method, an electronic device, and a non-transitory computer-readable storage medium. Background Art

[0002] A tensor is a multilinear map defined on the Cartesian product of some vector space and some dual space. For example, a scalar can be viewed as a 0-dimensional tensor, a vector as a 1-dimensional tensor, and a matrix as a 2-dimensional tensor. Tensors can have any number of dimensions. Tensor operations are widely used in processors such as parallel processors.

[0003] The development of artificial intelligence and machine learning has placed new demands on numerous parallel processor devices, such as multi-core processors and digital signal processors. In general-purpose computing, the computational units of parallel processors require large amounts of data, which is typically stored in the parallel processor's storage components, such as memory. Data load instructions extract this data from the storage components into a cache for computation, and data store instructions store the data in the cache into memory.

[0004] Current data storage or loading methods waste data transmission bandwidth, reduce bandwidth utilization, and fail to effectively utilize data bandwidth. Summary of the Invention

[0005] At least one embodiment of the present disclosure provides a data processing method, receiving a first original tensor, wherein the first original tensor is a 5-dimensional tensor, and the 5 dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a channel number dimension; performing dimension conversion processing on the first original tensor to obtain a second original tensor, wherein the dimension conversion processing includes performing dimension conversion on p dimensions of the 5 dimensions, and the p dimensions include at least two dimensions of the other four dimensions except the channel number dimension, and p is a positive integer greater than or equal to 2 and less than or equal to 4; performing size expansion on at least one dimension of the second original tensor, and storing a third original tensor obtained by the size expansion in a memory, wherein, in the memory, the third The starting tensor elements on each dimension in the original tensor are located at the starting positions of different storage units, wherein the starting tensor elements are tensor elements corresponding to the starting coordinates of each dimension in the preset coordinate system, and the memory performs read and write operations with the storage unit as the granularity, and the storage unit stores multiple tensor elements; receiving the shape information of the tensor to be processed and the starting coordinates in the preset coordinate system; combining the third original tensor, the shape information of the tensor to be processed and the starting coordinates of the tensor to be processed, determining multiple requests for loading the tensor to be processed into the cache area; sending the multiple requests in sequence, and writing the sub-data returned by each request into the cache area in sequence to load the tensor to be processed into the cache area.

[0006] For example, in the data processing method provided in at least one embodiment of the present disclosure, the expanded size of the at least one dimension after the size expansion is performed is determined based on the storage capacity of the storage unit.

[0007] For example, in the data processing method provided in at least one embodiment of the present disclosure, at least one dimension of the second original tensor is size-expanded, including: for any dimension of the second original tensor, in response to the total storage capacity of all tensor elements of the second original tensor with the same coordinates in a target dimension and above the target dimension not being an integer multiple of the storage capacity of the storage unit, determining to size-expand the any dimension, and determining the size of the any dimension after expansion based on the total storage capacity; in response to the total storage capacity being an integer multiple of the storage capacity of the storage unit, determining not to size-expand the any dimension; wherein, in response to the any dimension being the batch dimension, the target dimension is the any dimension, and in response to the any dimension not being the batch dimension, the target dimension is a dimension adjacent to and higher than the any dimension as indicated by the data storage format of the second original tensor. For example, in the data processing method provided in at least one embodiment of the present disclosure, the size of the any dimension after expansion is such that the total storage capacity of all tensor elements of the third original tensor with the same coordinates in the target dimension is an integer multiple of the storage capacity of the storage unit.

[0008] For example, in the data processing method provided by at least one embodiment of the present disclosure, in response to determining that when loading the tensor to be processed, continuous loading cannot be performed in the first dimension but continuous loading can be performed in dimensions lower than the first dimension, the request is divided on the second dimension, and each sub-data requested to be loaded is located in the memory or not located in the memory at all, and the sub-data loaded in response to the request are all located in the memory, the sub-data requested to be loaded come from the second original tensor and are stored continuously in the memory, wherein the storage format of the tensor to be processed indicates that the first dimension takes precedence over the second dimension when storing or loading, and the first dimension and the second dimension are adjacent.

[0009] For example, in the data processing method provided in at least one embodiment of the present disclosure, when the third original tensor is stored in the memory, the part that does not belong to the original content of the second original tensor due to the size expansion is stored in the memory in a predefined format, wherein the predefined format is used to indicate that the corresponding storage content is requested to be divided as data that is not located in the memory.

[0010] For example, in the data processing method provided in at least one embodiment of the present disclosure, in response to satisfying any one of the following conditions, it is determined that the tensor to be processed cannot be loaded continuously in the first dimension: (1) the size of the tensor to be processed in the first dimension is not equal to the size of the second original tensor in the first dimension; (2) the first coordinate value of the starting coordinate of the tensor to be processed in the first dimension is not equal to the second coordinate value of the starting coordinate of the third original tensor in the first dimension; (3) the distance interval corresponding to the second dimension is not equal to the predetermined value corresponding to the first dimension, wherein the distance interval corresponding to the second dimension is the distance interval between two target tensor elements stored in the memory, and the two target tensor elements are two adjacent tensor elements in the third original tensor in the second dimension, and the predetermined value corresponding to the first dimension is determined based on at least the shape size of the second original tensor.

[0011] For example, in the data processing method provided in at least one embodiment of the present disclosure, the distance interval corresponding to the second dimension is determined at least based on the shape size of the third original tensor. In response to the storage format of the third original tensor being NDHWC, the product of the shape sizes of each dimension in the third original tensor lower than the second dimension is determined as the distance interval corresponding to the second dimension, where N represents the batch dimension, D represents the depth dimension, W represents the width dimension, H represents the height dimension, and C represents the channel number dimension.

[0012] For example, in the data processing method provided by at least one embodiment of the present disclosure, in response to the storage format of the third original tensor being N(C / x)DHW(xC): when the second dimension is not the batch dimension, the product of the shape size of each dimension lower than the second dimension in the third original tensor and the storage capacity of x tensor elements is determined as the distance interval corresponding to the second dimension; when the second dimension is the batch dimension, the product of the shape size of each dimension lower than the second dimension in the third original tensor is determined as the distance interval corresponding to the second dimension; wherein N represents the batch dimension, D represents the depth dimension, W represents the width dimension, H represents the height dimension, C represents the channel number dimension, and x is a positive integer greater than 1.

[0013] For example, in the data processing method provided in at least one embodiment of the present disclosure, in response to the data storage format of the second original tensor being NDHWC: when the first dimension is the channel number dimension, the shape size of the second original tensor in the first dimension is determined as the predetermined value corresponding to the first dimension; when the first dimension is not the channel number dimension, the product of the shape size of the second original tensor in the first dimension and the distance interval corresponding to the first dimension is determined as the predetermined value corresponding to the first dimension.

[0014] For example, in the data processing method provided by at least one embodiment of the present disclosure, in response to the data storage format of the second original tensor being N(C / x)DHW(xC): when the second dimension is the height dimension, the product of the shape size of the second original tensor in the first dimension and the storage capacity of x tensor elements is determined as the predetermined value corresponding to the first dimension; when the second dimension is the depth dimension or the channel number dimension, the product of the shape size of the second original tensor in the first dimension and the distance interval corresponding to the first dimension is determined as the predetermined value corresponding to the first dimension; when the second dimension is the batch dimension, the quotient between the shape size of the second original tensor in the first dimension and the storage capacity of the x tensor elements is determined, and the product of the quotient and the distance interval corresponding to the first dimension is determined as the predetermined value corresponding to the first dimension.

[0015] For example, in the data processing method provided in at least one embodiment of the present disclosure, in combination with the third original tensor, the shape information of the tensor to be processed and the starting coordinates of the tensor to be processed, multiple requests for loading the tensor to be processed into the cache area are determined, including: based on the starting coordinates of the tensor to be processed and the shape information of the tensor to be processed, determining the first request sent among the multiple requests and the initial state of the first request entering the state machine; based on the initial state, in combination with the third original tensor, using the state machine to determine each request among the multiple requests except the first request.

[0016] For example, in the data processing method provided by at least one embodiment of the present disclosure, each request includes a data read address for indicating the starting position for reading data from the memory, a data write address for indicating the starting position for writing data to the cache area, and the data length loaded by the request, and based on the starting coordinates of the tensor to be processed and the shape information of the tensor to be processed, determining the first request sent among the multiple requests and the initial state of the first request entering the state machine, including: determining the initial state of the first request entering the state machine based on the starting coordinates of the tensor to be processed; using the starting coordinates of the tensor to be processed as the request initial coordinates corresponding to the first request; determining the data read address of the first request according to the distance intervals corresponding to each dimension in the third original tensor and the request initial coordinates corresponding to the first request; determining the starting address of writing the tensor to be processed in the cache area as the data write address of the first request; determining the data length loaded by the first request according to the starting coordinates of the tensor to be processed and the shape information of the tensor to be processed.

[0017] For example, in the data processing method provided by at least one embodiment of the present disclosure, based on the starting coordinates of the tensor to be processed, determining the initial state of the first request to enter the state machine includes: in response to the first coordinate value of the starting coordinate of the tensor to be processed in the first dimension being less than the second coordinate value of the starting coordinate of the third original tensor in the first dimension, determining the initial state to be the first state; in response to the first coordinate value being greater than or equal to the second coordinate value and less than the third coordinate value, determining the initial state to be the second state, wherein the difference between the second coordinate value and the third coordinate value is equal to the size of the second original tensor in the first dimension; in response to the first coordinate value being greater than or equal to the third coordinate value, determining the initial state to be the third state.

[0018] For example, in the data processing method provided in at least one embodiment of the present disclosure, based on the initial state and in combination with the third original tensor, the state machine is used to determine each request among the multiple requests except the first request, including: based on the initial state, using the state machine to determine the request initial coordinates corresponding to each request and the data length loaded by each request; determining the data read address of each request according to the distance interval corresponding to each dimension in the third original tensor and the request initial coordinates corresponding to each request; determining the data write address of each request according to the data length loaded by each request.

[0019] For example, in the data processing method provided in at least one embodiment of the present disclosure, dimension conversion processing is performed on the first original tensor to obtain the second original tensor, including: performing a transpose operation on the first original tensor in the p dimensions to obtain the second original tensor.

[0020] At least one embodiment of the present disclosure provides an electronic device, comprising: a memory, which non-transitorily stores computer-executable instructions; and a processor, configured to execute the computer-executable instructions, wherein the computer-executable instructions, when executed by the processor, implement the data processing method according to at least one embodiment of the present disclosure.

[0021] At least one embodiment of the present disclosure provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the data processing method according to at least one embodiment of the present disclosure is implemented.

[0022] In the data processing method provided in at least one embodiment of the present disclosure, dimension conversion is supported, which reduces the complexity of dimension conversion on data loading; size extension is set to ensure address alignment so that the starting tensor element of each dimension is located at the starting position of the storage unit, thereby improving bandwidth utilization, effectively utilizing data bandwidth, and improving loading efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, rather than limiting the present disclosure.

[0024] Figure 1 A schematic diagram of the structure of a general-purpose graphics processing unit (GPGPU); Figure 2A is a schematic structure of a tensor; Figure 2B This is a schematic diagram of the storage format of NDHWC; Figure 2C This is a schematic diagram of the storage format of N(C / 32)DHW(32C); Figure 3 A schematic flow chart of a data processing method provided in at least one embodiment of the present disclosure; Figure 4 A schematic structural diagram of a second original tensor provided for at least one embodiment of the present disclosure; Figure 5A A schematic diagram of a tensor to be processed provided in one embodiment of the present disclosure; Figure 5B A schematic diagram of a tensor to be processed provided in another embodiment of the present disclosure; Figure 6 A schematic diagram of a state machine provided in one embodiment of the present disclosure; Figure 7 A schematic structural diagram of a data processing device provided for at least one embodiment of the present disclosure; Figure 8 A schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure; Figure 9 An embodiment of the present disclosure provides a schematic block diagram of an electronic device. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0026] Unless otherwise defined, the technical or scientific terms used in this disclosure should have the usual meanings understood by people with ordinary skills in the field to which this disclosure belongs. The "first", "second" and similar words used in this disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits detailed descriptions of some known functions and known components.

[0027] Figure 1 A schematic structural diagram of a general-purpose graphics processing unit (GPGPU).

[0028] like Figure 1 As shown, the general purpose graphics processor is actually an array of programmable multiprocessors. For example, the programmable multiprocessor can be a streaming processor cluster (SPC), including Figure 1Streaming processor clusters 1, ..., and M are shown, where M is a positive integer greater than 1. In a general-purpose graphics processor, one streaming processor cluster processes one computing task, or multiple streaming processor clusters process one computing task. Multiple streaming processor clusters share data through a global cache or global memory.

[0029] like Figure 1 As shown, taking stream processor cluster 1 as an example, a stream processor cluster includes multiple computing units, such as Figure 1 In the CU, there are CU 1, CU 2, ..., CU N, where N is a positive integer. Each CU is used to perform arithmetic and logical operations, such as accumulation, reduction, conventional addition, subtraction, multiplication, and division. A CU includes multiple cores (also called computing cores or computing cores), each of which includes an arithmetic logic unit (ALU), a floating-point computing unit, etc. The computing core is used to perform specific computing tasks. In addition, the CU also includes registers (e.g. Figure 1 The register file in the computing unit and the shared memory are used to hierarchically store source data and destination data related to computing tasks. The shared memory in a computing unit is used to share data between the cores of the computing unit.

[0030] like Figure 1 As shown, each stream processor cluster also provides a buffer for caching data of N computing units in the stream processor cluster.

[0031] In parallel computing, computing tasks are generally executed by multiple threads. These threads are divided into multiple thread blocks before being executed in a general-purpose graphics processor (or parallel computing processor), and then distributed through the thread block distribution module ( Figure 1 (Not shown) Multiple thread blocks are distributed to various CUs. All threads in a thread block must be assigned to the same CU for execution. Thread blocks are also split into minimum execution warps (or simply warps). Each warp contains a fixed number (or less) of threads, for example, 32 threads. Multiple thread blocks can execute in the same CU or in different CUs.

[0032] In each computational unit, the warp scheduling / dispatching module ( Figure 1(not shown) schedules and allocates thread warps so that multiple cores of the compute unit can execute the warps. Depending on the number of cores in the compute unit, multiple warps in a thread block can execute simultaneously or in a time-sharing manner. Multiple threads in each warp execute the same instruction. Memory execution instructions are emitted to the shared memory of the compute unit or further emitted to the intermediate level cache, global cache, or global memory (e.g., Figure 1 The high bandwidth memory (HBM) in the memory is used for read and write operations.

[0033] like Figure 1 As shown, general computing operations, such as matrix operations or convolution operations, typically require large amounts of data. This data is typically stored in memory, such as HBM. During general computing operations, data must be loaded from memory (a load operation), and when the computation results are obtained, data must be stored back into memory (a store operation). The way data is stored in memory affects memory access bandwidth, which in turn affects the hardware utilization of the computing unit.

[0034] For example, the shape and size of a tensor can be represented by a1, a2, a3, a4, and a5, where a1, a2, a3, a4, and a5 respectively indicate the size of the tensor data in five dimensions, and a1, a2, a3, a4, and a5 are positive integers. For example, the five dimensions include [N, D, H, W, C], where the N dimension represents the batch size, that is, the number of data samples captured for one training session, the D dimension represents the depth, the H dimension represents the height, the W dimension represents the width, and the C dimension represents the number of channels. For example, taking the first tensor as an example, a1 can be an N-dimensional size, a2 can be a D-dimensional size, a3 can be an H-dimensional size, a4 can be a W-dimensional size, and a5 can be a C-dimensional size. Of course, the present disclosure does not impose any specific restrictions on this.

[0035] For example, a general computing operation includes a general matrix multiplication (GEMM). The data to be subjected to the general matrix multiplication includes a first tensor and a second tensor.

[0036] For example, general computing operations may include convolution operations, where the data required for the convolution operation includes a convolution kernel and input data (activation). For example, the input data may be an input image that requires the convolution operation. Convolution operations may include convolution multiplication calculations commonly used in neural network processing (e.g., convolutional neural networks) in fields such as artificial intelligence, such as multi-channel convolution and multi-dimensional convolution.

[0037] The elements of a tensor are stored in memory (e.g. Figure 1 There are many formats for placing tensors in memory, called data storage formats (layouts). The data storage format is used to indicate the storage order and dimension arrangement of tensors in the storage component.

[0038] For example, Figure 2A is a schematic structure of a tensor. Figure 2A In the tensor shown, a1 is the N dimension and is equal to 1, a2 is the D dimension and is equal to 1, a3 is the H dimension and is equal to 5, a4 is the W dimension and is equal to 4, and a5 is the C dimension and is equal to 64.

[0039] For example, Figure 2A The pixel elements of the tensor in are represented as 0, 1, 2, 3, ..., and so on. Figure 2A The tensors shown describe different data storage formats.

[0040] For example, the data storage format may include NDHWC, also known as Linear mode. Figure 2B This is a schematic diagram of the storage format of NDHWC.

[0041] For example, for NDHWC, Figure 2B As shown, from the first channel (a5=0, Figure 2B The first element of c0) in ( Figure 2B 0 in the , and then stores the second channel (a5=1, Figure 2B The first element of c1 in ( Figure 2B 20), and so on, until the first elements of all channels are laid out, for example, to the 64th channel (a5=63, Figure 2B The first element of c63) ( Figure 2B After the element 1260 in the , select the first channel (a5=0, Figure 2B The second element of c0 in ( Figure 2B 1 in the array), and then store the second channel (a5=1, Figure 2B The second element of c1 in ( Figure 2B ), and so on, until the second elements of all channels are laid out, and so on.

[0042] For example, the data storage format may further include N(C / x)DHW(xC), also known as an interleave mode, where x may be 8, 16, 32, etc. as needed.

[0043] N(C / x)DHW(xC) is similar to NDHWC, but with one key difference: in the memory layout of N(C / x)DHW(xC), the a5 channels are divided into a5 / x groups, each with x channels: the first group consists of channels a5=0 to a5=x-1, and the second group consists of channels a5=x to a5=2x-1, and each group is arranged in NDHWC format.

[0044] Figure 2C This is a schematic diagram of the storage format of N(C / 32)DHW(32C).

[0045] like Figure 2C As shown, the 64 channels are divided into two groups, each with 32 channels. The first group consists of channel a5=0 ( Figure 2C c0) to a5=31 ( Figure 2C The first group consists of channels a5=32 to a5=63. Each group is then arranged in NDHWC format.

[0046] When extracting or storing part of the original tensor, each dimension may not be extracted continuously during extraction. When loading or storing data, the requested data volume and data location cannot be accurately obtained to optimally load or store data, which greatly reduces the data bandwidth when loading data and reduces hardware computing efficiency.

[0047] The storage capacity of a memory cell corresponding to a storage address in data memory is typically greater than the bit width of a single tensor element. Taking HBM as an example, the storage capacity of a memory cell corresponding to a storage address in HBM is 512B. This means that a memory cell corresponding to a single address can typically store multiple tensor elements. For example, if the tensor elements are in FP16 data format, a single memory cell can store up to 256 tensor elements.

[0048] Currently, tensors are stored contiguously in memory. For example, assuming a tensor has a W dimension of 7, a C dimension of 32, and a BF16 data format for tensor elements, and the NDHWC storage format is used as an example, a single storage unit stores not only all elements with coordinate h in the H dimension, but also the 32 tensor elements with coordinate h+1 in the H dimension, 0 in the W dimension, and coordinates from 0 to 31 in the C dimension. In other words, the data stored in a single storage unit has changed in the H dimension.

[0049] Assume that the data to be loaded includes data with H-dimensional coordinates h+1 and W-dimensional coordinates 0 and 1. When loading this data, it is necessary not only to read data from the storage unit storing H-dimensional coordinates h+1 and W-dimensional coordinates 0, but also from another storage unit storing H-dimensional coordinates h+1 and W-dimensional coordinates 1. For memory, reading data from the beginning of the address is more hardware-friendly, while reading data from the middle of the storage unit reduces execution efficiency.

[0050] Furthermore, HBM typically operates on a per-memory cell granularity when loading data from memory to the cache (or vice versa). Therefore, all data in a memory cell is sent to the cache regardless of whether it is needed. Therefore, in the example above, the loaded data includes the entire data in at least two memory cells, thus including a significant amount of data that is not actually needed. This wastes data transmission bandwidth, reduces bandwidth utilization, and fails to effectively utilize data bandwidth.

[0051] In addition, in some scenarios, it is often necessary to perform dimension conversion operations on tensors of arbitrary dimensions, but the current loading method is not compatible with tensor dimension conversion.

[0052] At least one embodiment of the present disclosure provides a data processing method, an electronic device, and a non-transitory computer-readable storage medium.

[0053] The data processing method provided by at least one embodiment of the present disclosure includes: receiving a first original tensor, wherein the first original tensor is a 5-dimensional tensor, and the 5 dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a channel number dimension; performing dimension conversion processing on the first original tensor to obtain a second original tensor, wherein the dimension conversion processing includes performing dimension conversion on p dimensions in the 5 dimensions, and the p dimensions include at least two dimensions of the other four dimensions except the channel number dimension, and p is a positive integer greater than or equal to 2 and less than or equal to 4; performing size expansion on at least one dimension of the second original tensor, and storing a third original tensor obtained by the size expansion into a memory, wherein in the memory , the starting tensor elements on each dimension in the third original tensor are located at the starting positions of different storage units, wherein the starting tensor elements are tensor elements corresponding to the starting coordinates of each dimension in the preset coordinate system, and the memory performs read and write operations with the storage unit as the granularity, and a plurality of tensor elements are stored in the storage unit; receiving the shape information of the tensor to be processed and the starting coordinates in the preset coordinate system; combining the third original tensor, the shape information of the tensor to be processed and the starting coordinates of the tensor to be processed, determining a plurality of requests for loading the tensor to be processed into the cache area; sending a plurality of requests in sequence, and writing the sub-data returned by each request into the cache area in sequence to load the tensor to be processed into the cache area.

[0054] In the data processing method provided in at least one embodiment of the present disclosure, dimension conversion is supported, which reduces the complexity of dimension conversion on data loading; size extension is set to ensure address alignment so that the starting tensor element of each dimension is located at the starting position of the storage unit, thereby improving bandwidth utilization, effectively utilizing data bandwidth, and improving loading efficiency.

[0055] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings, but the present disclosure is not limited to these specific embodiments.

[0056] Figure 3 A schematic flowchart of a data processing method provided in at least one embodiment of the present disclosure.

[0057] like Figure 3 As shown, the data processing method provided by at least one embodiment of the present disclosure includes at least steps S10-S60.

[0058] For example, in step S10 , a first original tensor is received.

[0059] The first original tensor is a 5-dimensional tensor, and the 5 dimensions include batch dimension, depth dimension, height dimension, width dimension, and channel number dimension.

[0060] The first original tensor can be obtained from other computing units, streaming processor clusters, etc., or the first original tensor can also be obtained from memory, and the present disclosure does not impose specific limitations on this.

[0061] The shape and size of the first original tensor can be expressed as b1×b2×b3×b4×b5, where b1, b2, b3, b4, and b5 respectively indicate the sizes of the original tensor in the five dimensions and are all positive integers. The present disclosure does not limit the specific shape and size of the first original tensor.

[0062] In step S20, dimension conversion is performed on the first original tensor to obtain a second original tensor.

[0063] The dimensionality conversion process includes performing dimensionality conversion on p dimensions among the five dimensions, where the p dimensions include at least two dimensions among the other four dimensions except the channel number dimension, and p is a positive integer greater than or equal to 2 and less than or equal to 4.

[0064] For example, step S20 may include: performing a transposition operation on the first original tensor in p dimensions to obtain a second original tensor.

[0065] For example, in one embodiment, p = 2, and performing a transpose operation on the first original tensor in two dimensions includes swapping two dimensions in the tensor. For example, the shape of the first original tensor is expressed as b1×b2×b3×b4×b5. By swapping the second dimension (for b2) and the third dimension (corresponding to b3), the shape of the resulting second original tensor is expressed as b1×b3×b2×b4×b5, which has the same meaning as a conventional transpose. Here, the swapping includes swapping the specific content in the dimension. In other words, in this dimensional conversion process, only the positions of the selected p dimensions are swapped, and the other dimensions remain in place. This dimensional conversion is very commonly used in fields such as deep learning and scientific computing, for example, to adjust the convolution input format or rearrange tensors.

[0066] Figure 4 A schematic structural diagram of a second original tensor provided for at least one embodiment of the present disclosure.

[0067] Figure 4 In the example, each solid cube represents an element, and the tensor composed of multiple solid cubes is the second original tensor. The second original tensor has been dimensionally transformed as described above.

[0068] Figure 4 What is shown is a batch and a tensor at a certain depth in a batch, but the second original tensor may have multiple batches, and each batch may have multiple tensors in the depth dimension, with the same structure. Figure 4 The same as shown, no further description is given here.

[0069] Assumptions Figure 4 The element pointed by the arrow is the origin of the preset coordinate system where the second original tensor is located. C, W, and H represent the three coordinate axes respectively. Row is the coordinate value in the H-dimensional direction, col is the coordinate value in the W-dimensional direction, and c is the coordinate value in the C-dimensional direction. The coordinates of the element pointed by the arrow in the preset coordinate system are: c=0, row=0, and col=0. The channel, width, and height values ​​in the coordinates of other elements increase in the direction of the arrow. Of course, the second original tensor can also include D-dimensional and N-dimensional coordinates, which are no longer shown here.

[0070] It should be noted that, in the present disclosure, the coordinate systems used by the second original tensor, the third original tensor described later, the tensor to be processed, etc. are the same and are all preset coordinate systems.

[0071] For example, a preset coordinate system is determined by taking an element in the third original tensor as the origin of the coordinate system. For example, refer to the following Figures 5A-5B In the embodiments, the upper left corner vertex in the third original tensor is used as the origin of the preset coordinate system. Of course, the present disclosure is not limited to this.

[0072] For example, the starting coordinates of the tensor to be processed in the preset coordinate system determined by the third original tensor are as follows: Figure 5A-5B In the embodiment of the present invention, the coordinates of the upper left corner vertex of the tensor to be processed are as follows: The coordinate value of the starting coordinate of the tensor to be processed is the minimum coordinate value among the coordinate values ​​of all elements in the tensor to be processed.

[0073] For example, the shape size of the second original tensor is represented by tensor_c, tensor_w, tensor_h, tensor_d, and tensor_n, where tensor_n is the N dimension, tensor_d is the D dimension, tensor_h is the H dimension, tensor_w is the W dimension, and tensor_c is the C dimension. Figure 4 In the example shown, tensor_n=1, tensor_d=1, tensor_h=4, tensor_w=8, and tensor_c=8.

[0074] In step S30 , at least one dimension of the second original tensor is size-expanded, and a third original tensor obtained by the size expansion is stored in a memory.

[0075] In the memory, the starting tensor elements of each dimension in the third original tensor are located at the starting positions of different storage units, wherein the starting tensor elements are tensor elements corresponding to the starting coordinates of each dimension in the preset coordinate system. The memory performs read and write operations at the granularity of the storage unit, and multiple tensor elements are stored in the storage unit.

[0076] The starting coordinate corresponds to the tensor element with the minimum coordinate value in that dimension.

[0077] As mentioned previously, memory reads and writes are performed at the granularity of storage cells. All data in each storage cell corresponds to the same memory address. The capacity of a storage cell can be set as needed, for example, to 512 bytes. When reading or writing, the storage cell is treated as a whole, regardless of whether all the data in the storage cell is required.

[0078] For example, the starting tensor elements of each dimension in the third original tensor are located at the starting position of different storage units, thereby achieving address alignment of each dimension, so that the data stored in each dimension is an integer multiple of the capacity of the storage unit, and avoiding as much as possible the situation where each requested data loaded after the request is split also needs to read the storage unit containing data that does not need to be loaded. For example, in the aforementioned example, that is, the tensor has a W dimension size of 7 and a C dimension size of 32, and the data format of the tensor elements is BF16. Assuming that after the W dimension size is expanded so that the starting tensor elements of each row are located at the starting position of a storage unit, then for a request to load data with an H dimension coordinate of h+1 and a W dimension coordinate of 0 and 1, when loading the data, only one storage unit needs to be loaded, without having to load the storage unit containing the data with an H dimension coordinate of h as described above, which greatly reduces invalid data loading, improves bandwidth utilization, effectively utilizes data bandwidth, and improves loading efficiency.

[0079] For example, when the size of at least one dimension is expanded, the expanded size is determined based on the storage capacity of the storage unit.

[0080] For example, in some embodiments, size expansion of at least one dimension of the second original tensor may include: for any dimension in the second original tensor, in response to the total storage capacity of all tensor elements of the second original tensor with the same coordinates in the target dimension and above the target dimension being not an integer multiple of the storage capacity of the storage unit, determining to size expand the any dimension, and determining the expanded size of any dimension based on the total storage capacity; in response to the total storage capacity being an integer multiple of the storage capacity of the storage unit, determining not to size expand any dimension; wherein, in response to any dimension being a batch dimension, the target dimension is any dimension, and in response to any dimension not being a batch dimension, the target dimension is a dimension adjacent to and higher than any dimension as indicated by the data storage format of the second original tensor.

[0081] For example, the expanded size of any dimension is such that the total storage capacity of all tensor elements of the third original tensor having the same coordinate in the target dimension is an integer multiple of the storage capacity of the storage unit.

[0082] It should be noted that in the present disclosure, the data storage format of the second original tensor, the first original tensor, the third original tensor, and the tensor to be processed is the same, for example, they are all NDHWC, or they are all N(C / x)DHW(xC), where x is a positive integer greater than 1. Taking NDHWC as an example, the C dimension is the lowest dimension during storage, followed by the H dimension, then the W dimension, then the D dimension, and finally the N dimension. Taking N(C / x)DHW(xC) as an example, the W dimension is the lowest dimension during storage, followed by the H dimension, then the D dimension, then the C dimension, and finally the N dimension, and, for the W dimension, continuous xC is actually also considered.

[0083] For example, taking the data storage format of the second original tensor as NDHWC, when any dimension is C, the target dimension is W. If the total storage capacity of all tensor elements with the same coordinate in the W dimension of the second original tensor is not an integer multiple of the storage capacity of the storage unit, the C dimension is expanded. Taking the coordinate w in the W dimension as an example, all tensor elements with coordinate w include tensor_c tensor elements with different coordinates in the C dimension but the same coordinates in other dimensions, where tensor_c is the size of the second original tensor in the C dimension.

[0084] If the total memory capacity of tensor_c tensor elements is not an integer multiple of the memory capacity of the storage unit, then the C dimension is expanded. Specifically, if the expanded size of the C dimension is stride_c, then stride_c × bit_width is an integer multiple of the memory capacity of the storage unit, where bit_width is the bit width of the tensor element, which determines stride_c.

[0085] by Figure 4 For example, the expanded size makes the tensor element with c=0, row=0, col=0 at the starting position of one storage unit, and the tensor element with c=0, row=0, col=1 at the starting position of another storage unit.

[0086] When any dimension is W, the target dimension is H. If the total storage capacity of all tensor elements with the same coordinates in the H dimension of the second original tensor is not an integer multiple of the storage capacity of the storage unit, the W dimension is expanded. Taking the H dimension coordinate as an example, all tensor elements with the same H coordinate and the same coordinates in the D and N dimensions above the H dimension include, for example, a row of tensor elements and all elements of a row of tensor elements in the C dimension direction, that is, stride_c×tensor_w tensor elements, where tensor_w is the size of the second original tensor in the W dimension.

[0087] If the total memory capacity of stride_c × tensor_w tensor elements is not an integer multiple of the memory capacity of the storage unit, then the W dimension is expanded. Specifically, if the expanded size of the W dimension is stride_w, then stride_c × stride_w × bit_width is an integer multiple of the memory capacity of the storage unit, and stride_w is determined accordingly.

[0088] by Figure 4 For example, the expanded size makes the tensor element with c=0, row=0, col=0 at the starting position of one storage unit, and the tensor element with c=0, row=1, col=0 at the starting position of another storage unit.

[0089] The same applies to H dimension, D dimension and N dimension, which will not be repeated here.

[0090] For example, the third original tensor is obtained, and its shape and size can be expressed as stride_c, stride_w, stride_h, stride_d, and stride_n, where stride_n is the N dimension, stride_d is the D dimension, stride_h is the H dimension, stride_w is the W dimension, and stride_c is the C dimension. The starting tensor element of each dimension of the third original tensor stored in memory is located at the starting position of the corresponding storage unit, thereby achieving storage address alignment.

[0091] In a specific example, for example, the storage capacity of a single storage unit is 512B, the size of the second original tensor in the W dimension is tensor_w=7, the size of the second original tensor in the C dimension is tensor_c=32, the bit width of the tensor element is bit_width=16bit, and for the W dimension, the storage capacity occupied by a row of pixels is 7×32×2B<512B, so the W dimension can be expanded, for example, expanded to 8, so that 8×32×2B=512B, that is, the size of the third original tensor in the W dimension is stride_w=8.

[0092] For example, when storing the third original tensor into the memory, the portion that does not belong to the original content of the second original tensor due to the size expansion is stored in the memory in a predefined format, wherein the predefined format is used to indicate that the corresponding stored content is treated as data not located in the memory for the requested partitioning. For example, the predefined format is used to indicate that the corresponding stored content is treated as data not located in the memory for the subsequent requested partitioning, for example, it can be treated as Figure 6For example, the predefined format can be specified as needed, such as a special agreed data format, or can be directly empty. This disclosure does not impose any specific restrictions on this.

[0093] In step S40 , shape information of a tensor to be processed and its starting coordinates in a preset coordinate system are received.

[0094] For example, the shape information of the tensor to be processed is represented by a1, a2, a3, a4, and a5, where a1, a2, a3, a4, and a5 respectively indicate the sizes of the tensor to be processed in five dimensions and are all positive integers. The five dimensions include batch dimension, depth dimension, height dimension, width dimension, and channel number dimension. In the subsequent embodiments of this disclosure, a1 is the N dimension, a2 is the D dimension, a3 is the H dimension, a4 is the W dimension, and a5 is the C dimension.

[0095] For example, the third original tensor is stored in the memory, the tensor to be processed may be a partial tensor in the third original tensor, or the tensor to be processed may be the third original tensor itself.

[0096] Figure 5A A schematic diagram of a tensor to be processed provided in one embodiment of the present disclosure.

[0097] Figure 5A In the figure, each solid cube with a black frame on a white background represents an element stored in memory and belongs to the second original tensor. The tensor composed of multiple solid cubes is the second original tensor. The gray cubes with black frames represent the portion formed by the aforementioned size expansion. They constitute part of the third original tensor but are not part of the second original tensor and are stored and represented in memory using a predefined storage format.

[0098] Figure 5A What is shown is a batch and a tensor at a certain depth in a batch, but the original tensor may have multiple batches, and each batch may have multiple tensors in the depth dimension, with the same structure. Figure 5A The same as shown, no further description is given here.

[0099] For example, in Figure 5A In the example, the dotted box is the tensor to be processed, which includes some elements in the third original tensor. The dimensions of the tensor to be processed in each dimension are represented by a1, a2, a3, a4, and a5. For example, assuming that the N-dimensional dimension a5 and the D-dimensional dimension a4 are both equal to 1, Figure 5A In the example, the coordinates of the upper left corner element of the tensor to be processed are c=0, row=0, col=3, and a1=5, a2=3, and a3=3.

[0100] For example, in other embodiments, the tensor to be processed includes at least some elements of the third original tensor and tensor elements not stored in memory. For example, in padding mode, in addition to including at least some of the contents of the third original tensor, the tensor to be processed also includes multiple elements with predetermined values ​​(e.g., 0) added to the edge of the third original tensor. Of course, in other embodiments, the tensor to be processed may not even include any elements in the third original tensor at all, and a preset coordinate system may be used for data loading operations. This will not be elaborated here.

[0101] Figure 5A The tensor to be processed shown can be used to perform matrix calculation operations, for example. In some scenarios, the data processing method provided by at least one embodiment of the present disclosure can also be used for convolution calculation operations.

[0102] As an example, Figure 5B FIG. 1 shows a schematic diagram of obtaining tensors to be processed in a continuous manner according to an embodiment of the present disclosure. Figure 5B In , each tensor element is shown as a square, the third original tensor covers all solid squares (ie, both white squares and gray-shaded squares), and the third original tensor also covers dashed squares, which are obtained by size expansion in step S30.

[0103] The starting coordinates of the third original tensor are represented as (0, 0, 0), which means c=0, row=0, col=0. Figure 5B In the figure, the gray-shaded square represents the tensor to be processed, and its starting coordinate is represented as (c=0, row=0, col=2), that is, data is obtained from the second pixel in the first row of the third original tensor, and then, Figure 5B The figure schematically illustrates the process of acquiring the tensor to be processed in a sequential (or continuous) manner according to an embodiment of the present disclosure. Specifically, as indicated by the dashed arrow, starting from the starting point (c=0, row=0, col=2), data is acquired from the third original tensor, for example, pixel by pixel, until the number of acquired pixels reaches copy_pixel_num. It should be noted that, in the present invention, pixels are a tensor description method that does not consider the C dimension, meaning that the C dimension does not affect the number of pixels.

[0104] exist Figure 5B In the example of , the shape information of the tensor to be processed includes the number of pixels included in the tensor to be processed (for example, expressed as copy_pixel_num). Figure 5B Only the three dimensions H, W, and C are shown in FIG. It is understandable that if the data of these three dimensions still does not reach the number of pixels included in the tensor to be processed, higher-dimensional data can be further obtained, which is not limited here. In addition, if Figure 5B As shown in , the C dimension itself does not affect the number of pixels. Assume that Figure 5B The data storage format of the third original tensor in memory is NDHWC format. Figure 5B In the example shown, the total number of pixels in the tensor to be processed is copy_pixel_num=24.

[0105] In step S50 , a plurality of requests for loading the tensor to be processed into the cache are determined by combining the third original tensor, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed.

[0106] As mentioned above, for Figure 5A In the scenario shown for matrix calculation, the shape information of the tensor to be processed can be, for example, the shape size of the tensor to be processed in 5 dimensions; for example, Figure 5B In the scenario shown for convolution calculation, the shape information of the tensor to be processed can be, for example, the number of pixels to be loaded.

[0107] In step S60 , multiple requests are sent in sequence, and the sub-data returned by each request are sequentially written into the cache area to load the tensor to be processed into the cache area.

[0108] For example, the cache area is a buffer in a stream processor cluster.

[0109] The data processing method provided by at least one embodiment of the present disclosure also divides requests according to whether the tensor can be loaded continuously in the first dimension. The sub-data for loading in each request is either all located in the memory or not located in the memory. The division of data requests is more reasonable and more suitable for the loading and storage of tensor data, which greatly improves the bandwidth and efficiency of data memory access, thereby improving the hardware utilization of the computing unit and improving hardware performance.

[0110] For example, in response to determining that when loading the tensor to be processed, the first dimension cannot be loaded continuously but the dimensions lower than the first dimension can be loaded continuously, the requested partitioning is performed on the second dimension, each sub-data requested to be loaded is located in the memory or none of it is located in the memory, and in response to the sub-data requested to be loaded are all located in the memory, the sub-data requested to be loaded are from the second original tensor and are stored continuously in the memory, wherein the storage format of the tensor to be processed indicates that the first dimension takes precedence over the second dimension when storing or loading, and the first dimension and the second dimension are adjacent.

[0111] The object to be loaded, i.e. the tensor to be processed, is split into multiple different requests according to a certain rule and sent in sequence, and the returned data is written into the cache in sequence, thereby efficiently loading the tensor to be processed into the memory. One principle when splitting is that the sub-data loaded by each request are either all in the memory (the sub-data requested to be loaded all belong to the data range of the second original tensor) or not in the memory (the sub-data requested to be loaded do not belong to the data range of the second original tensor). This is because when the requested sub-data is in the memory, a request is sent to the memory. When the requested sub-data is not in the memory, the request can be converted into an operation such as writing a predetermined value to the cache by the hardware. This splitting method can more reasonably send corresponding loading requests to different hardware.

[0112] In addition, during splitting, if it is determined that when loading the tensor to be processed, the first dimension cannot be loaded continuously but the dimensions lower than the first dimension can be loaded continuously, then further division of the request is performed on the second dimension, so that the request can be split more reasonably, and the data stored continuously in the memory can be retained as much as possible, the number of requests can be reduced, the tensor to be processed can be loaded efficiently, and the bandwidth when loading or storing data from the memory can be greatly improved, thereby improving performance and improving the efficiency of the hardware computing unit.

[0113] For example, taking NDHWC as an example, when the first dimension is C dimension, the second dimension is W dimension, or when the first dimension is W dimension, the second dimension is H dimension, or when the first dimension is H dimension, the second dimension is D dimension, and so on.

[0114] In response to satisfying any one of the following conditions, it is determined that the tensor to be processed cannot be loaded continuously in the first dimension: (1) The size of the first dimension of the processed tensor copy_t is not equal to the size of the first dimension of the second original tensor tensor_t (2) The first coordinate value t_coord_b of the starting coordinate of the tensor to be processed in the first dimension is not equal to the second coordinate value of the starting coordinate of the third original tensor in the first dimension (3) The distance interval corresponding to the second dimension is not equal to the predetermined value corresponding to the first dimension.

[0115] Here, the distance interval corresponding to the second dimension is the distance interval between two target tensor elements stored in the memory, the two target tensor elements are two adjacent tensor elements in the third original tensor in the second dimension, and the predetermined value corresponding to the first dimension is determined based at least on the shape size of the second original tensor.

[0116] For example, the distance interval corresponding to the second dimension is determined at least based on the shape size of the third original tensor. In response to the storage format of the third original tensor being NDHWC, the product of the shape sizes of each dimension lower than the second dimension in the third original tensor is determined as the distance interval corresponding to the second dimension, where N represents the batch dimension, D represents the depth dimension, W represents the width dimension, H represents the height dimension, and C represents the channel number dimension.

[0117] Specifically, for NDHWC, the distance interval corresponding to the W dimension is global_stride0=stride_c, which is the size of the third original tensor in the C dimension. Figure 5A or Figure 5B , the coordinates of the two target tensor elements can be, for example, (c=0,row=1,col=3) and (c=0,row=1,col=4).

[0118] The distance interval corresponding to the H dimension is global_stride1=stride_c×stride_w, which is the product of the dimensions of the third original tensor in the C dimension and the W dimension. Figure 5A or Figure 5B , the coordinates of the two target tensor elements can be, for example, (c=0,row=1,col=3) and (c=0,row=2,col=3).

[0119] The distance interval corresponding to the D dimension is global_stride2=stride_c×stride_w×stride_h, which is the product of the dimensions of the third original tensor in the C dimension, W dimension, and H dimension.

[0120] The distance interval corresponding to the N dimension is global_stride2=stride_c×stride_w×stride_h×stride_d, which is the product of the dimensions of the third original tensor in the C dimension, W dimension, H dimension, and D dimension.

[0121] For example, in response to the data storage format of the second original tensor being NDHWC: when the first dimension is the channel number dimension, the shape size of the second original tensor in the first dimension is determined as the predetermined value corresponding to the first dimension; when the first dimension is not the channel number dimension, the product of the shape size of the second original tensor in the first dimension and the distance interval corresponding to the first dimension is determined as the predetermined value corresponding to the first dimension.

[0122] For example, the predetermined value corresponding to the C dimension is tensor_c, that is, the shape size of the second original tensor in the C dimension.

[0123] For example, the predetermined value corresponding to the W dimension is tensor_w×global_stride0, and the predetermined value corresponding to the H dimension is tensor_h×global_stride1, that is, the product of the shape size of the second original tensor in the H dimension and the distance interval corresponding to the H dimension.

[0124] For example, the predetermined value corresponding to the D dimension is tensor_d×global_stride2, that is, the product of the shape size of the second original tensor in the D dimension and the distance interval corresponding to the D dimension.

[0125] Therefore, taking the data storage format as NDHWC and the second coordinate value as 0 as an example, assuming that the first dimension is the C dimension and the second dimension is the W dimension, discontinuity in the C dimension is determined when any of the following conditions is met: (1) The starting coordinate c_coord_b of the tensor to be processed in the C dimension is not equal to 0 (2) The size of the processed tensor in the C dimension, copy_c, is not equal to the size of the second original tensor in the C dimension, tensor_c (3) The distance interval global_stride0 corresponding to the W dimension is not equal to the predetermined value tensor_c corresponding to the C dimension For example, the sub-data loaded by each request has different coordinates in the C dimension, but the coordinates in the W dimension, H dimension, D dimension, and N dimension are the same. At this time, it can be understood that the third original tensor is expanded into a one-dimensional vector composed of multiple pixels in the order of WHDN without considering the depth direction. The sub-data loaded by each request is the data within a pixel, that is, the request is divided in the W dimension, and the next request can be to load the data in the next adjacent pixel, that is, the request is divided in a pixel-hopping manner (abbreviated as perw).

[0126] Of course, it is understandable that continuity in the C dimension is determined when the following three conditions are met at the same time: (1) The starting coordinate c_coord_b of the tensor to be processed in dimension C is equal to 0 (2) The size of the processed tensor in the C dimension copy_c is equal to the size of the second original tensor in the C dimension tensor_c (3) The distance interval global_stride0 corresponding to the W dimension is equal to the predetermined value tensor_c corresponding to the C dimension For example, when the C dimension is continuous, the request can be split in a perh manner.

[0127] Still taking the data storage format as NDHWC and the second coordinate value as 0 as an example, assuming that the first dimension is the W dimension and the second dimension is the H dimension, discontinuity in the W dimension is determined when any of the following conditions is met: (1) The starting coordinate of the tensor to be processed in the W dimension is not equal to 0 (2) The size of the processed tensor in the W dimension copy_w is not equal to the size of the second original tensor in the W dimension tensor_w (3) The distance interval global_stride1 corresponding to the H dimension is not equal to the predetermined value tensor_w×global_stride0 corresponding to the W dimension In this case, when the C dimension is continuous, the sub-data to be loaded in each request has different coordinates in the C and W dimensions, but the coordinates in the H, D, and N dimensions are the same. This means that the sub-data loaded in each request belongs to the same row. That is, the requests are divided along the H dimension, and the sub-data loaded by different requests are located in different rows (abbreviated as perh).

[0128] Of course, it is understandable that continuity in the W dimension is determined when the following three conditions are met at the same time: (1) The starting coordinate of the tensor to be processed in the W dimension is equal to 0 (2) The size of the processed tensor in the W dimension copy_w is equal to the size of the second original tensor in the W dimension tensor_w (3) The distance interval global_stride1 corresponding to the H dimension is equal to the predetermined value tensor_w×global_stride0 corresponding to the W dimension For example, when the W dimension is continuous, the request can be split in a perd manner.

[0129] Assume that the first dimension is H dimension and the second dimension is D dimension. When any of the following conditions is met, it is determined that there is discontinuity in H dimension: (1) The starting coordinate of the tensor to be processed in the H dimension is not equal to 0 (2) The size of the processed tensor in the H dimension, copy_h, is not equal to the size of the second original tensor in the H dimension, tensor_h (3) The distance interval global_stride2 corresponding to the tensor to be processed in the D dimension is not equal to the predetermined value tensor_h×global_stride1 corresponding to the H dimension At this time, when both the C dimension and the W dimension are continuous, the coordinates of the sub-data to be loaded in each request are different in the C dimension, W dimension, and H dimension, but the coordinates in the D dimension and N dimension are the same (abbreviated as perd).

[0130] Of course, it is understandable that continuity in the H dimension is determined when the following three conditions are met at the same time: (1) The starting coordinate of the tensor to be processed in the H dimension is equal to 0 (2) The size of the processed tensor in the H dimension copy_h is equal to the size of the second original tensor in the H dimension tensor_h (3) The distance interval global_stride2 corresponding to the tensor to be processed in the D dimension is equal to the predetermined value tensor_h×global_stride1 corresponding to the H dimension For example, when the H dimension is continuous, the request can be split in a pern manner.

[0131] Assume that the first dimension is D and the second dimension is N. Discontinuity in dimension D is determined when any of the following conditions is met: (1) The starting coordinate of the tensor to be processed in dimension D is not equal to 0 (2) The size of the processed tensor in dimension D, copy_d, is not equal to the size of the second original tensor in dimension D, tensor_d (3) The distance interval global_stride3 corresponding to the tensor to be processed in the N dimension is not equal to the predetermined value tensor_d×global_stride2 corresponding to the D dimension At this time, when the C dimension, W dimension, and H dimension are all continuous, the sub-data to be loaded in each request has different coordinates in the C dimension, W dimension, H dimension, and D dimension, but the coordinates in the N dimension are the same, that is, the request is divided in the D dimension (abbreviated as pern).

[0132] Of course, it is understandable that continuity in N dimensions is determined when the following three conditions are met at the same time: (1) The starting coordinate of the tensor to be processed in dimension D is equal to 0 (2) The size of the processed tensor in dimension D, copy_d, is equal to the size of the second original tensor in dimension D, tensor_d (3) The distance interval global_stride3 corresponding to the tensor to be processed in the N dimension is equal to the predetermined value tensor_d×global_stride2 corresponding to the D dimension At this time, when the C dimension, W dimension, H dimension and D dimension are all continuous, the sub-data to be loaded in each request has the same coordinates in the CWHDN dimension. For the tensor data located in the memory, in fact, one request can be sent to the memory to load the data in the memory, and other requests can be used to load data that is not located in the memory (abbreviated as per1).

[0133] If the lower dimensions, namely C dimension, W dimension, H dimension and D dimension are discontinuous, the N dimension must also be discontinuous.

[0134] The request partitioning logic for pending tensors with data storage format N(C / x)DHW(xC) is the same as that of NDHWC. The difference is that the dimension arrangement of N(C / x)DHW(xC) is different from that of NDHWC. For N(C / x)DHW(xC), due to its special interleaved structure, (xC) is necessarily continuous by default. Therefore, the lowest dimension is considered to be W dimension, followed by H dimension, then D dimension, then C dimension, and the highest is N dimension.

[0135] Due to different storage formats, the methods for determining the distance interval corresponding to each dimension and the predetermined value corresponding to each dimension are also different.

[0136] In response to the storage format of the third original tensor being N(C / x)DHW(xC): when the second dimension is not a batch dimension, determine the product between the shape size of each dimension lower than the second dimension in the third original tensor and the storage capacity of x tensor elements as the distance interval corresponding to the second dimension; when the second dimension is a batch dimension, determine the product between the shape sizes of each dimension lower than the second dimension in the third original tensor as the distance interval corresponding to the second dimension.

[0137] Specifically, for N(C / x)DHW(xC), the distance interval corresponding to the H dimension is global_stride0=stride_w×x×bit_width, where x×bit_width represents the storage capacity of x tensor elements, and stride_w is the shape size of the third original tensor in the W dimension.

[0138] The distance interval corresponding to the D dimension is global_stride1=stride_h×stride_w×x×bit_width, where stride_h represents the shape size of the third original tensor in the H dimension.

[0139] The distance interval corresponding to the C dimension is global_stride2=stride_d×stride_h×stride_w×x×bit_width, where stride_d represents the shape size of the third original tensor in the D dimension.

[0140] The distance interval corresponding to the N dimension is global_stride3=stride_c×stride_d×stride_h×stride_w, where stride_c represents the shape size of the third original tensor in the C dimension.

[0141] In response to the storage format of the third original tensor being N(C / x)DHW(xC): when the second dimension is a height dimension, determine the product of the shape size of the second original tensor in the first dimension and the storage capacity of x tensor elements as the predetermined value corresponding to the first dimension; when the second dimension is a depth dimension or a channel number dimension, determine the product of the shape size of the second original tensor in the first dimension and the distance interval corresponding to the first dimension as the predetermined value corresponding to the first dimension; when the second dimension is a batch dimension, determine the quotient between the shape size of the second original tensor in the first dimension and the storage capacity of x tensor elements, and determine the product of the quotient and the distance interval corresponding to the first dimension as the predetermined value corresponding to the first dimension.

[0142] For example, for N(C / x)DHW(xC), the predetermined value corresponding to the W dimension is tensor_w×x×bit_width, that is, the product of the shape size of the second original tensor in the W dimension and the storage capacity of x tensor elements.

[0143] The predetermined value corresponding to the H dimension is tensor_h×global_stride0, that is, the product of the shape size of the second original tensor in the H dimension and the distance interval corresponding to the H dimension.

[0144] The predetermined value corresponding to the D dimension is tensor_d×global_stride1, that is, the product of the shape size of the second original tensor in the D dimension and the distance interval corresponding to the D dimension.

[0145] The predetermined value corresponding to the C dimension is tensor_c / (x×bit_width)×global_stride2, that is, the quotient of the storage capacity of the x tensor elements of the shape size of the second original tensor in the C dimension is determined, and the product of the quotient and the distance interval corresponding to the C dimension is multiplied.

[0146] For example, taking the data storage format as N(C / x)DHW(xC) and the second coordinate value as 0, assuming the first dimension is W and the second dimension is H, discontinuity in the W dimension is determined when any of the following conditions is met: (1) The starting coordinate of the tensor to be processed in the W dimension is not equal to 0 (2) The size of the processed tensor in the W dimension copy_w is not equal to the size of the second original tensor in the W dimension tensor_w (3) The distance interval global_stride0 corresponding to the H dimension is not equal to the predetermined value tensor_w×x×bit_width corresponding to the W dimension At this point, it can be understood that the sub-data loaded by each request belongs to the same row. That is, the requests are divided along the H dimension, and the sub-data loaded by different requests are located in different rows (abbreviated as perh).

[0147] Of course, it is understandable that continuity in the W dimension is determined when the following three conditions are met at the same time: (1) The starting coordinate of the tensor to be processed in the W dimension is equal to 0 (2) The size of the processed tensor in the W dimension copy_w is equal to the size of the second original tensor in the W dimension tensor_w (3) The distance interval global_stride0 corresponding to the H dimension is equal to the predetermined value tensor_w×x×bit_width corresponding to the W dimension For example, when the W dimension is continuous, the request can be split in a perd manner.

[0148] Assume that the first dimension is H dimension and the second dimension is D dimension. When any of the following conditions is met, it is determined that there is discontinuity in H dimension: (1) The starting coordinate of the tensor to be processed in the H dimension is not equal to 0 (2) The size of the processed tensor in the H dimension, copy_h, is not equal to the size of the second original tensor in the H dimension, tensor_h (3) The distance interval global_stride1 corresponding to the tensor to be processed in the D dimension is not equal to the predetermined value tensor_h×global_stride0 corresponding to the H dimension In this case, the special interleaving structure is by default continuous in (xC). In addition, when the W dimension is also continuous, the sub-data to be loaded in each request has different coordinates in the C dimension, W dimension, and H dimension, but the coordinates in the D dimension and N dimension are the same (abbreviated as perd).

[0149] Of course, it is understandable that continuity in the H dimension is determined when the following three conditions are met at the same time: (1) The starting coordinate of the tensor to be processed in the H dimension is equal to 0 (2) The size of the processed tensor in the H dimension copy_h is equal to the size of the second original tensor in the H dimension tensor_h (3) The distance interval global_stride1 corresponding to the tensor to be processed in the D dimension is equal to the predetermined value tensor_h×global_stride0 corresponding to the H dimension In addition, for N(C / x)DHW(xC), due to the special nature of the interleaving pattern (xC), pixels are continuous, so the first dimension from low to high is W, H, D, C, and N. For the conditions that cannot be loaded continuously when the data storage format is N(C / x)DHW(xC) and the division principles of requests, please refer to the relevant content of NDHWC and will not be listed here in detail.

[0150] Of course, for the data storage format N(C / x)DHW(xC), although the C dimension of this storage format must be continuous, the request can also be split using the perw method in the NDHWC mode. For example, the instruction carries corresponding information indicating that each request contains data in one pixel, that is, a pixel-skipping method. The perw method can also be used to divide the request of the N(C / x)DHW(xC) storage format. The specific division method is as described above and will not be repeated here.

[0151] The following describes in detail how multiple requests for loading tensors to be processed are determined.

[0152] For example, in some embodiments, step S50 may include: determining the first request sent among multiple requests and the initial state of the first request entering the state machine based on the starting coordinates of the tensor to be processed and the shape information of the tensor to be processed; based on the initial state, combined with the third original tensor, using the state machine to determine each request among the multiple requests except the first request.

[0153] Each request includes three parameters: a data read address indicating the starting position for reading data from the memory, a data write address indicating the starting position for writing data to the cache, and the length of the loaded data.

[0154] The data read address of each request is obtained by summing the sizes of all previous requests.

[0155] For example, taking the data storage format as NDHWC, the calculation formulas for the data read address Addr_1 and data write address Addr_2 of the nth request are as follows: Addr_1=u_addr_base + n_coord×global_stride3+ d_coord×global_stride2+ (Formula 1) h_coord×global_stride1+ w_coord×global_stride0+ c_coord×bit_width Addr_2=b_addr_base+req_size_1+ req_size_2+…+ req_size_n-1 (Formula 2) Among them, u_addr_base represents the storage address of the element at the starting coordinate position of the third original tensor in memory, n_coord, d_coord, h_coord, w_coord, and c_coord represent the initial coordinates of the request corresponding to the nth request, global_stride3, global_stride2, global_stride1, and global_stride0 represent the distance intervals corresponding to each dimension. The specific definitions are as described above. b_addr_base represents the starting address of the buffer area where the tensor to be processed is written, and req_size_1, req_size_2, ..., req_size_n-1 represent the data lengths loaded by the first n-1 requests sent.

[0156] For example, taking the data storage format of N(C / x)DHW(xC) as an example, the calculation formulas for the data read address Addr_3 and data write address Addr_4 of the nth request are as follows: Addr_3= n_coord×global_stride3+ c_coord / (x×bit_width)×global_stride2+ d_coord×global_stride1+ (Formula 3) h_coord×global_stride0+ w_coord×bit_width Addr_4=b_addr_base+req_size_1+ req_size_2+…+ req_size_n-1 (Formula 4) Where x×bit_width represents the size of the x tensor elements. The other parameter definitions are the same as those in Formula 1 and Formula 2, and the repeated parts are not repeated here.

[0157] For example, in some embodiments, based on the starting coordinates of the tensor to be processed, the shape information of the tensor to be processed, and the data storage format of the tensor to be processed, the first request sent among multiple requests and the initial state of the first request entering the state machine are determined, including: determining the initial state of the first request entering the state machine based on the starting coordinates of the tensor to be processed; using the starting coordinates of the tensor to be processed as the request initial coordinates corresponding to the first request; determining the data reading address of the first request based on the distance intervals corresponding to each dimension in the third original tensor and the request initial coordinates corresponding to the first request; determining the starting address of the tensor to be processed in the cache area as the data writing address of the first request; and determining the data length loaded by the first request based on the starting coordinates of the tensor to be processed and the shape information of the tensor to be processed.

[0158] For the first request sent, the corresponding initial coordinates of the request are the starting coordinates of the tensor to be processed. Therefore, the data read address of the first request can be calculated by referring to the above formula. The starting address of the tensor to be processed written in the buffer area is used as the data write address of the first request.

[0159] The length of the data loaded in the first request is determined based on the starting coordinates of the tensor to be processed and the shape information of the tensor to be processed. For example, if the tensor to be processed includes the shape dimensions of the tensor to be processed in five dimensions, when the sum of the first coordinate value t_coord_b of the starting coordinate of the tensor to be processed in the first dimension and the shape dimension copy_t of the tensor to be processed in the first dimension is less than the second coordinate value (i.e., the coordinate value of the starting coordinate of the second original tensor / third original tensor in the first dimension, for example, 0), the length of the data loaded in the first request is the shape dimension copy_t of the tensor to be processed in the first dimension. For example, if the sum of the first coordinate value t_coord_b of the starting coordinate of the tensor to be processed in the first dimension and the shape dimension copy_t of the tensor to be processed in the first dimension is greater than or equal to the second coordinate value, the length of the data loaded in the first request is the absolute value of the first coordinate value t_coord_b.

[0160] Considering that the state machine has the advantages of clear logical structure, easy maintenance and expansion, and is particularly suitable for processing multi-condition and multi-branch logic scenarios, the present disclosure adopts a state machine to automatically update the initial request coordinates and loaded data length corresponding to each request, avoiding complex conditional nesting, with clear logic, easy maintenance and strong scalability.

[0161] Figure 6 A schematic diagram of a state machine provided in accordance with an embodiment of the present disclosure.

[0162] For example, the state machine includes a first state s0, a second state s1, and a third state s2, and the initial state of the state machine is determined by the starting coordinate coord_b of the tensor to be processed.

[0163] For example, in response to the first coordinate value t_coord_b of the starting coordinate of the tensor to be processed in the first dimension being less than the second coordinate value of the starting coordinate of the second original tensor / third original tensor in the first dimension, the initial state is determined to be the first state; in response to the first coordinate value being greater than or equal to the second coordinate value and less than the third coordinate value, the initial state is determined to be the second state, wherein the difference between the second coordinate value and the third coordinate value is equal to the size of the second original tensor in the first dimension; in response to the first coordinate value being greater than or equal to the third coordinate value, the initial state is determined to be the third state.

[0164] refer to Figure 6 , the tensor_t direction represents the data of the first dimension, and the data whose coordinates of the t dimension are less than the second coordinate value and greater than the third coordinate value are not located in the memory ( Figure 6 The black part in the middle), the data with coordinates in the t dimension between the second and third coordinate values ​​( Figure 6 The white part in the middle is stored in the memory, which is the actual size of the second original tensor data in the first dimension. The black part can be regarded as the content that does not belong to the original memory data due to reasons such as padding.

[0165] In addition, as mentioned above, the part of the third original tensor that does not belong to the original second original tensor due to size expansion is also regarded as a black part that is not located in the memory. If the tensor to be processed contains this part of the data content, refer to the following text as a method for determining a request for data that does not belong to the memory to send a corresponding request.

[0166] A state will only switch to itself and adjacent states. For example Figure 6 In the embodiment, the first state s0 can jump to the first state s0 or the second state s1, the second state s1 can jump to the first state s0, the second state s1 and the third state s2, and the third state s2 can jump to the first state s0, the second state s1 and the third state s2.

[0167] Figure 6 The six cases (① to ⑥) in the figure mark the state jump changes that the tensor to be processed will undergo when its size in the first dimension is different.

[0168] For example, Figure 6 Case ① is the case where the first coordinate value is smaller than the second coordinate value, and the sum of the first coordinate value and the size of the tensor to be processed in the first dimension is smaller than the second coordinate value. The state machine always loops in the first state s0 on the left.

[0169] For example, Figure 6Case ② in the figure is that the first coordinate value is less than the second coordinate value, and the sum of the first coordinate value and the size of the tensor to be processed in the first dimension is greater than or equal to the second coordinate value and less than the third coordinate value. The state machine cyclically jumps between the first state s0 and the second state s1.

[0170] For example, Figure 6 Case ③ is the case where the first coordinate value is greater than or equal to the second coordinate value but less than the third coordinate value, and the sum of the first coordinate value and the size of the tensor to be processed in the first dimension is less than the third coordinate value. The state machine always loops in the second state s1.

[0171] For example, Figure 6 Case ④ is the case where the first coordinate value is less than the second coordinate value, and the sum of the first coordinate value and the size of the tensor to be processed in the first dimension is greater than or equal to the third coordinate value. The state machine cyclically jumps between the first state s0, the second state s1 and the third state s2.

[0172] For example, Figure 6 Case ⑤ is the case where the first coordinate value is greater than or equal to the second coordinate value, and the sum of the first coordinate value and the size of the tensor to be processed in the first dimension is greater than or equal to the third coordinate value. The state machine cyclically jumps between the second state s1 and the third state s2.

[0173] For example, Figure 6 Case ⑥ is the case where the first coordinate value is greater than or equal to the third coordinate value, and the state machine always loops and jumps in the third state s2.

[0174] After determining the initial state, the initial coordinates of the request corresponding to the second request sent are determined based on the initial state, and the state that the second request enters is determined in combination with the trigger condition. Then, based on the state that the second request entered, the initial coordinates of the request corresponding to the third request sent and the length of the data loaded by the second request sent are determined, and the state that the third request enters is determined in combination with the trigger condition, and so on.

[0175] For example, in some embodiments, based on the initial state and in combination with the third original tensor, a state machine is used to determine each request other than the first request in multiple requests, which may include: based on the initial state, using the state machine to determine the request initial coordinates corresponding to each request and the data length loaded by each request; determining the data reading address of each request according to the distance interval corresponding to each dimension in the third original tensor and the request initial coordinates corresponding to each request; and determining the data writing address of each request according to the data length loaded by each request.

[0176] For example, the state machine outputs the request initial coordinates corresponding to the current request and the data length loaded by the current request in each state. In addition, the state machine also prepares the request initial coordinates corresponding to the next request for the next state.

[0177] After determining the request initial coordinates corresponding to the current request, it can be determined whether the sub-data loaded by the current request is located in the memory according to the request initial coordinates.

[0178] For example, in response to the coordinate value of the initial request coordinate corresponding to the current request in the first dimension being greater than or equal to the second coordinate value and less than the third coordinate value, it is determined that all the sub-data to be loaded by the current request are located in the memory. In this case, the data read address and data write address of the current request can be determined with reference to Formulas 1-4 above, and then the current request is sent to the memory to load the corresponding sub-data into the cache area.

[0179] For example, in response to the coordinate value of the initial request coordinate corresponding to the current request in the first dimension being less than the second coordinate value or greater than or equal to the third coordinate value, it is determined that none of the sub-data to be loaded by the current request is located in the memory. In this case, in response to the fact that none of the sub-data to be loaded by the current request is located in the memory, the current request is converted to, for example, writing multiple predetermined values ​​to the cache using hardware, where the number of the multiple predetermined values ​​is determined by the length of the data to be loaded specified by the current request. For example, the predetermined value is 0.

[0180] The following specifically describes the process of using the state machine to determine the request initial coordinates and loaded data length corresponding to each request.

[0181] When the first coordinate value of the tensor to be processed in the first dimension is less than the second coordinate value, the first state s0 is entered.

[0182] For example, when the current request is the first request, the state of the current request is the initial state, the state machine outputs the request initial coordinates and the loaded data length corresponding to the current request, and prepares the request initial coordinates for the next state.

[0183] For example, in response to the current request being in the first state and satisfying a first condition, determining that the next request of the current request enters the first state. For example, the first condition includes that the sum of the size copy_t of the to-be-processed tensor in the first dimension and the first coordinate value is less than the second coordinate value. Taking the second coordinate value as 0, the first condition includes that the size copy_t of the to-be-processed tensor in the first dimension is less than the absolute value of the first coordinate value.

[0184] For example, if the current request is in the first state s0 and the size copy_t of the tensor to be processed in the first dimension is relatively small, for example, the sum of the size of the tensor to be processed in the first dimension and the first coordinate value is smaller than the second coordinate value, the state of the next request is still the first state s0. Figure 6Case ①s0->s0.

[0185] In this case, the coordinate value of the request initial coordinate corresponding to the next request in the first dimension is determined to be the first coordinate value t_coord_b, and the coordinate values ​​of the request initial coordinate corresponding to the next request in other dimensions except the first dimension are updated according to the coordinate values ​​lower than the other dimensions.

[0186] Specifically, for other dimensions, the update methods include incrementing (1 or x), returning to the initial coordinates, and leaving them unchanged. For example, for the second dimension, the coordinates of each request increment for the second dimension, while the third dimension and higher dimensions remain unchanged until the processed tensor reaches the boundary of the second dimension. After that, the coordinates of the second dimension are updated back to the starting coordinates and the coordinates of the third dimension increment. The third dimension is adjacent to and higher than the second dimension, and the coordinates of each dimension increment in this manner. The coordinates of other dimensions below the first dimension remain unchanged.

[0187] For example, in this case, the length of the data currently requested to be loaded, req_size, is the size of the first dimension of the tensor to be processed, copy_t.

[0188] For example, in response to the current request being in the first state and satisfying the second condition, determining that the next request of the current request enters the second state s1. For example, the second condition includes that the size copy_t of the tensor to be processed in the first dimension is greater than or equal to the absolute value of the first coordinate value t_coord_b. Figure 6 In case ② and case ④, s0->s1.

[0189] For example, if the current request is in the first state s0 and the size copy_t of the tensor to be processed in the first dimension is relatively large, for example, the sum of the size of the tensor to be processed in the first dimension and the first coordinate value is greater than or equal to the second coordinate value, the state of the next request jumps to the second state s1. For example, assuming the second coordinate value is 0, the second condition includes that the size copy_t of the tensor to be processed in the first dimension is greater than or equal to the absolute value of the first coordinate value t_coord_b.

[0190] In this case, the coordinate value of the request initial coordinate corresponding to the next request in the first dimension is determined to be the second coordinate value, for example 0, and the coordinate value of the request initial coordinate corresponding to the next request in other dimensions except the first dimension is determined to remain unchanged. Since the first state jumps to the second state, the data of the first dimension has not been loaded yet, so the coordinate value of the request initial coordinate corresponding to the next request in other dimensions except the first dimension is determined to remain unchanged.

[0191] For example, in this case, the length of the data currently requested to be loaded, req_size, is determined based on the first coordinate value and the second coordinate value, for example, is the difference between the first coordinate value and the second coordinate value.

[0192] For example, in response to the current request being in the second state s1 and satisfying the third condition, it is determined that the next request of the current request enters the first state s0. The third condition includes that the first coordinate value t_coord_b is less than the second coordinate value and the sum of the first coordinate value t_coord_b and the size copy_t of the tensor to be processed in the first dimension is less than the third coordinate value (for example, the second coordinate value is 0, and the third coordinate value is the size tensor_t of the second original tensor in the first dimension), for example Figure 6 In case ②, s1->s0. The sum of the first coordinate value t_coord_b and the size copy_t of the first dimension of the tensor to be processed is less than the third coordinate value, so the third state s2 will not be entered.

[0193] In this case, the coordinate value of the initial coordinate of the request corresponding to the next request in the first dimension is determined to be the first coordinate value t_coord_b, and the coordinate value of the initial coordinate of the request corresponding to the next request in other dimensions except the first dimension is determined to be updated according to the coordinate value lower than the other dimensions. Specifically, for other dimensions, the update methods include self-increment (1 or x), return to the initial coordinate, and unchanged. For example, for the second dimension, the coordinate of the second dimension of each request is self-incremented, and the third dimension and higher dimensions remain unchanged until the boundary of the tensor to be processed is on the second dimension. After that, the coordinate of the second dimension is updated back to the starting coordinate and the coordinate of the third dimension is self-incremented. The third dimension is adjacent to the second dimension and higher than the second dimension, and each dimension increases accordingly. The coordinates of other dimensions lower than the first dimension remain unchanged.

[0194] For example, in this case, the data length req_size currently requested to be loaded is determined based on the first coordinate value and the size copy_t of the tensor to be processed in the first dimension. For example, if the second coordinate value is 0, the data length req_size currently requested to be loaded is the sum of the first coordinate value and the size copy_t of the tensor to be processed in the first dimension.

[0195] For example, the state of the current request is the second state s1 and the fourth condition is met, and the next request is determined to enter the second state s1. The fourth condition includes that the first coordinate value is greater than or equal to the second coordinate value and the sum of the first coordinate value and the size of the tensor to be processed in the first dimension is less than the third coordinate value, and the difference between the second coordinate value and the third coordinate value is equal to the size of the second original tensor in the first dimension. For example, if the current request is in the second state, it means that the requested sub-data are all in the memory, that is, they are all valid data. When the size copy_t of the tensor to be processed in the first dimension is relatively small, for example, the sum of the size of the tensor to be processed in the first dimension and the first coordinate value is less than the third coordinate value, the state of the next request is still the second state s1 and will not jump to the third state s2. For example, Figure 6 Situation ③.

[0196] In this case, the coordinate value of the initial coordinate of the request corresponding to the next request in the first dimension is determined to be the first coordinate value t_coord_b, and the coordinate value of the initial coordinate of the request corresponding to the next request in other dimensions except the first dimension is determined to be updated according to the coordinate value lower than the other dimensions. Specifically, for other dimensions, the update methods include self-increment (1 or x), return to the initial coordinate, and unchanged. For example, for the second dimension, the coordinate of the second dimension of each request is self-incremented, and the third dimension and higher dimensions remain unchanged until the boundary of the tensor to be processed is on the second dimension. After that, the coordinate of the second dimension is updated back to the starting coordinate and the coordinate of the third dimension is self-incremented. The third dimension is adjacent to the second dimension and higher than the second dimension, and each dimension increases accordingly. The coordinates of other dimensions lower than the first dimension remain unchanged.

[0197] For example, in this case, the length of the data currently requested to be loaded, req_size, is the size of the first dimension of the tensor to be processed, copy_t.

[0198] For example, in response to the current request state being the second state s1 and satisfying the fifth condition, it is determined that the next request enters the third state s2. The fifth condition includes that the sum of the size of the to-be-processed tensor in the first dimension and the first coordinate value is greater than or equal to the third coordinate value. At this time, the size copy_t of the to-be-processed tensor in the first dimension is relatively large, and the state of the next request jumps to the third state s2. For example, Figure 6 s1->s2 in case ④ and case ⑤.

[0199] In this case, the coordinate value of the request initial coordinate corresponding to the next request in the first dimension is determined to be the third coordinate value, and the coordinate values ​​of the request initial coordinate corresponding to the next request in other dimensions except the first dimension are determined to remain unchanged.

[0200] For example, in this case, the data length req_size loaded by the current request is determined based on the coordinate value t_coord and the third coordinate value of the request initial coordinate corresponding to the current request in the first dimension. For example, the data length req_size loaded by the current request is the difference between the third coordinate value and the coordinate value t_coord.

[0201] For example, in response to the state of the current request being the third state, the state to be entered by the next request of the current request is determined according to the first coordinate value.

[0202] For example, when the first coordinate value is less than the second coordinate value, it is determined that the next request enters the first state s0, for example Figure 6 In case ④, s2->s0. When the first coordinate value is greater than or equal to the second coordinate value but less than the third coordinate value, it is determined that the next request enters the second state s1, for example Figure 6 In case ⑤, s2->s1. When the first coordinate value is greater than or equal to the third coordinate value, the next request is determined to enter the third state s2, for example Figure 6 In case ⑥, s2->s2.

[0203] In this case, the coordinate value of the initial coordinate of the request corresponding to the next request in the first dimension is determined to be the first coordinate value t_coord_b, and the coordinate values ​​of the initial coordinate of the request corresponding to the next request in other dimensions except the first dimension are updated according to the coordinate values ​​lower than the other dimensions. Specifically, for other dimensions, the update method includes self-increment (1 or x), return to the initial coordinate or remain unchanged. For example, for the second dimension, the coordinate of the second dimension of each request is self-incremented until it reaches the boundary of the second dimension of the tensor to be processed, and then it is updated back to the starting coordinate and the coordinate of the third dimension is self-incremented. The third dimension is adjacent to the second dimension and higher than the second dimension, and each dimension is incremented in this way. The coordinates of other dimensions lower than the first dimension remain unchanged.

[0204] For example, in this case, the data length req_size loaded by the current request is determined based on the coordinate value of the request initial coordinate corresponding to the current request in the first dimension and the coordinate value of the third coordinate. For example, the data length req_size loaded by the current request is the difference between the coordinate value of the request initial coordinate corresponding to the current request in the first dimension and the coordinate value of the third coordinate.

[0205] In other embodiments, the state machine may also update the remaining size (remain_copy_t) in the five dimensions, where the remaining size indicates the amount of data that has not yet been loaded in each dimension. When determining the state transition condition of the state machine, the relationship between the fourth coordinate value of the initial coordinate of the request corresponding to the current request in the first dimension and the remaining size is considered.

[0206] For example, taking the jump from the first state s0 to the first state s0 as an example, the first condition may include that the sum of the fourth coordinate value t_coord of the requested initial coordinate in the first dimension corresponding to the current request and the remaining size remain_copy_t of the tensor to be processed in the first dimension is less than the second coordinate value, and the fourth coordinate value t_coord is less than the second coordinate value. The remaining size remain_copy_t is the amount of remaining tensor data in the first dimension of the tensor to be processed that has not been requested to be loaded.

[0207] For example, taking the jump from the first state s0 to the second state s1 as an example, the second condition may include that the sum of the fourth coordinate value t_coord of the request initial coordinate corresponding to the current request in the first dimension and the remaining size remain_copy_t of the tensor to be processed in the first dimension is greater than or equal to the second coordinate value.

[0208] For example, taking the jump from the second state s1 to the first state s0 as an example, the third condition may include that the sum of the fourth coordinate value t_coord of the request initial coordinate corresponding to the current request in the first dimension and the remaining size remain_copy_t of the tensor to be processed in the first dimension is less than or equal to the third coordinate value, and the first coordinate value is less than the second coordinate value.

[0209] For example, taking the jump from the second state s1 to the second state s2 as an example, the fourth condition includes that the sum of the fourth coordinate value and the remaining size is less than the third coordinate value and the first coordinate value is greater than or equal to the second coordinate value.

[0210] For example, taking the jump from the second state s1 to the third state s2 as an example, the fifth condition may include that the sum of the request initial coordinate corresponding to the current request and the remaining size remain_copy_t of the tensor to be processed in the first dimension is greater than the third coordinate value.

[0211] For example, for the third state s2, the state to be entered by the next request of the current request is determined according to the first coordinate value. For details, please refer to the above content.

[0212] Of course, as the coordinates continue to increase, when the remaining size of each dimension is equal to 1, the state machine can end the coordinate determination of each request, and the state machine can enter the idle state.

[0213] The specific state machine state switching conditions and updated coordinate content can be changed and set according to actual needs. The logic is similar to the state machine logic described above, and examples are not given here one by one.

[0214] It should be noted that the above process of using the state machine to determine the initial coordinates of the request corresponding to the request and the length of the loaded data is basically the same for NDHWC and N(C / x)DHW(xC). The difference is that in response to the data storage format of the tensor to be processed being NDHWC, the coordinate value of the channel number dimension is automatically incremented by 1 when updated, and in response to the data storage format of the tensor to be processed being N(C / x)DHW(xC), the coordinate value of the channel number dimension is automatically incremented by x when updated.

[0215] For example, in the data processing method provided by at least one embodiment of the present disclosure, step S60 may include: for any request, if all sub-data loaded in response to the any request are located in the memory, sending the any request to the memory; if none of the sub-data loaded in response to the any request are located in the memory, converting the any request into writing a plurality of predetermined values ​​to the cache, wherein the number of the plurality of predetermined values ​​is determined by the length of the data loaded specified by the any request. For example, at this time, the current request is converted to, for example, using hardware to write a plurality of predetermined values ​​to the cache, wherein the number of the plurality of predetermined values ​​is determined by the length of the data loaded specified by the current request, for example, the predetermined value is 0.

[0216] For example, the following describes the data loading process when the first dimension is C, using the data storage format as NDHWC and the shape information of the tensor to be processed including the shape and size of the tensor to be processed in five dimensions. In this example, the first coordinate value is the coordinate value c_coord_b of the starting coordinate of the tensor to be processed in the C dimension, the second coordinate value is 0, and the third coordinate value is the size tensor_c of the second original tensor in the C dimension, and the predetermined value is 0.

[0217] For example, if any one of the following three conditions is met, it is determined that the tensor to be processed cannot be loaded continuously in the C dimension: the size copy_c of the tensor to be processed in the C dimension is not equal to the size tensor_c of the third original tensor in the C dimension, the first coordinate value c_coord_b is not equal to 0, and the distance interval global_stride0 corresponding to the W dimension is equal to the predetermined value stride_c corresponding to the C dimension.

[0218] First, in step S10, a first original tensor is received. The shape and size of the first original tensor are expressed as b1×b2×b3×b4×b5, for example. b1, b2, b3, b4, and b5 respectively indicate the sizes of the original tensor in the five dimensions and are all positive integers. Specifically, b1 represents the C dimension, b2 represents the W dimension, b3 represents the H dimension, b4 represents the D dimension, and b5 represents the N dimension.

[0219] Then, in step S20, the first original tensor is dimensionally transformed to obtain a second original tensor. For example, the W dimension and H dimension of the first original tensor are transposed to obtain a second original tensor. For example, the shape of the second original tensor is expressed as b1×b3×b2×b4×b5, where the size of the second original tensor in the C dimension is tensor_c=b1, the size of the second original tensor in the W dimension is tensor_w=b3, the size of the second original tensor in the H dimension is tensor_h=b2, the size of the second original tensor in the D dimension is tensor_d=b4, and the size of the second original tensor in the N dimension is tensor_n=b5.

[0220] Then, in step S30, at least one dimension of the second original tensor is size-expanded, and a third original tensor obtained by the size expansion is stored in the memory.

[0221] The specific process of size expansion and storage can be referred to the above content and will not be repeated here.

[0222] Thereafter, in step S40 , the shape information of the tensor to be processed and the starting coordinates of the tensor to be processed in a preset coordinate system are received.

[0223] For example, in this embodiment, the shape information of the tensor to be processed includes the shape dimensions of the tensor to be processed in five dimensions, namely copy_c / h / w / d / n, etc., and the starting coordinates of the tensor to be processed in the preset coordinate system, namely c / h / w / d / n_coord_b.

[0224] Thereafter, in step S50 , multiple requests for loading the tensors to be processed are determined based on the above information.

[0225] Referring to the process described above, the state machine determines the initial coordinates of each request and the length of the data to be loaded, and accordingly determines the data read address and data write address for each request. For example, in this example, the coordinates of the sub-data loaded by each request are different in the C dimension, but the coordinates in the H, W, D, and N dimensions are the same.

[0226] Table 1 shows the updating process of requesting initial coordinates and requesting loading data length provided by an embodiment of the present disclosure.

[0227] Table 1

[0228] As shown in Table 1, if the first coordinate value c_coord_b is less than 0, it is determined that the first request enters the first state s0. The data reading address and the data writing address of the first request refer to the above description and are not repeated here.

[0229] If the first condition is met, the next request is determined to enter the first state s0. For the first condition, refer to the above description. As shown in Table 1, at this time, the coordinate value c_coord of the initial coordinate of the request corresponding to the next request in the C dimension is the first coordinate value c_coord_b; the coordinate value w_coord of the initial coordinate of the request corresponding to the next request in the W dimension is incremented by 1, and returns to w_coord_b when reaching the boundary of the tensor to be processed in the W dimension; the coordinate value h_coord of the initial coordinate of the request corresponding to the next request in the H dimension is incremented by 1 when w_coord returns to w_coord_b, and remains unchanged if the boundary of the tensor to be processed in the H dimension is not reached. When the boundary of the tensor to be processed in the H dimension is reached, h_coord_b is returned; the coordinate value d_coord of the initial coordinate of the request in the D dimension corresponding to the next request is incremented by 1 when h_coord returns h_coord_b. If the boundary of the tensor to be processed in the D dimension has not been reached, it remains unchanged. When the boundary of the tensor to be processed in the D dimension is reached, d_coord_b is returned; the coordinate value n_coord of the initial coordinate of the request in the N dimension corresponding to the next request is incremented by 1 when d_coord returns d_coord_b, and other situations remain unchanged.

[0230] Moreover, the length of the data loaded by the first request is the size copy_c of the tensor to be processed in the C dimension.

[0231] If the second condition is met, the next request is determined to enter the second state s1. For the second condition, refer to the above description. At this time, the coordinate value c_coord of the request initial coordinate corresponding to the next request in the C dimension is 0; the coordinate value w_coord of the request initial coordinate corresponding to the next request in the W dimension, the coordinate value h_coord of the request initial coordinate corresponding to the next request in the H dimension, the coordinate value d_coord of the request initial coordinate corresponding to the next request in the D dimension, and the coordinate value n_coord of the request initial coordinate corresponding to the next request in the N dimension remain unchanged.

[0232] Furthermore, at this time, the length of the data loaded by the first request is the absolute value of the first coordinate value.

[0233] Since the coordinate value c_coord of the initial coordinate of the request corresponding to the first request in the C dimension is less than 0, the sub-data loaded by the first request are not in the memory. In step S60, the first request is converted into an operation of writing copy_c or |c_coord_b| 0s to the cache area, where |c_coord_b| represents the absolute value of c_coord_b.

[0234] In addition, it should be noted that if it is determined that the coordinate value c_coord of the initial coordinate of the request corresponding to the request in the C dimension is less than 0, there is no need to calculate the data read address and the data write address of the request, thereby reducing the amount of calculation.

[0235] As shown in Table 1, when the first coordinate value c_coord_b is greater than or equal to 0 and less than tensor_c, it is determined that the first request enters the second state s1. The data reading address and data writing address of the first request refer to the above description and are not repeated here.

[0236] If the third condition is met, the next request is determined to enter the first state s0. For the third condition, refer to the above description. As shown in Table 1, at this time, the coordinate value c_coord of the initial coordinate of the request corresponding to the next request in the C dimension is the first coordinate value c_coord_b; the coordinate value w_coord of the initial coordinate of the request corresponding to the next request in the W dimension is incremented by 1, and returns to w_coord_b when reaching the boundary of the tensor to be processed in the W dimension; the coordinate value h_coord of the initial coordinate of the request corresponding to the next request in the H dimension is incremented by 1 when w_coord returns to w_coord_b, and remains unchanged if the boundary of the tensor to be processed in the H dimension is not reached. When the boundary of the tensor to be processed in the H dimension is reached, h_coord_b is returned; the coordinate value d_coord of the initial coordinate of the request in the D dimension corresponding to the next request is incremented by 1 when h_coord returns h_coord_b. If the boundary of the tensor to be processed in the D dimension has not been reached, it remains unchanged. When the boundary of the tensor to be processed in the D dimension is reached, d_coord_b is returned; the coordinate value n_coord of the initial coordinate of the request in the N dimension corresponding to the next request is incremented by 1 when d_coord returns d_coord_b, and other situations remain unchanged.

[0237] Moreover, the length of the data loaded by the first request at this time is the sum of the size copy_c of the tensor to be processed in the C dimension and the first coordinate value c_coord_b.

[0238] If the fourth condition is met, the next request is determined to enter the second state s1. For the fourth condition, refer to the above description. As shown in Table 1, at this time, the coordinate value c_coord of the initial coordinate of the request corresponding to the next request in the C dimension is the first coordinate value c_coord_b; the coordinate value w_coord of the initial coordinate of the request corresponding to the next request in the W dimension is incremented by 1, and returns to w_coord_b when reaching the boundary of the tensor to be processed in the W dimension; the coordinate value h_coord of the initial coordinate of the request corresponding to the next request in the H dimension is incremented by 1 when w_coord returns to w_coord_b, and remains unchanged if the boundary of the tensor to be processed in the H dimension is not reached. When the boundary of the tensor to be processed in the H dimension is reached, h_coord_b is returned; the coordinate value d_coord of the initial coordinate of the request in the D dimension corresponding to the next request is incremented by 1 when h_coord returns h_coord_b. If the boundary of the tensor to be processed in the D dimension has not been reached, it remains unchanged. When the boundary of the tensor to be processed in the D dimension is reached, d_coord_b is returned; the coordinate value n_coord of the initial coordinate of the request in the N dimension corresponding to the next request is incremented by 1 when d_coord returns d_coord_b, and other situations remain unchanged.

[0239] Moreover, the length of the data loaded by the first request is the size copy_c of the tensor to be processed in the C dimension.

[0240] If the fifth condition is met, the next request is determined to enter the third state s2. For details about the fifth condition, refer to the previous description. As shown in Table 1, at this time, the coordinate value c_coord of the request initial coordinate corresponding to the next request in the C dimension is the third coordinate value tensor_c; the coordinate value w_coord of the request initial coordinate corresponding to the next request in the W dimension, the coordinate value h_coord of the request initial coordinate corresponding to the next request in the H dimension, the coordinate value d_coord of the request initial coordinate corresponding to the next request in the D dimension, and the coordinate value of the request initial coordinate corresponding to the next request in the N dimension remain unchanged.

[0241] Moreover, at this time, the length of the data loaded by the first request is the difference between the size tensor_c of the second original tensor in the C dimension and the coordinate value c_coord of the initial coordinate of the request corresponding to the next request in the C dimension.

[0242] Since the state of the first request is the second state s1, the sub-data loaded thereto are all located in the memory. In step S60, the first request is sent to the memory to load the sub-data in the second original tensor stored in the memory into the memory.

[0243] As shown in Table 1, when the first coordinate value c_coord_b is greater than or equal to tensor_c, it is determined that the first request enters the third state s2. The data reading address and data writing address of the first request refer to the above description and are not repeated here.

[0244] The state that the next request enters is determined according to the first coordinate value c_coord_b. For example, when the first coordinate value c_coord_b is less than 0, the next request is determined to enter the first state s0. When the first coordinate value c_coord_b is greater than or equal to 0 but less than tensor_c, the next request is determined to enter the second state s1. When the first coordinate value c_coord_b is greater than or equal to tensor_c, the next request is determined to enter the third state s2.

[0245] As shown in Table 1, no matter which state the next request enters, the update logic of the initial coordinates of the request and the length of the data loaded in the first request are exactly the same.

[0246] For example, taking the next request entering the first state s0 as an example, at this time, the coordinate value c_coord of the initial coordinate of the request corresponding to the next request in the C dimension is the first coordinate value c_coord_b; the coordinate value w_coord of the initial coordinate of the request corresponding to the next request in the W dimension is automatically increased by 1, and returns w_coord_b when reaching the boundary of the tensor to be processed in the W dimension; the coordinate value h_coord of the initial coordinate of the request corresponding to the next request in the H dimension is automatically increased by 1 when w_coord returns w_coord_b, and if it has not reached the boundary of the tensor to be processed in the H dimension The bounds remain unchanged, and h_coord_b is returned when the bounds of the tensor to be processed in the H dimension are reached. The coordinate value d_coord of the requested initial coordinate in the D dimension corresponding to the next request is incremented by 1 when h_coord returns h_coord_b. If the bounds of the tensor to be processed in the D dimension have not been reached, it remains unchanged and d_coord_b is returned when the bounds of the tensor to be processed in the D dimension are reached. The coordinate value n_coord of the requested initial coordinate in the N dimension corresponding to the next request is incremented by 1 when d_coord returns d_coord_b, and remains unchanged otherwise. The length of the data loaded by the first request is the difference between the size of the second original tensor in the C dimension, tensor_c, and the coordinate value c_coord of the requested initial coordinate in the C dimension corresponding to the next request.

[0247] Since the coordinate value c_coord of the initial coordinate of the first request in the C dimension is greater than tensor_c, the sub-data loaded by the first request are not in the memory. In step S60, the first request is converted into an operation of writing c_coord-tensor_c zeros to the cache area.

[0248] Afterwards, the above process is continued to determine the initial request coordinates and loaded data length corresponding to each subsequent request, and step S60 is executed to send multiple requests. The specific process will not be repeated here.

[0249] In the above embodiments, the data processing method provided by at least one embodiment of the present disclosure supports dimension conversion, reduces the complexity of dimension conversion on data loading, and is stored in memory after dimension conversion, after which the data request can be directly split and sent; size extension is set to ensure address alignment so that the starting tensor element of each dimension is located at the starting position of the storage unit, thereby improving bandwidth utilization, effectively utilizing data bandwidth, and improving loading efficiency. In addition, the present disclosure splits the object to be loaded, that is, the tensor to be processed, into multiple different requests according to a certain rule and sends them in sequence, and writes the returned data sequentially into the cache area, thereby efficiently loading the tensor to be processed into the memory, and the division of data requests is more reasonable, more suitable for the loading and storage of tensor data, greatly improving the bandwidth and efficiency of data memory access, thereby improving the hardware utilization of the computing unit and improving hardware performance. One principle for splitting is that the sub-data requested to be loaded in each request are either all in the memory (the sub-data requested to be loaded all belong to the data range of the second original tensor) or not in the memory (the sub-data requested to be loaded do not belong to the data range of the second original tensor). This is because when the requested sub-data is in the memory, a request is sent to the memory. When the requested sub-data is not in the memory, the request can be converted into an operation such as writing a predetermined value to the cache by the hardware. This splitting method can more reasonably send corresponding loading requests to different hardware.

[0250] At least one embodiment of the present disclosure further provides a data processing device. Figure 7 A schematic structural diagram of a data processing device provided for at least one embodiment of the present disclosure.

[0251] like Figure 7 As shown, the data processing device 100 provided by at least one embodiment of the present disclosure includes a first receiving module 101, a dimension conversion module 102, a size expansion module 103, a second receiving module 104, a request splitting module 105, a sending module 106, a memory 107 and a cache area 108.

[0252] The first receiving module 101 is configured to receive a first original tensor. The first original tensor is a 5-dimensional tensor, where the 5 dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a channel number dimension.

[0253] The dimension conversion module 102 is configured to perform dimension conversion processing on the first original tensor to obtain a second original tensor. The dimension conversion processing includes performing dimension conversion on p dimensions of the five dimensions, where the p dimensions include at least two dimensions of the other four dimensions except the channel number dimension, and p is a positive integer greater than or equal to 2 and less than or equal to 4.

[0254] The size expansion module 103 is configured to size expand at least one dimension of the second original tensor and store the third original tensor obtained by the size expansion in the memory 107. In the memory 107, the starting tensor elements of each dimension in the third original tensor are located at the starting position of different storage units, where the starting tensor elements are the tensor elements corresponding to the starting coordinates of each dimension in the preset coordinate system. The memory 107 performs read and write operations at the granularity of the storage unit, and the storage unit stores multiple tensor elements.

[0255] The second receiving module 104 is configured to receive shape information of a tensor to be processed and a starting coordinate in a preset coordinate system.

[0256] The request splitting module 105 is configured to determine a plurality of requests for loading the tensor to be processed into the cache 108 based on the third original tensor, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed.

[0257] The sending module 106 is configured to send multiple requests in sequence, and write the sub-data returned by each request into the buffer area 108 in sequence, so as to load the tensor to be processed into the buffer area 108.

[0258] For example, the first receiving module 101, the dimension conversion module 102, the size expansion module 103, the second receiving module 104, the request splitting module 105, and the sending module 106 include codes and programs stored in the memory. The first receiving module 101, the dimension conversion module 102, the size expansion module 103, the second receiving module 104, the request splitting module 105, and the sending module 106 are implemented as, for example, a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities. The processing unit can be a general-purpose processor, and can also be a single-chip microcomputer, a microprocessor, a digital signal processor, a dedicated image processing chip, or a field programmable logic array, etc. The first receiving module 101, the dimension conversion module 102, the size expansion module 103, the second receiving module 104, the request splitting module 105, and the sending module 106 execute the codes and programs to implement some or all of the functions of the first receiving module 101, the dimension conversion module 102, the size expansion module 103, the second receiving module 104, the request splitting module 105, and the sending module 106 as described above. For example, the first receiving module 101, the dimension conversion module 102, the size expansion module 103, the second receiving module 104, the request splitting module 105, and the sending module 106 may be a circuit board or a combination of multiple circuit boards, configured to implement the functions described above. In an embodiment of the present application, the circuit board or the combination of multiple circuit boards may include: (1) one or more processors; (2) one or more non-transitory memories connected to the processors; and (3) firmware stored in the memories and executable by the processors.

[0259] It should be noted that the first receiving module 101 can be used to implement Figure 3 Step S10 shown; the dimension conversion module 102 can be used to implement Figure 3 Step S20 shown; the size expansion module 103 can be used to implement Figure 3 Step S30 shown; the second receiving module 104 can be used to implement Figure 3 Step S40 shown; request splitting module 105 can be used to implement Figure 3 Step S50 shown; the sending module 106 can be used to implement Figure 3As shown in step S60. Therefore, for the specific description of the functions that the first receiving module 101 can implement, reference can be made to the relevant description of step S10 in the embodiment of the above-mentioned data processing method; for the specific description of the functions that the dimension conversion module 102 can implement, reference can be made to the relevant description of step S20 in the embodiment of the above-mentioned data processing method; for the specific description of the functions that the size expansion module 103 can implement, reference can be made to the relevant description of step S30 in the embodiment of the above-mentioned data processing method; for the specific description of the functions that the second receiving module 104 can implement, reference can be made to the relevant description of step S40 in the embodiment of the above-mentioned data processing method; for the specific description of the functions that the request splitting module 105 can implement, reference can be made to the relevant description of step S50 in the embodiment of the above-mentioned data processing method; for the specific description of the functions that the sending module 106 can implement, reference can be made to the relevant description of step S60 in the embodiment of the above-mentioned data processing method; and the repeated parts will not be repeated here. In addition, the data processing device 100 can achieve technical effects similar to those of the above-mentioned data processing method, which will not be repeated here.

[0260] It should be noted that in at least one embodiment of the present disclosure, the data processing device 100 may include more or fewer circuits or units, and the connection relationship between the various circuits or units is not limited and can be determined according to actual needs. The specific configuration of each circuit or unit is not limited and can be composed of analog devices according to circuit principles, or can be composed of digital chips, or constructed in other applicable ways.

[0261] For example, the data processing device 100 may be implemented in hardware, software, or a combination of hardware and software, and this disclosure does not impose any specific limitations on this.

[0262] In some embodiments, the data processing device provided in at least one embodiment of the present disclosure may be a chip, for example, a system-on-a-chip (SoC). The SoC includes a processor, which may be a single-core processor or a multi-core processor, a memory, an I / O interface, and the like.

[0263] Figure 8 A schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. Figure 8 As shown, the storage medium 200 may be a non-transitory computer-readable storage medium, and one or more computer-readable instructions 201 may be non-transitory stored on the storage medium 200. For example, when the computer-readable instructions 201 are executed by a processor, one or more steps in the data processing method described above may be performed.

[0264] For example, the storage medium 200 may be applied to an electronic device 300 . For example, the storage medium 200 may include a storage device 308 in the electronic device 300 .

[0265] For example, the storage device may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, a flash memory, etc. One or more computer-readable instructions may be stored on the computer-readable storage medium, and the processor may execute the computer-readable instructions to implement various functions of the processor. The storage medium may also store various application programs and various data.

[0266] For example, the storage medium may include a memory card of a smart phone, a cache component of a tablet computer, a hard disk of a personal computer, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), flash memory, or any combination of the above storage media, or other applicable storage media.

[0267] Figure 9 This is a schematic block diagram of an electronic device provided by an embodiment of the present disclosure. Figure 9 As shown, the electronic device 300 is suitable for implementing the data processing method provided by the embodiment of the present disclosure. It should be noted that Figure 9 The components of the electronic device 300 shown are merely exemplary and non-limiting. The electronic device 300 may also have other components according to actual application requirements.

[0268] like Figure 9 As shown, the electronic device 300 may include a processing device 301 (eg, a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to non-transitory computer-readable instructions stored in a memory to implement various functions.

[0269] For example, when the computer readable instructions are executed by the processing device 301, one or more steps of the data processing method according to any of the above embodiments may be executed. It should be noted that for a detailed description of the processing process of the data processing method, reference may be made to the relevant descriptions in the above embodiments of the data processing method.

[0270] For example, the memory may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) 303 and / or cache memory. For example, computer-readable instructions may be loaded from storage device 308 into RAM 303 to execute the computer-readable instructions. Non-volatile memory may include, for example, read-only memory (ROM) 302, a hard disk, an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, a flash memory, and the like. Various applications and various data, such as style images and various data used and / or generated by the applications, may also be stored in the computer-readable storage media.

[0271] For example, the processing device 301, the read-only memory (ROM) 302, and the random access memory (RAM) 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0272] Typically, the following devices may be connected to the input / output (I / O) interface 305: an input device 306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, a flash memory, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other electronic devices wirelessly or by wire to exchange data. Although Figure 9 While an electronic device 300 is shown with various devices, it should be understood that implementation or presence of all illustrated devices is not required, and the electronic device 300 may alternatively implement or possess more or fewer devices. For example, the processing device 301 may control other components in the electronic device 300 to perform desired functions. The processing device 301 may be a device with data processing and / or program execution capabilities, such as a central processing unit (CPU), a tensor processing unit (TPU), or a graphics processing unit (GPU). The central processing unit (CPU) may be of an X86, ARM, or RISC-V architecture. The GPU may be directly integrated into the SOC, directly integrated into the motherboard, or built into the motherboard's northbridge chip.

[0273] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0274] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit does not necessarily limit the unit itself.

[0275] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0276] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the present disclosure is not limited to technical solutions formed by specific combinations of the aforementioned technical features. It also encompasses other technical solutions formed by any combination of the aforementioned technical features or their equivalents, without departing from the scope of the above disclosure. For example, a technical solution formed by replacing the aforementioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0277] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0278] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

[0279] Regarding this disclosure, the following points need to be explained: (1) The drawings of the embodiments of the present disclosure only relate to the structures related to the embodiments of the present disclosure. Other structures may refer to conventional designs.

[0280] (2) In the absence of conflict, the embodiments of the present disclosure and the features therein may be combined with each other to form new embodiments.

[0281] The above description is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. The protection scope of the present disclosure shall be based on the protection scope of the claims.

Claims

1. A data processing method, comprising: Receive a first original tensor, wherein the first original tensor is a 5-dimensional tensor, and the 5 dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a channel number dimension; Performing a dimension conversion process on the first original tensor to obtain a second original tensor, wherein the dimension conversion process includes performing a dimension conversion on p dimensions of the five dimensions, the p dimensions including at least two dimensions of the other four dimensions excluding the channel number dimension, and p is a positive integer greater than or equal to 2 and less than or equal to 4; Performing size expansion on at least one dimension of the second original tensor, and storing a third original tensor obtained by the size expansion in a memory, wherein, in the memory, starting tensor elements on each dimension of the third original tensor are located at starting positions of different storage units, wherein the starting tensor elements are tensor elements corresponding to starting coordinates of each dimension in a preset coordinate system, the memory performs read and write operations with the storage unit as a granularity, and the storage unit stores multiple tensor elements; Receiving shape information of a tensor to be processed and a starting coordinate in the preset coordinate system; Determining, based on the third original tensor, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed, a plurality of requests for loading the tensor to be processed into a cache area; The multiple requests are sent in sequence, and the sub-data returned by each request are sequentially written into the cache area to load the tensor to be processed into the cache area.

2. The data processing method according to claim 1, wherein: The expanded size of the at least one dimension after the size expansion is performed is determined based on the storage capacity of the storage unit.

3. The data processing method according to claim 2, wherein: Expanding the size of at least one dimension of the second original tensor includes: For any dimension of the second original tensor, in response to a total storage capacity of all tensor elements of the second original tensor with the same coordinates in a target dimension and higher than the target dimension not being an integer multiple of the storage capacity of the storage unit, determining to expand the size of the any dimension, and determining the expanded size of the any dimension according to the total storage capacity; In response to the total storage capacity being an integer multiple of the storage capacity of the storage unit, determining not to perform size expansion on any one dimension; In which, in response to any one dimension being the batch dimension, the target dimension is the any one dimension; in response to any one dimension being not the batch dimension, the target dimension is a dimension adjacent to and higher than the any one dimension as indicated by the data storage format of the second original tensor.

4. The data processing method according to claim 3, wherein: The expanded size of any one dimension is such that the total storage capacity of all tensor elements of the third original tensor with the same coordinates in the target dimension is an integer multiple of the storage capacity of the storage unit.

5. The data processing method according to claim 1, wherein: In response to determining that, when loading the to-be-processed tensor, the first dimension cannot be loaded continuously but dimensions lower than the first dimension can be loaded continuously, performing the requested partitioning on the second dimension, wherein each sub-data requested to be loaded is located in the memory or none of the sub-data is located in the memory, and in response to the sub-data loaded by the request are all located in the memory, the sub-data requested to be loaded are from the second original tensor and are stored continuously in the memory, The storage format of the tensor to be processed indicates that the first dimension takes precedence over the second dimension when storing or loading, and the first dimension and the second dimension are adjacent. The data processing method according to claim 1 , wherein: When storing the third original tensor to the memory, the part that does not belong to the original content of the second original tensor due to the size expansion is stored in the memory in a predefined format, wherein the predefined format is used to indicate that the corresponding storage content is requested to be divided as data that is not located in the memory.

7. The data processing method according to claim 5, wherein: In response to satisfying any one of the following conditions, it is determined that the tensor to be processed cannot be loaded continuously in the first dimension: (1) The size of the to-be-processed tensor in the first dimension is not equal to the size of the second original tensor in the first dimension; (2) The first coordinate value of the starting coordinate of the tensor to be processed in the first dimension is not equal to the second coordinate value of the starting coordinate of the third original tensor in the first dimension; (3) The distance interval corresponding to the second dimension is not equal to the predetermined value corresponding to the first dimension, wherein the distance interval corresponding to the second dimension is the distance interval between two target tensor elements stored in the memory, and the two target tensor elements are two tensor elements in the third original tensor that are adjacent in the second dimension, and the predetermined value corresponding to the first dimension is determined based on at least the shape and size of the second original tensor.

8. The data processing method according to claim 7, wherein: The distance interval corresponding to the second dimension is determined at least according to the shape and size of the third original tensor, In response to the storage format of the third original tensor being NDHWC, determining the product of shape sizes of dimensions lower than the second dimension in the third original tensor as the distance interval corresponding to the second dimension, Wherein, N represents the batch dimension, D represents the depth dimension, W represents the width dimension, H represents the height dimension, and C represents the channel number dimension.

9. The data processing method according to claim 7, wherein: In response to the storage format of the third original tensor being N(C / x)DHW(xC): When the second dimension is not the batch dimension, determining a product of a shape size of each dimension lower than the second dimension in the third original tensor and a storage capacity of x tensor elements as a distance interval corresponding to the second dimension; In a case where the second dimension is the batch dimension, determining a product of shape sizes of dimensions lower than the second dimension in the third original tensor as a distance interval corresponding to the second dimension; Wherein, N represents the batch dimension, D represents the depth dimension, W represents the width dimension, H represents the height dimension, C represents the channel number dimension, and x is a positive integer greater than 1.

10. The data processing method according to claim 7, wherein: In response to the second raw tensor, the data storage format is NDHWC: In a case where the first dimension is the channel number dimension, determining a shape size of the second original tensor in the first dimension as a predetermined value corresponding to the first dimension; In the case where the first dimension is not the channel number dimension, the product of the shape size of the second original tensor in the first dimension and the distance interval corresponding to the first dimension is determined as the predetermined value corresponding to the first dimension.

11. The data processing method according to claim 7, wherein: In response to the second original tensor, the data storage format is N(C / x)DHW(xC): In a case where the second dimension is the height dimension, determining a product of a shape size of the second original tensor in the first dimension and a storage capacity of x tensor elements as a predetermined value corresponding to the first dimension; In a case where the second dimension is the depth dimension or the channel number dimension, determining a product of a shape size of the second original tensor in the first dimension and a distance interval corresponding to the first dimension as a predetermined value corresponding to the first dimension; In the case where the second dimension is the batch dimension, a quotient between the shape size of the second original tensor in the first dimension and the storage capacity of the x tensor elements is determined, and a product of the quotient and the distance interval corresponding to the first dimension is determined as a predetermined value corresponding to the first dimension.

12. The data processing method according to any one of claims 1 to 11, wherein: Determining, in combination with the third original tensor, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed, a plurality of requests for loading the tensor to be processed into a cache area, including: Determining, based on the starting coordinates of the tensor to be processed and the shape information of the tensor to be processed, a first request sent among the multiple requests and an initial state in the state machine entered by the first request; Based on the initial state and in combination with the third original tensor, the state machine is used to determine each request among the multiple requests except the first request.

13. The data processing method according to claim 12, wherein: Each request includes a data read address for indicating a starting position for reading data from the memory, a data write address for indicating a starting position for writing data to the cache, and the length of the data requested to be loaded. Determining, based on the starting coordinates of the tensor to be processed and the shape information of the tensor to be processed, a first request sent among the multiple requests and an initial state in the state machine entered by the first request, including: Determining, based on the starting coordinates of the tensor to be processed, that the first request enters the initial state in the state machine; Using the starting coordinates of the tensor to be processed as the initial request coordinates corresponding to the first request; determining a data read address for the first request according to distance intervals corresponding to dimensions in the third original tensor and initial request coordinates corresponding to the first request; Determine a starting address in the cache area for writing the to-be-processed tensor as a data write address for the first request; Determine the data length of the first requested load according to the starting coordinates of the tensor to be processed and the shape information of the tensor to be processed.

14. The data processing method according to claim 13, wherein: Determining, based on the starting coordinates of the tensor to be processed, that the first request enters the initial state in the state machine includes: In response to a first coordinate value of the starting coordinate of the to-be-processed tensor in the first dimension being smaller than a second coordinate value of the starting coordinate of the third original tensor in the first dimension, determining that the initial state is the first state, In response to the first coordinate value being greater than or equal to the second coordinate value and less than a third coordinate value, determining the initial state to be a second state, wherein a difference between the second coordinate value and the third coordinate value is equal to a size of the second original tensor in the first dimension, In response to the first coordinate value being greater than or equal to the third coordinate value, the initial state is determined to be the third state.

15. The data processing method according to claim 12, wherein: Determining, based on the initial state and in combination with the third original tensor, each request other than the first request in the plurality of requests using the state machine, includes: Based on the initial state, using the state machine to determine the request initial coordinates corresponding to each request and the data length loaded by each request; Determining a data read address for each request according to a distance interval corresponding to each dimension in the third original tensor and an initial request coordinate corresponding to each request; The data write address of each request is determined according to the length of the data loaded by each request.

16. The data processing method according to any one of claims 1 to 11, wherein: Performing dimension conversion on the first original tensor to obtain a second original tensor includes: Perform a transposition operation on the first original tensor in the p dimensions to obtain the second original tensor.

17. An electronic device comprising: a memory that non-transitorily stores computer-executable instructions; a processor configured to execute the computer-executable instructions, Wherein, when the computer executable instructions are executed by the processor, the data processing method according to any one of claims 1 to 16 is implemented.

18. A non-transitory computer-readable storage medium, wherein: The non-transitory computer-readable storage medium stores computer-executable instructions, When the computer-executable instructions are executed by a processor, the data processing method according to any one of claims 1 to 16 is implemented.

Images