Data Loading Method, Data Storage Method, Electronic Device, and Storage Medium

Abstract

The present disclosure provides a data loading method, a data storage method, an electronic device, and a storage medium. The data loading method is used to load a tensor to be processed from a first original tensor in a memory into a buffer area, and includes: obtaining a first starting coordinate of a first tensor to be processed in a first coordinate system determined by the first original tensor and a first number of pixels included in the first tensor to be processed; and directly determining, by a hardware device, a second starting coordinate of a second tensor to be processed in the first coordinate system according to the first starting coordinate and the first number of pixels, wherein a position of a last pixel included in the first tensor to be processed is adjacent to the second starting coordinate of the second tensor to be processed in the first coordinate system. In the present disclosure, the hardware device can directly determine the second starting coordinate, thereby omitting the step of resetting the second starting coordinate in a data loading instruction corresponding to the second tensor to be processed, and further simplifying the instruction structure and saving instruction resources.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to a data loading method, a data storage method, an electronic device, and a storage medium. Background Art

[0002] A tensor is a data structure of a multi-dimensional array. Tensor operations are widely used in processors such as parallel processors. For example, in the field of deep learning, the dimensions of the input data, the intermediate data processed during the deep learning process, and the output data are elastic and not exact. Therefore, an elastic data form is required to describe various types of data, and thus the concept of tensors is generated. In the field of deep learning, all data to be operated on can be stored and exist in the form of tensors. If the data is not in the form of tensors, it needs to be first converted into the data structure form of tensors. As an example, a scalar can be regarded as a 0-dimensional tensor, a vector can be regarded as a 1-dimensional tensor, a matrix can be regarded as a 2-dimensional tensor, and a tensor itself can have any number of dimensions. For example, it can be represented as a 5-dimensional array.

[0003] With the development of artificial intelligence and machine learning, new requirements are put forward for many parallel processing devices represented by parallel processors (such as multi-core processors, digital signal processors, etc.). In general computing, the computing units of parallel processors need to process a large amount of data, and these data are generally stored in the storage components of the parallel processors. For example, the storage component can be a high-speed memory. Through data loading instructions, these data can be loaded from the storage component to the buffer for calculation, and through data storage instructions, the data in the buffer can be stored in the memory.

[0004] How to provide a fast and efficient data loading / storing method is crucial for the computing performance of the device. Summary of the Invention

[0005] Embodiments of the present disclosure provide a data loading method, a data storage method, a processor, an electronic device, and a storage medium, which are used to simplify the instruction structure (or instruction logic) of data loading instructions, save instruction resources, and improve the instruction execution efficiency.

[0006] According to a first aspect of the present disclosure, a data loading method is provided for loading a tensor to be processed from a first original tensor in a memory into a buffer area. The data loading method includes: obtaining a first starting coordinate of a first tensor to be processed in a first coordinate system determined by the first original tensor and a first number of pixels included in the first tensor to be processed; and directly determining, by a hardware device, a second starting coordinate of a second tensor to be processed in the first coordinate system according to the first starting coordinate and the first number of pixels, wherein the position of the last pixel included in the first tensor to be processed is adjacent to the second starting coordinate of the second tensor to be processed in the first coordinate system; wherein the first coordinate system includes five dimensions, and the five dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a number of channels dimension. The data determined by the height dimension and the width dimension represents one pixel, and is accumulated step by step to higher dimensions. The number of pixels is not calculated for the number of channels dimension. The order of the five dimensions from the lower dimension to the higher dimension is the width dimension, the height dimension, the depth dimension, the batch dimension, and the number of channels dimension.

[0007] According to a second aspect of the present disclosure, another data loading method is provided, including: receiving a first data loading instruction to obtain a first starting coordinate of a first tensor to be processed in a first coordinate system determined by a first original tensor in a memory, a first number of pixels included in the first tensor to be processed, and a number of fallback pixels for a second tensor to be processed, wherein the first data loading instruction instructs to perform loading the first tensor to be processed from the first original tensor into the buffer area; determining an end coordinate of the last pixel in the first tensor to be processed in the first coordinate system according to the first starting coordinate and the first number of pixels; determining a second starting coordinate of a second tensor to be processed corresponding to a second data loading instruction in the first coordinate system according to the end coordinate and the number of fallback pixels, wherein the second data loading instruction instructs to perform loading the second tensor to be processed from the first original tensor into the buffer area; wherein the first coordinate system includes five dimensions, and the five dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a number of channels dimension. The data determined by the height dimension and the width dimension represents one pixel, and is accumulated step by step to higher dimensions. The number of pixels is not calculated for the number of channels dimension. The order of the five dimensions from the lower dimension to the higher dimension is the width dimension, the height dimension, the depth dimension, the batch dimension, and the number of channels dimension.

[0008] According to a third aspect of the present disclosure, a data storage method is provided for obtaining a tensor to be processed based on a second original tensor in a buffer and writing the tensor to be processed into memory. The data storage method includes: obtaining a third starting coordinate of a third tensor to be processed in a second coordinate system determined by the second original tensor and a third number of pixels included in the third tensor to be processed; and directly determining, by a hardware device, a fourth starting coordinate of a fourth tensor to be processed in the second coordinate system according to the third starting coordinate and the third number of pixels, wherein the position of the last pixel included in the third tensor to be processed is adjacent to the fourth starting coordinate of the fourth tensor to be processed in the second coordinate system; wherein the second coordinate system includes five dimensions, the five dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a number of channels dimension, the data determined by the height dimension and the width dimension is represented as a pixel, and is accumulated step by step to higher dimensions, the number of channels dimension does not calculate the number of pixels, and the order of the five dimensions from the low dimension to the high dimension is the width dimension, the height dimension, the depth dimension, the batch dimension, and the number of channels dimension.

[0009] According to a fourth aspect of the present disclosure, another data storage method is provided, including: receiving a first data storage instruction to obtain a third starting coordinate of a third tensor to be processed in a second coordinate system determined by a second original tensor in a buffer, a third number of pixels included in the third tensor to be processed, and a number of fallback pixels for a fourth tensor to be processed, wherein the first data storage instruction instructs to obtain the third tensor to be processed from the buffer and store the third tensor to be processed in memory; determining an end coordinate of the last pixel in the third tensor to be processed in the second coordinate system according to the third starting coordinate and the third number of pixels; determining a fourth starting coordinate of a fourth tensor to be processed corresponding to a second data storage instruction in the second coordinate system according to the end coordinate and the number of fallback pixels, wherein the second data storage instruction instructs to obtain the fourth tensor to be processed from the buffer and store the fourth tensor to be processed in memory; wherein the second coordinate system includes five dimensions, the five dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a number of channels dimension, the data determined by the height dimension and the width dimension is represented as a pixel, and is accumulated step by step to higher dimensions, the number of channels dimension does not calculate the number of pixels, and the order of the five dimensions from the low dimension to the high dimension is the width dimension, the height dimension, the depth dimension, the batch dimension, and the number of channels dimension.

[0010] According to a fifth aspect of the present disclosure, an electronic device is provided, including a processor and a memory connected to the processor. The processor includes a buffer, and the processor is configured to run computer-executable instructions, and the computer-executable instructions, when run by the processor, implement the data loading method or the data storage method according to the embodiments of the present disclosure.

[0011] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, a data loading method or a data storage method according to an embodiment of the present disclosure is implemented.

[0012] In the present disclosure, according to the first starting coordinate and the first number of pixels of the first tensor to be processed, the second starting coordinate of the second tensor to be processed (i.e., the next tensor to be processed) can be directly determined by a hardware device, without resetting the second starting coordinate in the data loading instruction corresponding to the second tensor to be processed, that is, the subsequent data loading instructions do not need to reset the starting coordinates of the corresponding tensors to be processed, thereby simplifying the instruction structure (or instruction logic) of the data loading instructions, saving instruction resources, and improving the instruction execution efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0014] Figure 1 shows a schematic structural diagram of a General-Purpose computing on Graphics Processing Unit (GPGPU);

[0015] Figure 2 shows a schematic structure of a tensor;

[0016] Figure 3A shows a schematic diagram of the data storage format of NDHWC;

[0017] Figure 3B shows a schematic diagram of the data storage format of N(C / x)DHW(xC), where x = 32;

[0018] Figure 4 shows a schematic diagram of sequentially obtaining tensors to be processed in units of pixel points in related technologies;

[0019] Figure 5 shows a schematic flowchart of a data loading method provided by at least one embodiment of the present disclosure;

[0020] Figure 6 shows a schematic diagram of continuously obtaining tensors to be processed according to the data storage format of NDHWC according to an embodiment of the present disclosure;

[0021] Figure 7A Shows a state diagram of a state machine provided according to some embodiments of the present disclosure;

[0022] Figure 7B Shows the state transition of the data storage format for NDHWC in the PerW division manner;

[0023] Figure 8A Shows a schematic diagram of setting boundary values for a first original tensor according to an embodiment of the present disclosure;

[0024] Figure 8B Shows a schematic diagram of continuous data acquisition when setting boundaries for a first original tensor according to an embodiment of the present disclosure;

[0025] Figure 8C Shows a schematic diagram of continuously obtaining a tensor to be processed in a continuous manner according to the set step size according to an embodiment of the present disclosure;

[0026] Figure 9 Shows a schematic diagram of obtaining a second tensor to be processed for the data storage format of NDHWC according to an embodiment of the present disclosure;

[0027] Figure 10 Shows a schematic flowchart of another data loading method provided by at least one embodiment of the present disclosure;

[0028] Figure 11 Shows a schematic flowchart of a data storage method provided by at least one embodiment of the present disclosure;

[0029] Figure 12 Shows a schematic flowchart of another data storage method provided by at least one embodiment of the present disclosure;

[0030] Figure 13 Shows a schematic block diagram of an electronic device according to some embodiments of the present disclosure;

[0031] Figure 14 Shows a block diagram of an example computing device implementing some embodiments of the present disclosure; and

[0032] Figure 15 Shows a schematic block diagram of a computer-readable storage medium according to some embodiments of the present disclosure. Detailed implementation manners

[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the following will clearly and completely describe the technical solutions of the embodiments of the present disclosure with reference to the accompanying drawings of the embodiments of the present disclosure. Obviously, the described embodiments are a part rather than all of the embodiments of the present disclosure. All other embodiments obtained by those of ordinary skill in the art based on the described embodiments of the present disclosure without creative efforts shall fall within the scope of protection of the present disclosure.

[0034] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure shall have the ordinary meanings understood by those of ordinary skill in the art to which the present disclosure pertains. The terms "first", "second", and similar terms used in the present disclosure do not denote any order, quantity, or importance, but are only used to distinguish different components. Words such as "including" or "comprising" mean that the elements or items appearing before the word cover the elements or items listed after the word and their equivalents, without excluding other elements or items. Terms such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. Terms such as "upper", "lower", "left", "right", etc. are only used to represent relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship may also change accordingly. To keep the following description of the embodiments of the present disclosure clear and concise, some detailed descriptions of known functions and known components are omitted in the present disclosure.

[0035] Figure 1 A schematic structural diagram of a GPGPU is shown. As Figure 1 shown, a GPGPU is actually an array of programmable multi-processors. For example, the programmable multi-processors can be Streaming Processor Clusters (SPCs), such as including Figure 1 the Streaming Processor Cluster 1 shown, …, the Streaming Processor Cluster M, where M is a positive integer. In a general-purpose graphics processor, 1 streaming processor cluster processes one computing task, or multiple streaming processor clusters process one computing task. As an example, data sharing between multiple streaming processor clusters is performed through a global cache or High Bandwidth Memory (HBM).

[0036] As Figure 1 shown, taking the Streaming Processor Cluster 1 as an example, 1 streaming processor cluster can include multiple Compute Units (CUs), such as Figure 1The computing units 1, 2, ..., K in it, where K is a positive integer. Each computing unit is used to perform arithmetic and logical operations, such as accumulation, reduction, conventional addition, subtraction, multiplication, division, etc. A computing unit may include multiple cores (also known as computing cores or computing kernels), and each computing core includes an arithmetic logic unit (ALU), a floating-point computing unit, etc. The computing core is used to perform specific computing tasks. In addition, the computing unit also includes registers (such as Figure 1 the register file in it) and shared memory, which are used to hierarchically store the source data and destination data related to the computing task. The shared memory in a computing unit is used to share data among the cores of the computing unit. In addition, the buffer can be understood as being used to share data among the computing units within the streaming processor cluster.

[0037] In parallel computing, computing tasks are generally executed by multiple threads. These threads are divided into multiple thread blocks before being executed in a general-purpose graphics processor (or parallel computing processor), and then the multiple thread blocks are distributed to each computing unit via a thread block distribution module ( Figure 1 not shown in it). All the threads in a thread block must be assigned to the same computing unit for execution. At the same time, the thread block will be split into the smallest execution thread bundle (or simply called thread bundle, warp), and each thread bundle contains a fixed number (or less than this fixed number) of threads. For example, 32 threads. Multiple thread blocks can be executed in the same computing unit or in different computing units.

[0038] In each computing unit, the thread bundle scheduling / distribution module ( Figure 1 not shown in it) schedules and allocates the thread bundles so that the multiple computing cores in the computing unit can run the thread bundles. According to the number of computing cores in the computing unit, the multiple thread bundles in a thread block can be executed simultaneously or time-divisionally. The multiple threads in each thread bundle will execute the same instructions. The memory execution instructions will be issued to the shared memory in the computing unit or further issued to the middle-level cache or global cache or high-bandwidth memory for read and write operations, etc.

[0039] As Figure 1 shown, general computing operations, such as computing operations on matrices in the field of artificial intelligence, usually require a large amount of data. These data are usually stored in a memory, such as in a high-bandwidth memory HBM. When performing general computing operations, data needs to be loaded from the memory (Load operation), and when obtaining the computing result, data needs to be stored in the memory (Store operation). The storage method of data in the memory will affect the memory access bandwidth, and thus affect the hardware utilization rate of the computing unit.

[0040] For example, general computing operations include General Matrix Multiplication (GEMM). As an example, the data for general matrix multiplication is represented as two 5D arrays, such as the matrix multiplication calculation of two tensors A and tensor B. In addition, general computing operations also include convolution operations, which are manifested as data dot products. It can be understood that in the field of artificial intelligence, other computing operations are also involved, which will not be listed one by one here. The data involved in these calculations is usually embodied in the form of tensors.

[0041] For example, for a certain tensor A in the buffer, its shape dimensions can be represented by a1, a2, a3, a4, a5. a1, a2, a3, a4, a5 respectively indicate the dimensions of the tensor data in 5 dimensions, and a1, a2, a3, a4, a5 are positive integers. For example, the 5 dimensions include [N, D, H, W, C]. The N dimension represents the batch size, that is, N represents the batch dimension, which is the number of data samples captured in one training. The D dimension represents the depth dimension, the H dimension represents the height dimension of the input data, the W dimension represents the width dimension of the input data, and the C dimension represents the number of channels dimension. For example, taking tensor A as an example, a1 can be the N dimension size, a2 can be the D dimension size, a3 can be the H dimension size, a4 can be the W dimension size, and a5 can be the C dimension size. Of course, the present disclosure does not make specific limitations on this.

[0042] As an example, Figure 2 shows a schematic structure of a tensor. In Figure 2 the shown tensor, a1 is the N dimension size and is equal to 1, a2 is the D dimension size and is equal to 1, a3 is the H dimension size and is equal to 5, a4 is the W dimension size and is equal to 4, and a5 is the C dimension size and is equal to 64. For example, Figure 2 the pixel elements of the tensor in

[0043] The placement of tensors in the memory (such as memory or buffer) can have multiple formats, called data storage formats (layout). The data storage format is used to indicate the storage order and dimension arrangement of tensors in the storage component. The following uses the Figure 2 shown tensor to describe different data storage formats.

[0044] In the related art, the data storage format can include NDHWC, also known as the Linear mode. Figure 3A shows a schematic diagram of the NDHWC data storage format.

[0045] For example, for the NDHWC linear mode, as Figure 3A shown, from the first channel (a5 = 0,Figure 3A starting from the first element of c0 in Figure 3A element 0 in, and then storing the first element of the second channel (a5 = 1, Figure 3A the first element of c1 in Figure 3A element 20 in, and so on until the first elements of all channels are laid out, for example, until the first element of the 64th channel (a5 = 63, Figure 3A c63 in Figure 3A element 1260 in. After that, select the second element of the first channel (a5 = 0, Figure 3A the second element of c0 in Figure 3A element 1 in, and then store the second element of the second channel (a5 = 1, Figure 3A the second element of c1 in Figure 3A element 21 in, and so on until the second elements of all channels are laid out, and so on.

[0046] In the related art, the data storage format may also include N(C / x)DHW(xC), also known as the Interleave mode, where x can be set to 8, 16, 32, etc. according to needs.

[0047] The N(C / x)DHW(xC) data storage format is similar to the NDHWC data storage format, but there is a key difference. In the layout of N(C / x)DHW(xC), a5 channels are divided into a5 / x groups, with each group having x channels: the first group consists of channels a5 = 0 to a5 = x - 1, the second group consists of channels a5 = x to a5 = 2x - 1, and each group is arranged in the NDHWC format.

[0048] Figure 3B Shows a schematic diagram of the data storage format of N(C / x)DHW(xC), where x = 32.

[0049] As Figure 3B shown, 64 channels are divided into two groups, with each group having 32 channels. The first group consists of channels a5 = 0 ( Figure 3B c0 in) to a5 = 31 ( Figure 3B c31 in), and the second group consists of channels a5 = 32 to a5 = 63. Then each group is arranged in the NDHWC format. <s

[0050] In memory, for example, for a certain tensor B, similar to a certain tensor A in the buffer, the tensor B can be stored in memory according to one of the two data storage formats described above. The shape dimensions of the tensor can be similarly represented as b1×b2×b3×b4×b5, where b1, b2, b3, b4, b5 respectively indicate the dimensions of the tensor B in these 5 dimensions and are all positive integers.

[0051] It can be understood that in the related art and possible future developments, the data storage format of tensors is not limited to the above-described N(C / x)DHW(xC) data storage format and NDHWC data storage format. The method described in this disclosure does not limit this. Further, for multiple tensors that can be stored in the memory and cache during the calculation process, generally the storage space of the memory is much larger than that of the cache, but it is farther from the calculation unit, and the data transfer efficiency is lower than that of the cache.

[0052] In the related art, during the calculation process of a processing device, a large amount of calculation data will be generated, for example, in the form of tensors, which can be temporarily stored in the cache. For example, the cache here can refer to Figure 1 the cache in the streaming processor cluster shown in Figure 1 Furthermore, these data can also be transferred from the cache or directly stored in the memory. For example, the memory can be

[0053] It can be understood that in this article, the data storage process of storing the tensor in the cache into the memory and the data loading process of loading the tensor in the memory into the cache can be implemented in a similar manner. Therefore, for the sake of convenience of description, in some embodiments or examples, only the data loading process is described as an example, and those skilled in the art can apply it similarly to the data storage process.

[0054] For example, Figure 4 shows a schematic diagram of sequentially obtaining the tensor to be processed in units of pixel points in the related art.

[0055] In Figure 4 , tensor A represents the first original tensor in the memory. Tensor A is placed in the memory in the NDHWC data storage format, that is, tensor A is a 5D array. In Figure 4 , only the three dimensions of W, H, and C are schematically shown. For example, Figure 4The left side shows the first coordinate system of the dimensions of tensor A, where the dimensions are the W dimension, the H dimension, and the C dimension respectively. In addition, the data determined by the H dimension and the W dimension is represented as a pixel, and it accumulates step by step to higher dimensions. The C dimension does not affect the calculation of the number of pixels. That is, it accumulates step by step in the order of the width dimension, the height dimension, the depth dimension, and the batch dimension. In Figure 4 each pixel point is shown as a square, and tensor A covers all the squares, that is, it includes both white squares and gray-shaded squares. The starting point of this first original tensor is represented as (C = 0, W = 0, H = 0), for example, to indicate the position of this first original tensor in memory.

[0056] During the data loading process, all or part of the data in tensor A can be loaded into the buffer unit by unit of pixel points through a loading instruction. For example, when loading tensor B (i.e., the tensor to be processed) in tensor A into the buffer, the loading instruction can indicate the starting point (C = 0, W = 3, H = 0) of tensor B in the first coordinate system of tensor A and the number of pixel points copy_pixel_num included in tensor B (for example, 27). During the data loading process, as shown by the dashed arrow in Figure 4 starting from the starting point (C = 0, W = 3, H = 0) (i.e., the 4th pixel point in the first row of tensor A), data is obtained from tensor A pixel by pixel until the number of obtained pixel points reaches copy_pixel_num. In Figure 4 the example of, only the three dimensions W, H, and C are shown. It can be understood that if the data of these 3 dimensions still does not reach copy_pixel_num, data of higher dimensions can be further obtained, which is not limited here.

[0057] It is found in the present disclosure that after executing the first loading instruction, before executing subsequent loading instructions, it is necessary to reset the starting point (hereinafter also referred to as "starting coordinate") of the corresponding tensor to be processed in the first coordinate system of the first original tensor in the subsequent loading instructions.

[0058] The present disclosure provides a data loading method, a processor, an electronic device, and a non-transitory computer-readable storage medium. In the present disclosure, according to the first starting coordinate and the first number of pixels of the first tensor to be processed, the second starting coordinate of the second tensor to be processed (i.e., the next tensor to be processed) can be directly determined by a hardware device, without resetting the second starting coordinate in the data loading instruction corresponding to the second tensor to be processed. That is, subsequent data loading instructions do not need to reset the starting coordinates of the corresponding tensors to be processed, thereby simplifying the instruction structure (or instruction logic) of the data loading instructions, saving instruction resources, and improving the instruction execution efficiency.

[0059] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings, but the present disclosure is not limited to these specific embodiments.

[0060] Figure 5 It is a schematic flowchart of a data loading method provided for at least one embodiment of the present disclosure. As Figure 5 shown, the data loading method at least includes step S110 and step S120.

[0061] In step S110: Obtain the first starting coordinate of the first tensor to be processed in the first coordinate system determined by the first original tensor and the number of first pixels included in the first tensor to be processed.

[0062] In step S120: According to the first starting coordinate and the number of first pixels, directly determine the second starting coordinate of the second tensor to be processed in the first coordinate system by a hardware device, where the position of the last pixel included in the first tensor to be processed is adjacent to the second starting coordinate of the second tensor to be processed in the first coordinate system.

[0063] The data loading method provided for at least one embodiment of the present disclosure is used to load the tensor to be processed from the first original tensor in the memory to the buffer area. Here, the memory can be, for example, a high-bandwidth memory HBM, and the buffer area is, for example, a buffer buffer in the streaming processor cluster. The present disclosure does not limit this.

[0064] In the data loading method provided for the embodiments of the present disclosure, the data storage format of the first original tensor in the memory is the same as the data storage format of the tensor to be processed in the buffer area. For example, both are stored in the NDHWC linear manner or both are stored in the N(C / x)DHW(xC) interleaved manner. The present disclosure does not limit this. The data loading method proposed by the present disclosure is applicable to the above two data storage formats. Regarding the characteristics of the above two data storage formats, reference can be made to the description above in combination with Figure 3A - Figure 3B and will not be repeated here.

[0065] For example, the first tensor to be processed and the second tensor to be processed in the above steps S110 and S120 can be used for convolution operations in the computing units within the processor.

[0066] For example, the first coordinate system determined by the first original tensor in the above steps S110 and S120 includes 5 dimensions, and the 5 dimensions include the batch N dimension, the depth D dimension, the height H dimension, the width W dimension, and the number of channels C dimension. Among them, the data determined by the height dimension and the width dimension represents one pixel (or, can also be called one element), and accumulates step by step to higher dimensions, and the number of channels dimension does not calculate the number of pixels, that is, accumulates step by step in the order of the width dimension, the height dimension, the depth dimension, and the batch dimension. For example, as Figure 4As shown, the values of each W dimension and H dimension can determine a pixel. For example, W = 0 and H = 0 correspond to the first pixel (or element) in tensor A, and W = 1 and H = 0 correspond to the second pixel in tensor A. In a tensor, the number of channels dimension does not affect the number of pixels. That is, starting from the first starting coordinate of the first tensor to be processed in the first original tensor as the starting point for data acquisition, pixel data is acquired in the order of the width dimension, height dimension, depth dimension, and batch dimension until the number of acquired pixels reaches the number of the first pixels included in the first tensor to be processed.

[0067] For example, the hardware device in the above step S120 can be a Tensor Memory Accelerator (TMA). TMA is a hardware unit specifically designed to optimize tensor operations and is commonly found in high-performance computing and AI acceleration scenarios. By efficiently managing the loading, storage, and address calculation of tensor data, TMA can significantly improve the performance of tensor operations. For example, TMA can directly parse the dimension information, starting address, and the number of pixels included in the tensor to be processed, and quickly calculate the starting address of the next tensor to be processed in memory based on this information. This can not only reduce the address calculation overhead at the software level but also improve the parallelism and efficiency of data access.

[0068] In the embodiments of the present disclosure, according to the first starting coordinate of the first tensor to be processed and the number of the first pixels included in the first tensor to be processed, the coordinate of the last pixel included in the first tensor to be processed in the first coordinate system of the first original tensor (also referred to as the "ending coordinate") can be determined. Since the first tensor to be processed and the second tensor to be processed are adjacent in the first original tensor, specifically, the position of the last pixel included in the first tensor to be processed is adjacent to the second starting coordinate of the second tensor to be processed in the first coordinate system. Therefore, based on the determined ending coordinate of the first tensor to be processed, the second starting coordinate of the second tensor to be processed can be determined.

[0069] In some embodiments of the present disclosure, the above step S120 may include steps S121 and S122.

[0070] In step S121: According to the first starting coordinate and the number of the first pixels, determine multiple requests for loading the first tensor to be processed, where the multiple requests are used to sequentially acquire data from the first original tensor with the first starting coordinate as the starting point until the number of acquired data is equal to the number of the first pixels.

[0071] In step S122: Based on the last request among the multiple requests, determine the second starting coordinate of the second tensor to be processed in the first coordinate system.

[0072] For the multiple requests determined in step S121 above, in the actual implementation of the processor, it can be achieved by setting up a state machine. For example, multiple parameters required to determine the above multiple requests can be set for the state machine, and according to the specific values of the parameters and the initial parameters of the first original tensor and the tensor to be processed (for example, the first tensor to be processed), the specific tensor to be processed in the first original tensor is loaded into the buffer area.

[0073] The implementation solution related to determining multiple requests for loading the tensor to be processed as described above will be described below in conjunction with the state machine.

[0074] For example, step S121, determining multiple requests for loading the first tensor to be processed according to the first starting coordinate and the first number of pixels, may include step S1211 and step S1212.

[0075] In step S1211: According to the first starting coordinate and the first number of pixels, determine the first request among the multiple requests and the initial state when the first request enters the state machine, where the first request indicates the number of pixels to be acquired by the first request.

[0076] In step S1212: Based on the initial state, in combination with the first number of pixels, the number of pixels to be acquired by the first request, and the data storage format of the first original tensor, use the state machine to determine each request among the multiple requests after the first request.

[0077] For example, for step S1211, when performing the task of loading the first tensor to be processed in the first original tensor in the memory into the buffer area, the first request can be determined first according to the initial parameters of the first tensor to be processed. For example, based on the first number of pixels (copy_pixel_num) included in the first tensor to be processed, the data storage format of the first original tensor (for example, NDHWC or N(C / x)DHW(xC)), and the first starting coordinate of the first tensor to be processed (the first starting coordinate indicates the position of the first pixel of the first tensor to be processed in the first coordinate system of the first original tensor, for example, represented as its positions in 5 dimensions, c_coord_b, w_coord_b, h_coord_b, d_coord_b, n_coord_b), determine the first request among the multiple requests and the initial state when the first request enters the state machine, where the first request indicates the number of pixels to be acquired by the request (req_size_1).

[0078] For example, Figure 6 FIG. shows a schematic diagram of continuously acquiring the tensor to be processed according to the data storage format of NDHWC according to an embodiment of the present disclosure.

[0079] Specifically, taking the data storage format of the first original tensor as NDHWC, the number of first pixels copy_pixel_num as 40, and the first starting coordinates as w_coord_b = 2, h_coord_b = 2, d_coord_b = 0, n_coord_b = 0, c_coord_b = 0 as an example, combined with Figure 6 Describe how to determine the requested partitioning method and the first request.

[0080] In Figure 6 In the example shown, the first original tensor corresponds to the data shown in the square in the figure. Among them, the size of the first original tensor in the C dimension is 8 (the C dimension is not shown in the figure), the size in the W dimension is 4 (W_dim = 4), the size in the H dimension is 4 (H_dim = 4), the size in the D dimension is 2 (D_dim = 2), and the size in the N dimension is 2 (N_dim = 2). Taking the N dimension as an example, N_dim = 2 corresponds to Figure 6 N0 and N1 in Figure 6 Based on the above parameters, the total number of pixel points included in the first original tensor can be obtained as 64. The starting point coordinates of the first tensor to be processed in the first original tensor are expressed as w_coord_b = 2, h_coord_b = 2, d_coord_b = 0, n_coord_b = 0, c_coord_b = 0, and the number of pixels included in the first tensor to be processed copy_pixel_num = 40. Based on the above information, it can be obtained that the starting position of the first tensor to be processed in the first original tensor is Figure 6 The pixel points where W = 2 and H = 2 in the N = 0 and D = 0 dimensions shown in

[0081] As Figure 6 shown, in the first original tensor, starting from the starting pixel point corresponding to the first starting coordinates (the pixel points where W = 2 and H = 2 in the N = 0 and D = 0 dimensions), according to the data storage format of the data (i.e., NDHWC), in the order from low dimension to high dimension (i.e., in the order of C, W, H, D, N, as shown by the data acquisition schematic arrow in Figure 6 ), sequentially acquire data until the number of acquired data is equal to the number of pixels included in the first tensor to be processed.

[0082] In Figure 6In the illustrated example, when determining the partitioning method of the requests and the first request, since the data storage format is NDHWC, that is, the C dimension is the lowest dimension, it is necessary to first determine whether the first tensor to be processed can be continuously loaded in the C dimension. Specifically, in response to the size relationship between the first tensor to be processed and the first original tensor in the C dimension such that the first tensor to be processed cannot be continuously loaded in the C dimension during loading, the PerW request partitioning method is adopted, that is, the first tensor to be processed is partitioned request by request for each W (each pixel), that is, each pixel is loaded into the buffer by one request.

[0083] For example, the steps of determining whether the first tensor to be processed can be continuously loaded in the C dimension include: in response to the size of the first tensor to be processed in the C dimension (copy_c) not being equal to the size of the first original tensor in the C dimension (tensor_c), and / or the first coordinate value (c_coord_b) of the first starting coordinate of the first tensor to be processed in the C dimension not being equal to the coordinate value of the starting coordinate of the first original tensor in the C dimension (for example, 0), it is determined that the tensor to be processed cannot be continuously loaded in the C dimension. Summarized as follows:

[0084] When any of the following conditions holds, it is determined that the data of the first tensor to be processed is not continuous in the C dimension:

[0085] (1) The first coordinate value c_coord_b of the first starting coordinate of the first tensor to be processed in the C dimension is not equal to 0;

[0086] (2) The size copy_c of the first tensor to be processed in the C dimension is greater than the size tensor_c of the first original tensor in the C dimension;

[0087] (3) The size copy_c of the first tensor to be processed in the C dimension is less than the size tensor_c of the first original tensor in the C dimension.

[0088] For example, when the PerW request partitioning method is adopted, the data loaded by each request has different coordinates in the C dimension, but the same coordinates in the W dimension, H dimension, D dimension, and N dimension. At this time, it can be understood that without considering the channel number C direction, the first original tensor is unfolded into a one-dimensional vector composed of multiple pixels (pixel) in the order of WHDN, and the data loaded by each request is the data within one pixel, and the next request can be to load the data in the adjacent next pixel, that is, the requests are partitioned in a pixel-skipping manner.

[0089] For example, in step S1211, after determining the partitioning method, for example, in the PerW manner, the first request among multiple requests for loading the first tensor to be processed is determined. Here, the first request is used to load the starting pixel corresponding to the first starting coordinate of the first tensor to be processed. The initial state of the first request is determined based on the starting coordinate of the first tensor to be processed (for the specific determination method, please refer to the following text). Also, the number of pixels to be acquired by the first request is req_size_1 = 1 because the subsequent requests are partitioned in a per-pixel manner. The specific acquisition order is as shown by the arrow in Figure 6 until the number of pixels to be acquired is equal to the number of pixels of the first tensor to be processed, copy_pixel_num = 40.

[0090] For example, in response to determining that the first tensor to be processed can be continuously loaded in the C dimension, it is then continued to determine whether the first tensor to be processed can be continuously loaded in the W dimension adjacent to and higher than the C dimension. In response to the inability to continuously load in the W dimension and the ability to continuously load in the C dimension lower than the W dimension, the PerH request partitioning method is adopted, that is, the first tensor to be processed is partitioned by rows for requests. That is to say, each row of pixels is loaded into the buffer by one request.

[0091] Specifically, taking the data storage format as NDHWC, the coordinate value of the starting coordinate of the first original tensor in the W dimension is 0, and the first tensor to be processed can be continuously loaded in the C dimension as an example, the first tensor to be processed is determined to be discontinuous in the W dimension when any of the following conditions holds:

[0092] (1) The starting coordinate of the first tensor to be processed in the W dimension is not equal to 0;

[0093] (2) The size copy_w of the first tensor to be processed in the W dimension is greater than the size tensor_w of the first original tensor in the W dimension;

[0094] (3) The size copy_w of the first tensor to be processed in the W dimension is less than the size tensor_w of the first original tensor in the W dimension.

[0095] At this time, when the C dimension is continuous, the coordinates of the data to be loaded by each request in the C dimension and the W dimension are different, but the coordinates in the H dimension, D dimension, and N dimension are the same. At this time, it can be understood that the data loaded by each request belongs to the same row, that is, the requests are partitioned in the H dimension, and the data loaded by different requests is located in different rows.

[0096] It can be understood that, in the case where the data storage format of the first original tensor and the first tensor to be processed is NDHWC, the division rules of other dimensions are deduced accordingly and will not be repeated here. According to the above description, the division method of the request can also include: according to the request division rule of the D dimension (PerD, that is, the data in the H dimension cannot be loaded continuously but can be loaded continuously in the W and C dimensions below the H dimension), according to the request division rule of the N dimension (PerN, that is, the data in the D dimension cannot be loaded continuously but can be loaded continuously in the H, W, and C dimensions below the D dimension) and according to the request division rule of the overall dimension (Per1, that is, the data in the N dimension cannot be loaded continuously but can be loaded continuously in the D, H, W, and C dimensions below the N dimension).

[0097] In an embodiment of the present disclosure, the request partitioning logic of the first to-be-processed tensor whose data storage format is N(C / x)DHW(xC) is the same as that of NDHWC, except that the dimension arrangement of N(C / x)DHW(xC) is different from that of NDHWC. For N(C / x)DHW(xC), due to its special interleaved structure, (xC) is necessarily continuous by default, so it is considered that the lowest dimension is the W dimension, followed by the H dimension, then the D dimension, then the C dimension, and the highest is the N dimension. Regarding the conditions for not being able to load continuously when the data storage format is N(C / x)DHW(xC) and the division principles of the requests, please refer to the relevant content of NDHWC, which will not be listed one by one here.

[0098] In some embodiments of the present disclosure, for step S1211, determining the first request among multiple requests and the initial state of the first request entering the state machine based on the first starting coordinate and the first number of pixels may include: determining the initial state of the first request entering the state machine based on the first starting coordinate of the first tensor to be processed; using the first starting coordinate of the first tensor to be processed as the request initial coordinate corresponding to the first request; determining the data read address of the first request based on the shape size of the first original tensor, the request initial coordinate corresponding to the first request, and the data storage format of the first tensor to be processed; determining the starting address of writing the first tensor to be processed in the cache as the data write address of the first request; and determining the number of pixels to be obtained by the first request based on the first number of pixels included in the first tensor to be processed, the data storage format of the first original tensor, the first starting coordinate of the first tensor to be processed, and the shape size of the first original tensor.

[0099] For example, when the data storage format of the first original tensor is NDHWC, for multiple requests for loading the first to-be-processed tensor, determining the data read address of the nth request in the multiple requests includes:

[0100] Calculate the data reading address Addr1_n of the nth request according to the following formula:

[0101] Addr1_n = u_addr_base + ((((n_coord * tensor_d + d_coord) * tensor_h + h_coord) * tensor_w + w_coord) * tensor_c + c_coord),

[0102] where u_addr_base represents the storage address in memory of the pixel at the starting coordinate position of the first original tensor, n_coord, d_coord, h_coord, w_coord, c_coord represent the request initial coordinates corresponding to the nth request, and tensor_n, tensor_d, tensor_h, tensor_w, tensor_c represent the shape dimensions of the first original tensor in 5 dimensions.

[0103] For example, when the data storage format of the first original tensor is N(C / x)DHW(xC), for multiple requests for loading the first tensor to be processed, determining the data reading address of the nth request among the multiple requests includes:

[0104] Calculate the data reading address Addr2_n of the nth request according to the following formula:

[0105] Addr2_n = u_addr_base +

[0106] n_coord * tensor_c * tensor_d * tensor_h * tensor_w +

[0107] c_coord * tensor_d * tensor_h * tensor_w +

[0108] d_coord * tensor_h * tensor_w * xC +

[0109] h_coord * tensor_w * xC +

[0110] w_coord * xC,

[0111] Among them, u_addr_base represents the storage address in memory of the pixel at the starting coordinate position of the first original tensor, n_coord, d_coord, h_coord, w_coord, c_coord represent the initial coordinates of the request corresponding to the nth request, and tensor_n, tensor_d, tensor_h, tensor_w, tensor_c represent the shape sizes of the first original tensor in five dimensions, where xC represents the size occupied by x channels.

[0112] For example, for multiple requests for loading the first tensor to be processed, determining the data write address Addr3_n of the nth request among the multiple requests includes:

[0113] Addr3_n = b_addr_base + req_size_1 + req_size_2 + … + req_size_n-1,

[0114] Among them, b_addr_base is the starting address for writing the first tensor to be processed in the buffer, and req_size_1, req_size_2, ..., req_size_n-1 represent the number of pixels to be obtained by the first n-1 requests respectively.

[0115] For the first sent request, its corresponding initial coordinates of the request are the first starting coordinates of the first tensor to be processed. Thus, referring to the above formula, the data read address of the first request can be calculated. The starting address for writing the first tensor to be processed in the buffer is used as the data write address of the first request.

[0116] The data length loaded by the first request can be determined according to parameters such as the determined partitioning method of the request, the first starting coordinates of the first tensor to be processed, and the size of the first original tensor. For example, if the sum of the first coordinate value t_coord_b of the first starting coordinates of the first tensor to be processed in the first dimension and the shape size copy_t of the first tensor to be processed in the first dimension is less than the second coordinate value (i.e., the coordinate value of the starting coordinate of the first original tensor in the first dimension, for example, 0), the data length loaded by the first request is the shape size copy_t of the first tensor to be processed in the first dimension. For example, if the sum of the first coordinate value t_coord_b of the first starting coordinates of the first tensor to be processed in the first dimension and the shape size copy_t of the first tensor to be processed in the first dimension is greater than or equal to the second coordinate value (i.e., the coordinate value of the starting coordinate of the first original tensor in the first dimension, for example, 0), the data length loaded by the first request is the absolute value of the first coordinate value t_coord_b. The above data length represents the number of pixels to be obtained by the request.

[0117] Considering the advantages of the state machine having a clear logical structure, being easy to maintain and expand, and being particularly suitable for processing logical scenarios with multiple conditions and branches, in the present disclosure, the state machine is adopted to automatically update the request initial coordinates and the loaded data length corresponding to each request, avoiding complex conditional nesting, with clear logic, easy to maintain, and strong scalability.

[0118] For example, for step S 1211< / , after determining the first request among the multiple requests, the initial state of the first request entering the state machine can be determined based on the first starting coordinate of the first tensor to be processed, which may specifically include: in response to the first coordinate value of the first starting coordinate of the first tensor to be processed in the first dimension being less than the second coordinate value of the starting coordinate of the first original tensor in the first dimension, determining the initial state as the first state; in response to the first coordinate value being greater than or equal to the second coordinate value and less than the third coordinate value, determining the initial state as the second state, where the difference between the second coordinate value and the third coordinate value is equal to the size of the first original tensor in the first dimension; and in response to the first coordinate value being greater than or equal to the third coordinate value, determining the initial state as the third state.

[0119] It should be noted that the "first dimension" mentioned above is determined by the data storage format of the first original tensor and the tensor to be processed and the request partitioning method. For example, in response to the data storage format being NDHWC and the request partitioning method being PerW, the first dimension is the C dimension; in response to the data storage format being NDHWC and the request partitioning method being PerH, the first dimension is the W dimension.

[0120] For example, for the above step S 1212< / , based on the initial state, in combination with the first number of pixels, the number of pixels to be obtained by the first request, and the data storage format of the first original tensor, the state machine is used to determine each request after the first request among the multiple requests, including: based on the initial state, using the state machine to determine the request initial coordinates of the second request after the first request and the number of pixels to be obtained by the second request; determining the data read address of the second request according to the shape and size of the first original tensor, the request initial coordinates corresponding to the second request, and the data storage format of the first original tensor; determining the data write address of the second request according to the number of pixels to be obtained by the second request; updating the state machine based on the information related to the second request, and sequentially determining the subsequent requests in each request based on the updated state machine.

[0121] Figure 7A Shows a state diagram of the state machine provided according to some embodiments of the present disclosure.

[0122] For example, the states of the state machine include a first state s0, a second state s1, and a third state s2. The initial state of entering the state machine is determined by the starting coordinates c_coord_b, w_coord_b, h_coord_b, d_coord_b, and n_coord_b of the tensor to be processed (e.g., the first tensor to be processed).

[0123] For example, in response to the first coordinate value t_coord_b of the starting coordinate of the first tensor to be processed in the first dimension being less than the second coordinate value of the starting coordinate of the first original tensor in the first dimension, the initial state is determined to be the first state; in response to the first coordinate value being greater than or equal to the second coordinate value and less than the third coordinate value, the initial state is determined to be the second state, where the difference between the second coordinate value and the third coordinate value is equal to the size of the first original tensor in the first dimension; in response to the first coordinate value being greater than or equal to the third coordinate value, the initial state is determined to be the third state.

[0124] Reference Figure 7A , the tensor_t direction represents the data in the first dimension. The data with coordinates in the t dimension (i.e., the first dimension) less than the second coordinate value and greater than the third coordinate value is not in memory or not within the data range of the first original tensor ( Figure 7A the part shown by the diagonal shading, and this part of the data can be called invalid data), and the data with the size of the t dimension between the second coordinate value and the third coordinate value ( Figure 7A the white part) is stored in memory, which is the actual size of the first original tensor data in the first dimension.

[0125] The state only switches to itself and adjacent states. For example Figure 7A in it, the first state s0 can jump to the first state s0 or the second state s1, the second state s1 can jump to the first state s0, the second state s1, or the third state s2, and the third state s2 can jump to the first state s0, the second state s1, or the third state s2.

[0126] Figure 7A The 6 cases (① to ⑥) in

[0127] For example Figure 7A show the state transition changes that the tensor to be processed (e.g., the first tensor to be processed) in different sizes in the first dimension has to go through. In case ① in

[0128] For example Figure 7AIn case ②, the first coordinate value is less than the second coordinate value, and the sum of the first coordinate value and the size of the tensor to be processed in the first dimension is greater than or equal to the second coordinate value and less than the third coordinate value. The state machine loops and jumps between the first state s0 and the second state s1.

[0129] For example, Figure 7A In case ③, the first coordinate value is greater than or equal to the second coordinate value but less than the third coordinate value, and the sum of the first coordinate value and the size of the tensor to be processed in the first dimension is less than the third coordinate value. The state machine always loops and jumps in the second state s1.

[0130] For example, Figure 7A In case ④, the first coordinate value is less than the second coordinate value, and the sum of the first coordinate value and the size of the tensor to be processed in the first dimension is greater than or equal to the third coordinate value. The state machine loops and jumps among the first state s0, the second state s1, and the third state s2.

[0131] For example, Figure 7A In case ⑤, the first coordinate value is greater than or equal to the second coordinate value, and the sum of the first coordinate value and the size of the tensor to be processed in the first dimension is greater than or equal to the third coordinate value. The state machine loops and jumps between the second state s1 and the third state s2.

[0132] For example, Figure 7A In case ⑥, the first coordinate value is greater than or equal to the third coordinate value. The state machine always loops and jumps in the third state s2.

[0133] After determining the initial state, based on the initial state, determine the request initial coordinates corresponding to the second sent request, and combine with the trigger condition to determine the state entered by the second sent request. Then, based on the state entered by the second sent request, determine the request initial coordinates corresponding to the third sent request and the data length loaded by the second sent request, and combine with the trigger condition to determine the state entered by the third sent request, and so on.

[0134] For example, the state machine outputs the request initial coordinates corresponding to the current request and the data length loaded by the current request in each state. In addition, the state machine also prepares the request initial coordinates corresponding to the next request for the next state. After determining the request initial coordinates corresponding to the current request, it can be judged whether the data loaded by the current request is in the memory according to the request initial coordinates.

[0135] For example, in response to the coordinate value of the request initial coordinate corresponding to the current request in the first dimension being greater than or equal to the second coordinate value and less than the third coordinate value, it is determined that all the data to be loaded by the current request is located in the memory. It can be understood that for the data range that all belongs to the first original tensor, it will all be within the memory range (this part of the data is actually stored in the memory), and for the data range that all does not belong to the first original tensor, it will all not be within the memory range and is not the actually stored data. That is, the data exceeding the range of the first original tensor can be understood as invalid data. For example, this part of the invalid data can be directly filled with zeros in the buffer. At this time, the data read address and data write address of the current request can be determined with reference to the above formula, and then the current request is sent to the memory to load the corresponding data into the buffer.

[0136] For example, in response to the coordinate value of the request initial coordinate corresponding to the current request in the first dimension being less than the second coordinate value or greater than or equal to the third coordinate value, it is determined that all the data to be loaded by the current request is not located in the memory. At this time, in response to all the data to be loaded by the current request not being located in the memory, the current request is converted to, for example, using hardware to write multiple predetermined values into the buffer, where the number of the multiple predetermined values is determined by the length of the loaded data specified by the current request. For example, the predetermined value is 0.

[0137] The process of using the state machine to determine the request initial coordinate and the length of the loaded data corresponding to each request will be specifically described below.

[0138] Figure 7B Shows the state transition for the NDHWC data storage format according to the PerW partitioning method. The condition for the PerW partitioning method is that c_coord is not equal to 0, or copy_c is not equal to tensor_c. At this time, the two adjacent pixels to be obtained are not continuous in the memory, and the requests need to be split pixel by pixel to obtain the data.

[0139] At Figure 7B On the left side in Figure 7B only two states, s1 and s2, are shown for the C dimension. This is considering that in actual applications, generally the coordinate of the C dimension is not less than 0, that is, there is no first state. Specifically, s1 can correspond to the above-mentioned second state, and s2 can correspond to the above-mentioned third state. In addition, in the state loop schematic diagram on the right side of

[0140] Table 1

[0141]

[0142] That is to say, atFigure 7B In the example, the blank box on the right corresponds to the size of the first original tensor in the C dimension (tensor_c), and its left boundary can correspond to, for example, Figure 7A the second coordinate value in, for example, equal to 0, and its right boundary can correspond to, for example, Figure 7B the third coordinate value in, for example, equal to tensor_c, corresponding to state s1 when the size copy_c of the first tensor to be processed in the C dimension is between 0 and tensor_c, and corresponding to state s2 when copy_c is greater than tensor_c.

[0143] Figure 7B Jumps can be made between the various states shown in. The following Table 2 shows the conditions for jumping between the various states:

[0144] Table 2

[0145]

[0146] In the above Table 2, the parameter remain_copy_p represents the number of remaining pixels not yet fetched. For example, before the first request, remain_copy_p is equal to the number of pixels copy_pixel_num corresponding to the first tensor to be processed, and after the first request, remain_copy_p is equal to copy_pixel_num - req_size_1. remain_copy_c represents the size of the remaining data not yet fetched in the C dimension, used to characterize the data fetching information in the C dimension. For example, before the first request, remain_copy_c = copy_c.

[0147] The following describes the information that needs to be updated in each state and how to update it.

[0148] First, in state s1, to prepare for the next state, that is, as the initial state of the next state, the state machine can update the parameters according to the following table:

[0149] Table 3

[0150]

[0151] It should be noted that in the example of the above Table 3, the situation of setting boundary values and data fetching step sizes for the first original tensor is further considered. Among them, right_bound represents the right boundary value set for the W dimension, down_bound represents the boundary value in the H dimension, and hind_bound represents the boundary value in the D dimension. The setting of boundary values and step sizes can be referred to in the following in combination with Figure 8A - Figure 8CDescription. In addition, for the case where the boundary values are not set, it can be understood that each boundary is equal to the size of the first original tensor in that dimension. In Table 3, stride_w, stride_h, and stride_d respectively represent the stride values in the W dimension, H dimension, and D dimension. As an implementation, the strides can all be set to 1. In addition, it can be understood that in other embodiments according to the present disclosure, boundary values and strides can also be set for the C dimension, for example, and there is no limitation here.

[0152] The parameter update rules in Table 3 are described below. First, for the C dimension with the lowest priority, the parameter c_coord represents the coordinate of the current request in the C dimension. When c_coord + remain_copy_c <= tensor_c, that is, the next request for the C dimension does not exceed the boundary tensor_c of the first original tensor in the C dimension, the parameter c_coord is updated to the starting coordinate c_coord_b of the C dimension; when c_coord + remain_copy_c > tensor_c, that is, the data acquisition for the next request in the C dimension exceeds the boundary tensor_c of the first original tensor in the C dimension, the parameter c_coord is updated to tensor_c.

[0153] The coordinate update logic for the other dimensions in Table 3 is the same as that of the W dimension, which is described separately as follows. For the W dimension, the parameter w_coord represents the coordinate of the current request in the W dimension. For the case of jumping from state s1 to s1, that is, when the data to be acquired by the next request is still valid data, when w_coord + stride_w >= right_bound, that is, when the W dimension exceeds the boundary, the coordinate w_coord of the W dimension is updated to w_coord + stride_w - copy_w, where copy_w represents the size of the tensor to be processed in the W dimension; otherwise (w_coord + stride_w < right_bound), that is, when it does not exceed the boundary, the coordinate w_coord of the W dimension is updated to w_coord + stride_w. When stride_w = 1, it means that w_coord is incremented by 1, that is, the data to be acquired by the next request is the next data immediately following the data to be acquired by the current request. Otherwise (for cases other than jumping from state s1 to s1, that is, jumping from state s1 to state s2 or idle), the coordinate w_coord of the W dimension remains unchanged.

[0154] For the H dimension, the parameter h_coord represents the coordinate of the current request in the H dimension. For the case of jumping from state s1 to s1, that is, the case where the data to be fetched by the next request is still valid data, at this time, when w_coord + stride_w >= right_bound and h_coord + stride_h >= down_bound, when h_coord + stride_h >= down_bound, that is, when the H dimension exceeds the boundary, the coordinate h_coord of the H dimension is updated to h_coord + stride_h - copy_h, where copy_h represents the size of the tensor to be processed in the H dimension; otherwise (h_coord + stride_h < down_bound), that is, when it does not exceed the boundary, the coordinate h_coord of the H dimension is updated to h_coord + stride_h. Otherwise (for cases other than jumping from state s1 to s1, that is, jumping from state s1 to state s2 or idle), the coordinate h_coord of the H dimension remains unchanged.

[0155] For the D dimension, the parameter d_coord represents the coordinate of the current request in the D dimension. For the case of jumping from state s1 to s1, that is, the case where the data to be fetched by the next request is still valid data, at this time, when w_coord + stride_w >= right_bound and h_coord + stride_h >= down_bound, when d_coord + stride_d >= hind_bound, that is, when the D dimension exceeds the boundary, the coordinate d_coord of the D dimension is updated to d_coord + stride_d - copy_d, where copy_d represents the size of the tensor to be processed in the D dimension; otherwise (d_coord + stride_d < hind_bound), that is, when it does not exceed the boundary, the coordinate d_coord of the D dimension is updated to d_coord + stride_d. Otherwise (for cases other than jumping from state s1 to s1, that is, jumping from state s1 to state s2 or idle), the coordinate d_coord of the D dimension remains unchanged.

[0156] For the N dimension, the parameter n_coord represents the coordinate of the current request in the N dimension. For the case of jumping from state s1 to s1, that is, the case where the data to be retrieved by the next request is still valid data, at this time w_coord + stride_w >= right_bound and h_coord + stride_h >= down_bound and d_coord + stride_d >= hind_bound, the coordinate n_coord of the N dimension is incremented by 1, that is, updated to n_coord + 1; otherwise (for cases other than jumping from state s1 to s1, that is, jumping from state s1 to state s2 or idle), the coordinate n_coord of the N dimension remains unchanged. It can be seen that the update logic of the N dimension is slightly different from that of the W, H, and D dimensions, because generally, no boundary values and step sizes are set for the N dimension.

[0157] In addition, in addition to the above dimension coordinate parameters, the state machine also needs to maintain two parameters remain_copy_p and remain_copy_c. Among them, remain_copy_p represents the number of remaining pixels that have not been retrieved. For example, before the first request, remain_copy_p is equal to the number of pixels copy_pixel_num corresponding to the tensor to be processed. After the first request, remain_copy_p is equal to copy_pixel_num - req_size_1; remain_copy_c represents the size of the remaining data to be retrieved in the C dimension. For example, before the first request, remain_copy_c = copy_c. After the first request, remain_copy_c is equal to copy_c minus the size corresponding to the C dimension retrieved by the first request.

[0158] Specifically, as shown in Table 3, for remain_copy_c, when c_coord + remain_copy_c <= tensor_c, remain_copy_c is updated to copy_c; otherwise (c_coord + remain_copy_c > tensor_c), remain_copy_c is updated to c_coord + remain_copy_c - tensor_c.

[0159] For remain_copy_p, when c_coord + remain_copy_c <= tensor_c, remain_copy_p is updated to remain_copy_p - 1; otherwise (c_coord + remain_copy_c > tensor_c), remain_copy_p remains unchanged.

[0160] Second, in the s2 state, to prepare for the next state, i.e., as the initial state of the next state, the state machine can update the parameters according to the following table:

[0161] Table 4

[0162]

[0163] For Table 4, the state updates of each parameter can refer to the description for Table 3 and will not be elaborated here. Based on the above update logic, the update results of Table 4 can be derived similarly.

[0164] The state transitions for the NDHWC data storage format according to the PerW partitioning method are described above in combination with Tables 1 - 4, which involve states s1 and s2, and the update outputs of the state machine in these two states. It can be understood that the principles and update logics of the state machine for other partitioning methods (e.g., PerH) of the NDHWC data storage format are similar to those described above and will not be described here.

[0165] Of course, as the coordinates continue to increase, when the number of pixels obtained is equal to the number of pixels included in the first tensor to be processed, the state machine can enter the idle state, indicating that the data for the current pen instruction has been obtained. Taking the NDHWC data storage format as an example, for the PerW request partitioning method, the idle state is entered when the remaining values of the N, D, H, and W dimensions are equal to 1 and the data in the C dimension has been obtained; for the PerH request partitioning method, the idle state is entered when the remaining values of the N, D, and H dimensions are equal to 1 and the data in the W dimension has been obtained.

[0166] The specific state transition conditions of the state machine and the updated coordinate content can be changed and set according to actual needs, and the logic is similar to the state machine logic described above, and no further examples will be given here.

[0167] It should be noted that the process of using the state machine to determine the initial coordinates of the request and the length of the loaded data is basically the same for NDHWC and N(C / x)DHW(xC). The difference is that in response to the data storage format of the first tensor to be processed being NDHWC, the coordinate value of the channel number dimension is incremented by 1 when updated, and in response to the data storage format of the first tensor to be processed being N(C / x)DHW(xC), the coordinate value of the channel number dimension is incremented by x when updated.

[0168] Based on the above description of the state machine that can enter the idle state, the last request among multiple requests can be determined according to whether the state machine is about to jump to the idle state.

[0169] In some embodiments of the present disclosure, in response to the request currently processed by the state machine satisfying a preset condition and jumping to the idle state, it is determined that the request currently processed by the state machine is the last request among multiple requests, where the idle state represents the state after the first tensor to be processed is loaded.

[0170] For example, in the case where the data storage format is NDHWC and the request partitioning method of PerW is adopted, the above preset conditions include: (1) the current state of the state machine is state s2, and the remain_copy_p (i.e., the remaining number of pixels not yet acquired) corresponding to the request Q currently processed by the state machine is 1; (2) the current state of the state machine is state s1, the remain_copy_p corresponding to the request Q currently processed by the state machine is 1, and the sum of the coordinate value c_coord of the request initial coordinates corresponding to request Q in the C dimension and the remain_copy_c (i.e., the remaining number of data not yet acquired) corresponding to request Q is less than the size tensor_c of the first original tensor in the C dimension. When the state machine enters the idle state, it indicates that the data of the current pen command (e.g., the first tensor to be processed) has been acquired. Therefore, when the request currently processed by the state machine satisfies the preset condition (1), it indicates that the state machine will jump from state s2 to the idle state after processing this request, and this request is the last request among the multiple requests determined by the request partitioning method of PerW. When the request currently processed by the state machine satisfies the preset condition (2), it indicates that the state machine will jump from state s1 to the idle state after processing this request, and this request is the last request among the multiple requests determined by the request partitioning method of PerW.

[0171] In some embodiments of the present disclosure, the last request may indicate the number of pixels to be acquired in the last request and the request initial coordinates corresponding to the last request. For example, based on the number of pixels indicated in the last request, the request initial coordinates, and the adopted request partitioning method, the second starting coordinates of the second tensor to be processed in the first coordinate system can be determined.

[0172] For example, in response to partitioning requests in the height dimension (i.e., adopting the PerH request partitioning method), according to the number of pixels to be acquired in the last request, the request initial coordinates corresponding to the last request are updated to obtain the second starting coordinates. For example, the coordinate values of the request initial coordinates corresponding to the last request in five dimensions are represented as w_cood, h_cood, d_cood, n_cood, and c_cood, and the coordinate values of the second starting coordinates in five dimensions are represented as w_cood_f, h_cood_f, d_cood_f, n_cood_f, and c_cood_f. The process of determining the second starting coordinates specifically includes: determining the sum of the coordinate value w_cood of the request initial coordinates in the width dimension and the number of pixels to be acquired in the last request (i.e., the remaining number of unacquired pixels remain_copy_p) as the coordinate value w_cood_f of the second starting coordinates in the width dimension, that is, w_cood_f = w_cood + remain_copy_p; and determining the coordinate values of the request initial coordinates in the remaining dimensions other than the width dimension as the coordinate values of the second starting coordinates in the remaining dimensions other than the width dimension, that is, h_cood_f = h_cood, d_cood_f = d_cood, n_cood_f = n_cood, and c_cood_f = c_cood.

[0173] The data loading method provided by some embodiments of the present disclosure may further include: obtaining boundary values for at least a part of the five dimensions of the first original tensor respectively; and obtaining step values set for obtaining data in at least a part of the five dimensions, where the step value is a positive integer.

[0174] In some embodiments of the present disclosure, the boundary value is used to define the boundary range for acquiring data from the first original tensor. The boundary value can be any value compared to the size value of the first original tensor in this dimension, that is, the boundary value can exceed the range of the first original tensor itself. For the dimension with the boundary value set, the data loaded by each request belongs to the effective data range defined by the first original tensor and the boundary value or does not belong to the effective data range. For the data that does not belong to the effective data range (also referred to as "invalid data"), the buffer directly fills the tensor to be processed with a predetermined value, where the predetermined value is equal to 0.

[0175] For example, Figure 8A FIG. shows a schematic diagram of setting boundary values for a first original tensor according to an embodiment of the present disclosure.

[0176] As shown in Figure 8A FIG., the rectangular box represents the range covered by the first original tensor, which can be any dimension in the first original tensor, such as the C dimension, the W dimension, the H dimension, or the D dimension. Generally, boundary values are not set for the N dimension. In the six sub - figures of Figure 8A , the left boundary value and the right boundary value (shown as "L" and "R" in Figure 8A ) are respectively shown in relation to the size values of the first original tensor in this dimension.

[0177] For example, in the first sub - figure of Figure 8A , both the left boundary value L and the right boundary value R are on the left side of the first original tensor, that is, the data to be obtained in this dimension are all invalid data. In this case, for invalid data, for example, this part of the data can be automatically filled by sending a zero - filling instruction to the buffer area. Also for example, in the second sub - figure of Figure 8A , the left boundary value L is on the left side of the left boundary of the data range of the first original tensor, and the right boundary value R is on the left side of the right boundary of the first original tensor, that is, for the tensor to be processed to be obtained, part is invalid data and part is valid data in the first original tensor. Schematically, in Figure 8A , the data corresponding to the slanted shaded part is represented as valid data, and the rest are all invalid data. In this case, for valid data, for example, it is carried out according to the order described in Figure 6 , and for invalid data, for example, this part of the data can be automatically filled by sending a zero - filling instruction to the buffer area.

[0178] In some embodiments of the present disclosure, by setting boundary values for the first original tensor, the range of the tensor to be processed (for example, the first tensor to be processed) to be obtained from the first original tensor can be further delimited. In practical applications, this implementation method can adapt to the characteristics of operations such as convolution operations. For example, it is beneficial to reduce the amount of calculation and greatly improve the flexibility of data. As an example, assume that the first original tensor corresponds to an intermediate tensor for feature extraction of an entire input picture, and the input picture includes a specific target, such as an object to be recognized, and the object does not cover the entire picture, that is, the picture includes a background part. In this case, by setting boundary values, the range of the tensor to be processed to be obtained can be limited to the part corresponding to the specific target in the first original tensor, so as to reduce the amount of calculation of subsequent operations such as convolution operations and improve the processing efficiency.

[0179] As an example, Figure 8BThe figure shows a schematic diagram of continuous data acquisition when boundaries are set for the first original tensor. Among them, compared with Figure 4 the situation shown in Figure 8B it can be understood that boundary values (bound) are respectively set for the C dimension, W dimension, and H dimension of tensor A in Figure 4 , and the boundaries are all within the size ranges of tensor A in the C dimension, W dimension, and H dimension, which is equivalent to Figure 8A the boundary situation shown in the fourth sub-picture in Figure 8B . Specifically, Figure 4 the outer box in Figure 8B corresponds to tensor A as a whole (corresponding to tensor A in Figure 8B ). After setting the boundaries, according to the data loading method of the embodiments of the present disclosure, data will be sequentially acquired starting from the starting point coordinates within the set boundary range until the number of acquired data is equal to the number of pixels included in the tensor to be processed. In Figure 4 , the data part composed of squares corresponds to the data range framed by the boundary values set for the C dimension, W dimension, and H dimension, and data is sequentially acquired from the data range framed by the boundary. Data located outside the data range framed by the boundary can be regarded as invalid data. Regarding

[0180] the process of sequentially acquiring data in

[0181] Figure 8C reference may be made to the description in combination with Figure 6 , and details will not be elaborated here.

[0182] In some embodiments of the present disclosure, the set step value is used to specify the step (Stride) for sequentially acquiring data from the first original tensor. For example, for the data of the first original tensor, starting from the first starting coordinate of the first tensor to be processed, data is sequentially acquired from the first original tensor according to the step value.

[0181] Figure 8C The figure shows a schematic diagram of continuously acquiring the first tensor to be processed according to the set step value. Among them, the step is equal to 2, that is, one data is taken every other pixel point, that is, only the pixels shown in the shaded part are sequentially acquired. The step being equal to 2 means skipping one pixel. Similarly, when the step is equal to 3, it means skipping two pixels, and so on. It can be understood that when the set step is equal to 1, it corresponds to Figure 6 the data loading method of each pixel point (PerW) shown. In practical applications, by setting the step, the computational complexity of subsequent operations such as convolution operations on the data can be further reduced, the processing efficiency can be improved, and in addition, the flexibility of data loading can be further enhanced.

[0182] For example, in response to a request division in the width dimension (i.e., adopting the PerW request division method), according to the set boundary value, step value, and the request initial coordinates corresponding to the last request, the second starting coordinates are obtained by updating the request initial coordinates. Table 5 shows the update rule of the request initial coordinates in the case of adopting the PerW request division method.

[0183] Table 5

[0184]

[0185] It should be noted that in the example of Table 5, the case of setting the boundary value for the first original tensor and the step value for data acquisition is further considered. Among them, right_bound represents the right boundary value set for the W dimension, and down_bound represents the lower boundary value in the H dimension. In addition, for the case where no boundary value is set, it can be understood that each boundary is equal to the size of the first original tensor in that dimension. In Table 5, the coordinate values of the request initial coordinates corresponding to the last request in the five dimensions are represented as w_cood, h_cood, d_cood, n_cood, and c_cood, and the coordinate values of the second starting coordinates in the five dimensions are represented as w_cood_f, h_cood_f, d_cood_f, n_cood_f, and c_cood_f. In Table 5, stride_w, stride_h, and stride_d respectively represent the step values in the W dimension, H dimension, and D dimension. As an implementation method, the step can be set to 1 uniformly. In addition, it can be understood that in other embodiments according to the present disclosure, the boundary value and step can also be set for the C dimension, for example, and no limitation is made here.

[0186] Next, the rule for updating the request initial coordinates in Table 5 to obtain the second initial coordinates will be described. First, for the C dimension with the lowest priority, the coordinate value c_coord_f of the second initial coordinates in the C dimension is equal to the coordinate value c_coord of the request initial coordinates corresponding to the last request in the C dimension.

[0187] For the W dimension, when w_coord + stride_w < right_bound, that is, the W dimension does not exceed the right boundary right_bound of the first original tensor in the W dimension, the coordinate value of the requested initial coordinate in the W dimension is updated to w_coord + stride_w, so as to obtain the coordinate value w_coord_f = w_coord + stride_w of the second initial coordinate in the W dimension; correspondingly, when w_coord + stride_w < right_bound, the coordinate values of the requested initial coordinate in the H dimension, D dimension, and N dimension remain unchanged, so the coordinate values of the second initial coordinate in the H dimension, D dimension, and N dimension are h_coord_f = h_coord, d_coord_f = d_coord, and n_coord_f = n_coord. When w_coord + stride_w >= right_bound, the W dimension has exceeded the right boundary right_bound of the first original tensor in the W dimension, the coordinate value of the requested initial coordinate in the W dimension is updated to w_coord + stride_w - copy_w, so as to obtain the coordinate value w_coord_f = w_coord + stride_w - copy_w of the second initial coordinate in the W dimension, where copy_w represents the size of the first tensor to be processed in the W dimension.

[0188] For the H dimension, when w_coord + stride_w >= right_bound, if h_coord + stride_h >= down_bound, the H dimension exceeds the lower boundary of the first original tensor in the H dimension, and the coordinate value of the requested initial coordinate in the H dimension is updated to h_coord + stride_h - copy_h, so as to obtain the coordinate value h_coord_f = h_coord + stride_h - copy_h of the second initial coordinate in the H dimension, where copy_h represents the size of the first tensor to be processed in the H dimension. When w_coord + stride_w >= right_bound, if h_coord + stride_h < down_bound, the H dimension does not exceed the boundary down_bound of the H dimension, and the coordinate value h_coord is updated to h_coord + stride_h. Otherwise (for the case of w_coord + stride_w < right_bound), the coordinate value h_coord_f of the second initial coordinate in the H dimension is h_coord.

[0189] For dimension D, when w_coord + stride_w >= right_bound and h_coord + stride_h >= down_bound, if the size tensor_d of the first original tensor in dimension D is 1, then the coordinate value d_coord_f of the second initial coordinate in dimension D is 0; otherwise (the size tensor_d of the first original tensor in dimension D is not 1), the coordinate value d_coord_f of the second initial coordinate in dimension D is d_coord + 1. When w_coord + stride_w < right_bound or h_coord + stride_h < down_bound, the coordinate value d_coord_f of the second initial coordinate in dimension D is d_coord.

[0190] For dimension N, when w_coord + stride_w >= right_bound, h_coord + stride_h >= down_bound, and tensor_d = 1, the coordinate value n_coord_f of the second initial coordinate in dimension N is n_coord + 1; otherwise, the coordinate value n_coord_f of the second initial coordinate in dimension N is n_coord.

[0191] In some embodiments of the present disclosure, the first starting coordinate and the first number of pixels of the first tensor to be processed are used to determine, by a state machine, the last request among multiple requests for loading the first tensor to be processed, and based on the adopted request partitioning method, the request initial coordinate indicated in the last request, and other parameters (such as the boundary values of the first original tensor, the size of the first tensor to be processed, etc.), the second starting coordinate of the second tensor to be processed adjacent to the first tensor to be processed can be directly determined, thereby omitting the step of resetting the second starting coordinate in the data loading instruction corresponding to the second tensor to be processed, and further simplifying the instruction structure (or instruction logic) of the data loading instruction, saving instruction resources, and improving the instruction execution efficiency.

[0192] In some embodiments of the present disclosure, the first tensor to be processed and the second tensor to be processed are adjacent in the coordinate system of the original tensor, and before loading the second tensor to be processed, a padding operation is performed on the second tensor to be processed, resulting in an expansion of the range of the second tensor to be processed in the first original tensor, such that at least part of the data of the first tensor to be processed and the second tensor to be processed overlaps, that is, the second data loading instruction corresponding to the second tensor to be processed after the padding operation uses part of the data in the first data loading instruction corresponding to the first tensor to be processed.

[0193] For example, Figure 9A schematic diagram of obtaining a second tensor to be processed for NDHWC data storage format according to an embodiment of the present disclosure is shown.

[0194] In Figure 9 the example shown, the data storage format of the first original tensor is NDHWC, the first pixel number copy_pixel_num of the first tensor to be processed is 40, and the first starting coordinates are w_coord_b = 2, h_coord_b = 2, d_coord_b = 0, n_coord_b = 0, c_coord_b = 0 respectively.

[0195] As Figure 9 shown, the solid arrows indicate that the first tensor to be processed is sequentially (or continuously) loaded into the buffer in units of pixels, where the gray squares represent the pixels included in the first tensor to be processed. Since padding is performed on the second tensor to be processed, the range of the second tensor to be processed in the first original tensor expands, and there are 2 overlapping pixels (i.e., 2 squares) between the first tensor to be processed and the second tensor to be processed, so that the actual second starting coordinates of the second tensor to be processed need to be retreated by 2 coordinate units based on the end coordinates (w_coord_f = 2, h_coord_f = 0, d_coord_f = d1, n_coord_f = n1, c_coord_f = c_coord) of the first tensor to be processed (as shown by the dashed arrows in Figure 9 , retreating 2 squares), so as to obtain the actual second starting coordinates (w_coord_f = 0, h_coord_f = 0, d_coord_f = d1, n_coord_f = n1, c_coord_f = c_coord) of the second tensor to be processed, and use the pixel point corresponding to the actual second starting coordinates as the starting point, and sequentially (or continuously) load the second tensor to be processed into the buffer in units of pixels.

[0196] In this regard, some embodiments of the present disclosure also provide a data loading method, which can directly determine the second starting coordinates of the second tensor to be processed through a hardware device based on the first starting coordinates of the first tensor to be processed indicated in the first data loading instruction, the first pixel number included in the first tensor to be processed, and the number of pixels to be retreated for the second tensor to be processed, omitting the steps of resetting the second starting coordinates before retreat and the number of pixels to be retreated in the second data loading instruction and the retreat operation when loading the second tensor to be processed, thereby simplifying the instruction structure (or instruction logic) of the data loading instruction, saving instruction resources, and improving the instruction execution efficiency.

[0197] Figure 10The figure shows a schematic flowchart of another data loading method provided by at least one embodiment of the present disclosure. As Figure 10 shown, the data loading method at least includes steps S210 to S230.

[0198] In step S210: Receive a first data loading instruction to obtain a first starting coordinate of a first tensor to be processed in a first coordinate system determined by a first original tensor in a memory, a first number of pixels included in the first tensor to be processed, and a number of fallback pixels for a second tensor to be processed.

[0199] In step S220: Determine an end coordinate of the last pixel in the first tensor to be processed in the first coordinate system according to the first starting coordinate and the first number of pixels.

[0200] In step S230: Determine a second starting coordinate of the second tensor to be processed corresponding to the second data loading instruction in the first coordinate system according to the end coordinate and the number of fallback pixels.

[0201] For example, for step S210, the first data loading instruction instructs to load the first tensor to be processed from the first original tensor into the buffer area. In the first data loading instruction, the first starting coordinate, the first number of pixels of the first tensor to be processed, and the number of fallback pixels for the second tensor to be processed can be indicated. The first starting coordinate and the first number of pixels of the first tensor to be processed are used to determine the position of the first tensor to be processed in the memory. The number of fallback pixels (i.e., the number of overlapping pixels between the first tensor to be processed and the second tensor to be processed) is the number of coordinates that need to be fallback in the first coordinate system when loading the second tensor to be processed. In addition, for the calculation of the first coordinate system and the number of pixels determined by the first original tensor, reference can be made to the description of step S110 above, which will not be elaborated here.

[0202] For example, the data loading instruction can be a machine instruction, or the data loading instruction can also be a micro-instruction. For example, the data loading instruction is implemented in the form of a Load instruction.

[0203] For example, a state machine can be used to determine the end coordinate corresponding to the first tensor to be processed. For example, step S220 may include steps S221 to S223.

[0204] In step S221: Determine a plurality of requests for loading the first tensor to be processed according to the first starting coordinate and the first number of pixels, where the plurality of requests are used to sequentially obtain data from the first original tensor starting from the first starting coordinate until the number of obtained data is equal to the first number of pixels.

[0205] In step S222: Determine the last request among the plurality of requests through the state machine.

[0206] In step S223: Determine the end coordinate of the last pixel in the first processing tensor in the first coordinate system according to the request initial coordinate corresponding to the last request and the number of pixels to be acquired by the last request.

[0207] For example, for step S222, in response to the request currently processed by the state machine satisfying a preset condition and jumping to the idle state (i.e., the idle state), it is determined that the request currently processed by the state machine is the last request among multiple requests. The preset condition and the determination process of the last request can refer to the relevant descriptions above and will not be elaborated here. The descriptions of steps S221 to S223 can refer to the relevant descriptions of steps S121 and S122 above and will not be elaborated here.

[0208] For example, for step S230, after determining the end coordinate of the first processing tensor, the end coordinate can be regressed by regr_pixel_num pixels (or coordinate units) in the first coordinate system of the first original tensor according to the number of regressed pixels (i.e., regr_pixel_num), so as to determine the second start coordinate of the second processing tensor corresponding to the second data loading instruction in the first coordinate system, where the second data loading instruction instructs to load the second processing tensor from the first original tensor into the buffer area.

[0209] In some embodiments of the present disclosure, before specifically determining the second start coordinate of the second processing tensor, boundary values for at least a part of the five dimensions of the first original tensor can be obtained first, where the boundary values are used to define the boundary range for obtaining data from the first original tensor. For the dimensions with boundary values set, the data loaded by each request belongs to the valid data range defined by the first original tensor and the boundary values or does not belong to the valid data range; then, step values for obtaining data set on at least a part of the five dimensions are obtained, where the step value is a positive integer.

[0210] The specific descriptions of the "boundary value" and the "step value" here can refer to the above and in combination with Figure 8A - 8C and will not be elaborated here.

[0211] It should be noted that in the embodiments of the present disclosure, when the step value is not set or the default step value is set to 1, the number of regressed pixels (e.g., regr_pixel_num) is less than or equal to the width size of the first processing tensor in the width dimension. When the step value is set, the product of the number of regressed pixels and the step value in the width dimension (e.g., regr_pixel_num×stride_w) is less than or equal to the width size of the first processing tensor in the width dimension.

[0212] For example, the above step S230 may specifically include: performing a backward operation on the end coordinates according to the number of backward pixels and the stride; and updating the end coordinates to obtain the second start coordinates according to whether the backward operation crosses the boundary corresponding to the boundary value of at least a part of the five dimensions of the first original tensor.

[0213] For example, in the case where the stride value is set (or the stride value is not 1), the backward operation means moving the end coordinates backward by regr_pixel_num × stride_w pixels.

[0214] The process of determining the second start coordinates based on whether the backward operation crosses the boundary value of the first original tensor is specifically introduced below. Among them, the sizes of the first tensor to be processed in the width dimension, height dimension, and depth dimension are respectively represented as copy_w, copy_h, and copy_d, the strides set in the width dimension, height dimension, and depth dimension are respectively represented as stride_w, stride_h, and stride_d, the coordinate values of the end coordinates in the five dimensions are respectively represented as w_cood_f, h_cood_f, d_cood_f, n_cood_f, and c_cood_f, and the coordinate values of the second start coordinates in the five dimensions are respectively represented as w_cood_upd, h_cood_upd, d_cood_upd, n_cood_upd, and c_cood_upd. The left boundary value set by the first original tensor in the W dimension is represented as left_bound, the upper boundary value set in the H dimension is represented as up_bound, and the front boundary value set in the D dimension is represented as front_bound.

[0215] For example, in response to the backward operation not crossing the boundary of the first original tensor in the width dimension (i.e., w_coord_f - regr_pixel_num × stride_w < left_bound), the coordinate value of the end coordinates in the width dimension is updated to w_cood_f - regr_pixel_num × stride_w, and the updated end coordinates are used as the second start coordinates. At this time, the coordinate values of the second start coordinates in the five dimensions are:

[0216] w_cood_upd = w_cood_f - regr_pixel_num × stride_w;

[0217] h_cood_upd = h_cood_f;

[0218] d_cood_upd = d_cood_f;

[0219] n_cood_upd = n_cood_f; and

[0220] c_cood_upd = c_cood_f.

[0221] For example, in response to the backward operation crossing the boundary of the first original tensor in the width dimension (i.e., w_coord_f - regr_pixel_num × stride_w >= left_bound) and not crossing the boundary of the first original tensor in the height dimension (i.e., h_coord_f - stride_h < up_bound), update the coordinate value of the end point in the width dimension to w_cood_f + copy_w - regr_pixel_num × stride_w, update the coordinate value of the end point in the height dimension to h_cood_f - stride_h, and use the updated end point coordinates as the second starting coordinates. At this time, the coordinate values of the second starting coordinates in the 5 dimensions are as follows:

[0222] w_cood_upd = w_cood_f + copy_w - regr_pixel_num × stride_w;

[0223] h_cood_upd = h_cood_f - stride_h;

[0224] d_cood_upd = d_cood_f;

[0225] n_cood_upd = n_cood_f; and

[0226] c_cood_upd = c_cood_f.

[0227] For example, in response to a backward operation crossing the boundary of the first original tensor in the height dimension (i.e., w_coord_f - regr_pixel_num × stride_w >= left_bound, and h_coord_f - stride_h >= up_bound), and not crossing the boundary of the first original tensor in the depth dimension (i.e., d_coord_f - stride_d < front_bound), update the coordinate value of the end point in the width dimension to w_cood_f + copy_w - regr_pixel_num × stride_w, update the coordinate value of the end point in the height dimension to h_cood_f + copy_h - stride_h, update the coordinate value of the end point in the depth dimension to d_cood_f - stride_d, and use the updated end point coordinates as the second starting coordinates. At this time, the coordinate values of the second starting coordinates in the 5 dimensions are as follows:

[0228] w_cood_upd = w_cood_f + copy_w - regr_pixel_num × stride_w;

[0229] h_cood_upd = h_cood_f + copy_h - stride_h;

[0230] d_cood_upd = d_cood_f - stride_d;

[0231] n_cood_upd = n_cood_f; and

[0232] c_cood_upd = c_cood_f.

[0233] For example, in response to a backward operation crossing the boundary of the first original tensor in the depth dimension (i.e., w_coord_f - regr_pixel_num × stride_w >= left_bound, h_coord_f - stride_h >= up_bound, and d_coord_f - stride_d >= front_bound), update the coordinate value of the end point in the width dimension to w_cood_f + copy_w - regr_pixel_num × stride_w, update the coordinate value of the end point in the height dimension to h_cood_f + copy_h - stride_h, update the coordinate value of the end point in the depth dimension to d_cood_f + copy_d - stride_d, update the coordinate value of the end point in the batch dimension to n_cood_f minus 1, and use the updated end point coordinates as the second starting coordinates.

[0234] At this time, the coordinate values of the second starting coordinate in the five dimensions are as follows:

[0235] w_cood_upd = w_cood_f + copy_w - regr_pixel_num × stride_w;

[0236] h_cood_upd = h_cood_f + copy_h - stride_h;

[0237] d_cood_upd = d_cood_f + copy_d - stride_d;

[0238] n_cood_upd = n_cood_f - 1; and

[0239] c_cood_upd = c_cood_f.

[0240] At least one embodiment of the present disclosure further provides a data storage method for writing a to-be-processed tensor in a buffer into a memory.

[0241] For example, ​ FIG. shows a schematic flowchart of a data storage method provided by at least one embodiment of the present disclosure.

[0242] As ​ shown, the data storage method provided by at least one embodiment of the present disclosure at least includes steps S310 - S320.

[0243] In step S310: Obtain the third starting coordinate of the third to-be-processed tensor in the second coordinate system determined by the second original tensor and the third number of pixels included in the third to-be-processed tensor.

[0244] In step S320: According to the third starting coordinate and the third number of pixels, directly determine the fourth starting coordinate of the fourth to-be-processed tensor in the second coordinate system by a hardware device, where the position of the last pixel included in the third to-be-processed tensor is adjacent to the fourth starting coordinate of the fourth to-be-processed tensor in the second coordinate system.

[0245] The above-mentioned second original tensor, third to-be-processed tensor, and fourth to-be-processed tensor are original tensors in the buffer. The relevant descriptions of steps S310 and S320 can refer to the relevant content of steps S110 and S120 in the foregoing data loading method, which will not be elaborated here.

[0246] For example, the data storage format of the second original tensor is the same as that of the tensor to be processed (e.g., the third tensor to be processed or the fourth tensor to be processed), such as NDHWC, or N(C / x)DWH(Cx), where x is a positive integer greater than 1.

[0247] For the data storage method provided by the embodiments of the present disclosure, the fourth starting coordinate of the fourth tensor to be processed in the buffer (i.e., the next tensor to be processed of the third tensor to be processed in the buffer) can be directly determined by the hardware device according to the third starting coordinate and the third number of pixels of the third tensor to be processed in the buffer, without resetting the fourth starting coordinate in the second data storage instruction corresponding to the fourth tensor to be processed. That is, the subsequent data storage instructions do not need to reset the starting coordinates of the corresponding tensors to be processed, thereby simplifying the instruction structure (or instruction logic) of the data storage instructions, saving instruction resources, and improving the instruction execution efficiency.

[0248] At least one embodiment of the present disclosure also provides another data storage method for writing the tensor to be processed in the buffer into the memory.

[0249] For example, ​ FIG. shows a schematic flowchart of another data storage method provided by at least one embodiment of the present disclosure.

[0250] As ​ shown, another data storage method provided by at least one embodiment of the present disclosure at least includes steps S410 - S430.

[0251] In step S410: Receive a first data storage instruction to obtain the third starting coordinate of the third tensor to be processed in the second coordinate system determined by the second original tensor in the buffer, the third number of pixels included in the third tensor to be processed, and the number of fallback pixels for the fourth tensor to be processed.

[0252] In step S420: Determine the end coordinate of the last pixel in the third tensor to be processed in the second coordinate system according to the third starting coordinate and the third number of pixels.

[0253] In step S430: Determine the fourth starting coordinate of the fourth tensor to be processed corresponding to the second data storage instruction in the second coordinate system according to the end coordinate and the number of fallback pixels.

[0254] The above-mentioned second original tensor, third tensor to be processed, and fourth tensor to be processed are original tensors in the buffer. For the relevant descriptions of steps S410 - S430, reference can be made to the relevant content of steps S210 - S230 in the foregoing data loading method, which will not be elaborated here.

[0255] For example, the data storage format of the second original tensor is the same as that of the tensor to be processed (e.g., the third tensor to be processed or the fourth tensor to be processed), such as NDHWC, or N(C / x)DWH(Cx), where x is a positive integer greater than 1.

[0256] In ​ In the data storage method shown, based on the third starting coordinate of the third tensor to be processed indicated in the first data storage instruction, the third number of pixels included in the third tensor to be processed, and the number of fallback pixels for the fourth tensor to be processed, the fourth starting coordinate of the fourth tensor to be processed can be directly determined by the hardware device, omitting the steps of resetting the fourth starting coordinate before fallback and the number of fallback pixels in the second data storage instruction and the fallback operation when storing the fourth tensor to be processed, thereby simplifying the instruction structure (or instruction logic) of the data storage instruction, saving instruction resources, and improving the instruction execution efficiency.

[0257] According to some embodiments of the present disclosure, an electronic device is also provided. ​ A schematic block diagram of an electronic device according to some embodiments of the present disclosure is shown.

[0258] As ​ As shown, the electronic device 2000 may include a processor 2010 and a memory 2020 connected to the processor 2010. In addition, the processor 2010 may further include a buffer. According to the embodiments of the present disclosure, the memory 2020 may be implemented in the form of a high-bandwidth memory HBM, which is not limited thereto. Specifically, according to the embodiments of the present disclosure, the processor 2010 is configured to run computer-executable instructions, and when the computer-executable instructions are run by the processor 2010, a data loading method according to the embodiments of the present disclosure is implemented to load the first tensor to be processed and the second tensor to be processed from the first original tensor in the memory into the buffer, or a data storage method according to the embodiments of the present disclosure is implemented to write (or store) the third tensor to be processed and the fourth tensor to be processed from the second original tensor in the buffer into the memory.

[0259] The processor 2010 can perform various actions and processes according to a program stored in a non-transitory memory such as a non-volatile memory. Specifically, the processor 2010 can refer to a processor chip capable of performing parallel computing. For example, it can be any one of a GPU (Graphics Processing Unit), a TPU (Tensor Processing Unit), an NPU (Neural Network Processing Unit), a DPU (Deep Learning Processing Unit), an APU (Accelerated Processing Unit), and a GPGPU (General-Purpose computing on Graphics Processing Unit). In addition, the processor 2010 can also be implemented as other conventional types of processors, which are not limited herein.

[0260] Regarding the specific implementation processes of the data loading method and the data storage method, reference can be made to the above description, which will not be repeated here. The processor provided in at least one embodiment of the present disclosure can achieve technical effects similar to those of the foregoing data loading method or data storage method, and the repeated parts will not be elaborated.

[0261] ​ A block diagram showing an example computing device implementing some embodiments of the present disclosure is shown. As ​ shown, the computing device 3000 is, for example, suitable for implementing the data loading method or the data storage method provided by the embodiments of the present disclosure. It should be noted that ​ the components of the computing device 3000 shown are exemplary and not restrictive. According to actual application needs, the computing device 3000 may also have other components.

[0262] As ​ shown, the computing device 3000 may include a processing device 3010 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to non-transitory computer-readable instructions stored in the memory to implement various functions.

[0263] For example, when the computer-readable instructions are run by the processing device 3010, one or more steps in the data loading method according to any of the above embodiments, or one or more steps in the data storage method according to any of the above embodiments can be executed. It should be noted that for a detailed description of the processing procedure of the data loading method, reference can be made to the relevant descriptions in the embodiments of the data loading method above, and for a detailed description of the processing procedure of the data storage method, reference can be made to the relevant descriptions in the embodiments of the data storage method above.

[0264] For example, the processing device 3010, read-only memory (ROM) 3020, and random access memory (RAM) 3030 are connected to each other through the bus 3040. The input / output (I / O) interface 3050 is also connected to the bus 3040.

[0265] For example, the memory may include any combination of one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) 3030 and / or cache, etc. For example, the computer-readable instructions can be loaded from the storage device 3080 into the random access memory (RAM) 3030 to run the computer-readable instructions. Non-volatile memory may include, for example, read-only memory (ROM) 3020, hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), flash memory, etc. Various application programs and various data can also be stored in the computer-readable storage media, as well as various data used and / or generated by the application programs, etc.

[0266] Generally, the following devices can be connected to the I / O interface 3050: an input device 3060 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 3070 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 3080 including, for example, a magnetic tape, a hard disk, a flash memory, etc.; and a communication device 3090. The communication device 3090 can allow the computing device 3000 to communicate with other electronic devices wirelessly or wiredly to exchange data. Although ​A computing device 3000 with various devices is shown. However, it should be understood that it is not required to implement or have all the shown devices, and the computing device 3000 may alternatively implement or have more or fewer devices. For example, the processing device 3010 may control other components in the computing device 3000 to perform desired functions. The processing device 3010 may be a central processing unit (CPU), a tensor processing unit (TPU), or a graphics processing unit (GPU), etc., which has data processing capabilities and / or program execution capabilities. The GPU may be directly integrated into a system on chip (SOC), directly integrated onto the motherboard, or built into the northbridge chip of the motherboard.

[0267] According to some embodiments of the present disclosure, a non-transitory computer-readable storage medium is also provided, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, a data loading method or a data storage method according to the embodiments of the present disclosure is implemented.

[0268] ​ It is a schematic diagram of a non-transitory computer-readable storage medium provided by at least one embodiment of the present disclosure. For example, as ​ shown, the computer-readable storage medium 4000 may be a non-transitory computer-readable storage medium, and one or more computer-readable instructions 4010 may be non-temporarily stored on the storage medium 4000. For example, when the computer-readable instructions 4010 are executed by a processor, one or more steps of the data loading method according to any of the above embodiments may be executed, or one or more steps of the data storage method according to any of the above embodiments may be executed. It should be noted that for a detailed description of the processing process of the data loading method, reference may be made to the relevant descriptions in the embodiments of the data loading method above, and for a detailed description of the processing process of the data storage method, reference may be made to the relevant descriptions in the embodiments of the data storage method above.

[0269] As an example, the storage medium 4000 may be applied to the electronic device 2000 and / or the computing device 3000. For example, the storage medium 4000 may be implemented as the storage device 3080 in the computing device 3000.

[0270] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0271] The units involved in the embodiments described in the present disclosure can be implemented in software or in hardware. In some cases, the name of the unit does not constitute a limitation on the unit itself.

[0272] The functions described above can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), Application Specific Standard Parts (ASSP), System on Chip (SOC), Complex Programmable Logic Device (CPLD), and so on. The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles applied. Those skilled in the art should understand that the scope of the disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, technical solutions formed by mutually replacing the above features with (but not limited to) technical features having similar functions disclosed in the present disclosure.

[0273] In addition, although the operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in a sequential order. In certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the foregoing description, these should not be construed as limiting the scope of the present disclosure. Certain features that are described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, the various features that are described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0274] Although the subject matter has been described in language specific to structural features and / or methodological acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

[0275] For the present disclosure, the following points also need to be noted:

[0276] (1) The drawings of the embodiments of the present disclosure only relate to the structures involved in the embodiments of the present disclosure, and other structures can refer to the general design.

[0277] (2) Without conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0278] The above are only the specific implementation manners of the present disclosure, but the protection scope of the present disclosure is not limited thereto. The protection scope of the present disclosure shall be subject to the protection scope of the appended claims.

Claims

1. A data loading method for loading a tensor to be processed from a first original tensor in memory into a buffer, the data loading method comprising: Obtaining a first starting coordinate of a first tensor to be processed in a first coordinate system determined by the first original tensor and a first number of pixels included in the first tensor to be processed; And According to the first starting coordinate and the first number of pixels, directly determining, by a hardware device, a second starting coordinate of a second tensor to be processed in the first coordinate system, wherein a position of a last pixel included in the first tensor to be processed is adjacent to the second starting coordinate of the second tensor to be processed in the first coordinate system; Wherein the first coordinate system includes five dimensions, the five dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a number of channels dimension, data represented by the height dimension and the width dimension is represented as a pixel, and is accumulated step by step to higher dimensions, the number of channels dimension does not calculate the number of pixels, and the order of the five dimensions from low dimension to high dimension is the width dimension, the height dimension, the depth dimension, the batch dimension, and the number of channels dimension.

2. The data loading method according to claim 1, wherein, The directly determining, by a hardware device, a second starting coordinate of a second tensor to be processed in the first coordinate system according to the first starting coordinate and the first number of pixels includes: Determining a plurality of requests for loading the first tensor to be processed according to the first starting coordinate and the first number of pixels, wherein the plurality of requests are used to sequentially obtain data from the first original tensor starting from the first starting coordinate until the number of obtained data is equal to the first number of pixels; and Based on a last request among the plurality of requests, determining the second starting coordinate of the second tensor to be processed in the first coordinate system.

3. The data loading method according to claim 2, wherein, The determining a plurality of requests for loading the first tensor to be processed according to the first starting coordinate and the first number of pixels includes: Determining a first request among the plurality of requests and an initial state of the first request entering a state machine according to the first starting coordinate and the first number of pixels, wherein the first request indicates the number of pixels to be obtained by the first request; and Based on the initial state, combining the first number of pixels, the number of pixels to be obtained by the first request, and a data storage format of the first original tensor, determining each request after the first request among the plurality of requests by using the state machine.

4. The data loading method according to claim 3, further comprising: In response to a request currently processed by the state machine satisfying a preset condition and jumping to an idle state, determining that the request currently processed by the state machine is the last request among the plurality of requests, wherein the idle state represents a state after loading the first tensor to be processed.

5. The data loading method according to claim 2, wherein, The last request indicates the number of pixels to be obtained by the last request and a request initial coordinate corresponding to the last request. Determining the second starting coordinate of the second tensor to be processed in the first coordinate system based on the last one of the multiple requests includes: In response to partitioning requests in the height dimension, updating the request initial coordinate corresponding to the last request according to the number of pixels to be acquired by the last request to obtain the second starting coordinate.

6. The data loading method according to claim 5, wherein, The step of, in response to partitioning requests in the height dimension, updating the request initial coordinate corresponding to the last request according to the number of pixels to be acquired by the last request to obtain the second starting coordinate includes: Determining the coordinate value of the second starting coordinate in the width dimension as the sum of the coordinate value of the request initial coordinate in the width dimension and the number of pixels to be acquired by the last request, and determining the coordinate values of the second starting coordinate in the remaining dimensions other than the width dimension as the coordinate values of the request initial coordinate in the remaining dimensions other than the width dimension.

7. The data loading method according to claim 2, further comprising: Obtaining boundary values for at least a part of the five dimensions of the first original tensor respectively, where the boundary values are used to define the boundary range for acquiring data from the first original tensor, and for the dimensions with boundary values set, the data loaded by each request belongs to the valid data range defined by the first original tensor and the boundary values or does not belong to the valid data range; and Obtaining step sizes set for at least a part of the five dimensions for acquiring data, where the step sizes are positive integers.

8. The data loading method according to claim 7, wherein, The last request indicates the request initial coordinate corresponding to the last request, Determining the second starting coordinate of the second tensor to be processed in the first coordinate system based on the last one of the multiple requests includes: In response to partitioning requests in the width dimension, obtaining the second starting coordinate according to the boundary values, the step sizes, and the request initial coordinate.

9. A data loading method includes: Receiving a first data loading instruction to obtain a first starting coordinate of a first tensor to be processed in a first coordinate system determined by a first original tensor in a memory, a first number of pixels included in the first tensor to be processed, and a number of fallback pixels for a second tensor to be processed, where the first data loading instruction instructs to load the first tensor to be processed from the first original tensor into a buffer; Determining an end coordinate of the last pixel in the first tensor to be processed in the first coordinate system according to the first starting coordinate and the first number of pixels; Determining a second starting coordinate of the second tensor to be processed corresponding to a second data loading instruction in the first coordinate system according to the end coordinate and the number of fallback pixels, where the second data loading instruction instructs to load the second tensor to be processed from the first original tensor into the buffer; Among them, the first coordinate system includes 5 dimensions, and the 5 dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a number of channels dimension. The data determined by the height dimension and the width dimension is represented as a pixel, and is accumulated step by step to higher dimensions. The number of channels dimension does not calculate the number of pixels. The order of the 5 dimensions from the lower dimension to the higher dimension is the width dimension, the height dimension, the depth dimension, the batch dimension, and the number of channels dimension.

10. The data loading method according to claim 9, wherein, The determining of the end coordinate of the last pixel in the first coordinate system of the first tensor to be processed according to the first starting coordinate and the first number of pixels includes: Determining a plurality of requests for loading the first tensor to be processed according to the first starting coordinate and the first number of pixels, wherein the plurality of requests are used to sequentially obtain data from the first original tensor starting from the first starting coordinate until the number of obtained data is equal to the first number of pixels; Determining the last request among the plurality of requests through a state machine; and Determining the end coordinate of the last pixel in the first coordinate system of the first tensor to be processed according to the request initial coordinate corresponding to the last request and the number of pixels to be obtained by the last request.

11. The data loading method according to claim 9 further includes: Obtaining boundary values for at least a part of the 5 dimensions of the first original tensor respectively, wherein the boundary values are used to define the boundary range for obtaining data from the first original tensor. For the dimensions with boundary values set, the data loaded by each request belongs to the valid data range defined by the first original tensor and the boundary values or does not belong to the valid data range; and Obtaining step values set for at least a part of the 5 dimensions for obtaining data, wherein the step values are positive integers.

12. The data loading method according to claim 11, wherein, The product of the number of pixels to be rolled back and the step value in the width dimension is less than or equal to the width size of the first tensor to be processed in the width dimension. The determining of the second starting coordinate of the second tensor to be processed corresponding to the second data loading instruction in the first coordinate system according to the end coordinate and the number of pixels to be rolled back includes: Performing a rollback operation on the end coordinate according to the number of pixels to be rolled back and the step value; and Updating the end coordinate to obtain the second starting coordinate according to whether the rollback operation crosses the boundary corresponding to the boundary values of at least a part of the 5 dimensions of the first original tensor.

13. The data loading method according to claim 12, wherein, The number of fallback pixels is denoted as regr_pixel_num, the dimensions of the first tensor to be processed in the width dimension, height dimension, and depth dimension are respectively denoted as copy_w, copy_h, and copy_d, the stride values set in the width dimension, height dimension, and depth dimension are respectively denoted as stride_w, stride_h, and stride_d, and the coordinate values of the end point in the five dimensions are respectively denoted as w_cood_f, h_cood_f, d_cood_f, n_cood_f, and c_cood_f. Updating the end point coordinates to obtain the second start coordinates according to whether the fallback operation crosses the boundary corresponding to the boundary value of at least a part of the five dimensions of the first original tensor includes: In response to the fallback operation not crossing the boundary of the first original tensor in the width dimension, updating the coordinate value of the end point in the width dimension to w_cood_f - regr_pixel_num × stride_w, and using the updated end point coordinates as the second start coordinates.

14. The data loading method according to claim 13, wherein, Updating the end point coordinates to obtain the second start coordinates according to whether the fallback operation crosses the boundary corresponding to the boundary value of at least a part of the five dimensions of the first original tensor includes: In response to the fallback operation crossing the boundary of the first original tensor in the width dimension and not crossing the boundary of the first original tensor in the height dimension, updating the coordinate value of the end point in the width dimension to w_cood_f + copy_w - regr_pixel_num × stride_w, updating the coordinate value of the end point in the height dimension to h_cood_f - stride_h, and using the updated end point coordinates as the second start coordinates.

15. The data loading method according to claim 13, wherein, Updating the end point coordinates to obtain the second start coordinates according to whether the fallback operation crosses the boundary corresponding to the boundary value of at least a part of the five dimensions of the first original tensor includes: In response to the fallback operation crossing the boundary of the first original tensor in the height dimension and not crossing the boundary of the first original tensor in the depth dimension, updating the coordinate value of the end point in the width dimension to w_cood_f + copy_w - regr_pixel_num × stride_w, updating the coordinate value of the end point in the height dimension to h_cood_f + copy_h - stride_h, updating the coordinate value of the end point in the depth dimension to d_cood_f - stride_d, and using the updated end point coordinates as the second start coordinates.

16. The data loading method according to claim 13, wherein Updating the end coordinates to obtain the second starting coordinates according to whether the rollback operation crosses the boundary corresponding to the boundary value of at least a part of the five dimensions of the first original tensor includes: In response to the rollback operation crossing the boundary of the first original tensor in the depth dimension, updating the coordinate value of the end coordinates in the width dimension to w_cood_f + copy_w - regr_pixel_num × stride_w, updating the coordinate value of the end coordinates in the height dimension to h_cood_f + copy_h - stride_h, updating the coordinate value of the end coordinates in the depth dimension to d_cood_f + copy_d - stride_d, updating the coordinate value of the end coordinates in the batch dimension to n_cood_f minus 1, and using the updated end coordinates as the second starting coordinates.

17. A data storage method for obtaining a tensor to be processed based on a second original tensor in a buffer and writing the tensor to be processed into memory, the data storage method includes: Obtaining a third starting coordinate of a third tensor to be processed in a second coordinate system determined by the second original tensor and the number of third pixels included in the third tensor to be processed; And Directly determining, by a hardware device, a fourth starting coordinate of a fourth tensor to be processed in the second coordinate system according to the third starting coordinate and the number of third pixels, where the position of the last pixel included in the third tensor to be processed is adjacent to the fourth starting coordinate of the fourth tensor to be processed in the second coordinate system; Wherein, the second coordinate system includes five dimensions, the five dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a channel number dimension, the data represented by the height dimension and the width dimension is represented as a pixel, and is accumulated step by step to a higher dimension, the channel number dimension does not calculate the number of pixels, and the order of the five dimensions from a lower dimension to a higher dimension is the width dimension, the height dimension, the depth dimension, the batch dimension, and the channel number dimension.

18. A data storage method includes: Receiving a first data storage instruction to obtain a third starting coordinate of a third tensor to be processed in a second coordinate system determined by a second original tensor in a buffer, the number of third pixels included in the third tensor to be processed, and the number of rollback pixels for a fourth tensor to be processed, where the first data storage instruction instructs to obtain the third tensor to be processed from the buffer and store the third tensor to be processed in memory; Determining an end coordinate of the last pixel in the third tensor to be processed in the second coordinate system according to the third starting coordinate and the number of third pixels; Determine the fourth starting coordinate of the fourth to-be-processed tensor corresponding to the second data storage instruction in the second coordinate system according to the destination coordinate and the number of pixels to be backed off, where the second data storage instruction instructs to obtain the fourth to-be-processed tensor from the buffer and store the fourth to-be-processed tensor into the memory; Wherein, the second coordinate system includes five dimensions, the five dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a number-of-channels dimension. The data determined by the height dimension and the width dimension represents a pixel, and accumulates step by step to higher dimensions. The number-of-channels dimension does not calculate the number of pixels. The order of the five dimensions from the lower dimension to the higher dimension is the width dimension, the height dimension, the depth dimension, the batch dimension, and the number-of-channels dimension.

19. An electronic device, comprising a processor and a memory connected to the processor, wherein, The processor includes a buffer, wherein, The processor is configured to run computer-executable instructions, and when the computer-executable instructions are run by the processor, implement the data loading method according to any one of claims 1-16, or implement the data storage method according to claim 17 or 18.

20. A non-transitory computer-readable storage medium, wherein, The non-transitory computer-readable storage medium stores computer-executable instructions, When the computer-executable instructions are executed by a processor, implement the data loading method according to any one of claims 1-16, or implement the data storage method according to claim 17 or 18.

Images