Data loading method, data storage method, processor, electronic device, and medium

Abstract

The present disclosure provides a data loading method, a data storage method, a processor, an electronic device, and a medium. The data loading method is used to load a to-be-processed tensor from an original tensor in a memory into a cache, comprising: obtaining the number of pixels included in the to-be-processed tensor and the starting coordinates of the to-be-processed tensor in a coordinate system determined by the original tensor; sequentially obtaining data from the original tensor with the starting coordinates as the starting point until the number of data obtained is equal to the number of pixels included in the to-be-processed tensor; and loading the obtained data into the cache.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to a data loading method, a data storage method, a processor, an electronic device, and a medium. Background Art

[0002] A tensor is a multidimensional array data structure. Tensor operations are widely used in processors such as parallel processors. For example, in the field of deep learning, the dimensions of input data, intermediate data processed during the deep learning process, and output data are flexible and not exact, so a flexible data form is needed to describe various types of data, which gives rise to the concept of tensors. In the field of deep learning, all data to be calculated can be stored and exist in the form of tensors. If the data is not in the form of a tensor, it must first be converted into a tensor data structure. As an example, a scalar can be considered a 0-dimensional tensor, a vector can be considered a 1-dimensional tensor, and a matrix can be considered a 2-dimensional tensor. A tensor itself can have any number of dimensions. For example, it can be represented as a 5-dimensional array.

[0003] The development of artificial intelligence and machine learning has placed new demands on numerous parallel processing devices, represented by parallel processors (e.g., multi-core processors and digital signal processors). In general computing, the computational units of parallel processors must process large amounts of data, which is typically stored in the parallel processor's storage components, such as high-speed memory. Data load instructions can be used to load this data from the storage components into a cache for computation, and data store instructions can be used to store data in the cache into memory.

[0004] Providing fast and efficient data loading / storage methods is crucial to the computing performance of the device. Summary of the Invention

[0005] Embodiments of the present disclosure provide a data loading method, a data storage method, a processor, an electronic device, and a medium for providing a fast and efficient data loading / storage method, improving memory access bandwidth, and enhancing hardware utilization of computing devices.

[0006] According to a first aspect of the present disclosure, a data loading method is provided for loading a tensor to be processed from an original tensor in a memory into a cache area, wherein the original tensor is a 5-dimensional tensor, and its shape and size are represented by 5 parameters b1, b2, b3, b4, and b5, b1, b2, b3, b4, and b5 respectively indicate the sizes of the original tensor in 5 dimensions and are all positive integers, and the 5 dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a channel number dimension; wherein, for the 5-dimensional tensor of the original tensor, the data determined by the height dimension and the width dimension is represented as a pixel, and is accumulated step by step toward higher dimensions, wherein the number of pixels is not calculated in the channel number dimension, and the data loading method comprises: obtaining the number of pixels included in the tensor to be processed and the starting coordinates of the tensor to be processed in the coordinate system determined by the original tensor; for the data of the original tensor, starting from the starting coordinates, sequentially obtaining data from the original tensor until the number of data obtained is equal to the number of pixels included in the tensor to be processed; and loading the obtained data into the cache area.

[0007] According to some embodiments of the present disclosure, a plurality of tensors are stored in a memory, an original tensor is one of the plurality of tensors, and a data storage format of the plurality of tensors in the memory may be any one of NDHWC or N(C / x)DHW(xC), and the data storage format is used to indicate the storage order and dimensional arrangement of the tensors in the storage component, wherein, for the data storage format of NDHWC, N corresponds to b1, indicating a batch dimension, D corresponds to b2, indicating a depth dimension, H corresponds to b3, indicating a height dimension, W corresponds to b4, indicating a width dimension, and C corresponds to b5, indicating a channel number dimension; for the data storage format of N(C / x)DHW(xC), N corresponds to b1, indicating a batch dimension, (C / x) corresponds to b2, indicating a channel number dimension, D corresponds to b3, indicating a depth dimension, H corresponds to b4, indicating a height dimension, and W(xC) corresponds to b5, indicating a width dimension. In the data storage format of N(C / x)DHW(xC), x is a positive integer, and the number of channels of xC is bound to the width dimension.

[0008] According to some embodiments of the present disclosure, the tensor to be processed is used for a convolution operation in a computing unit within a processor.

[0009] According to some embodiments of the present disclosure, multiple tensors are stored in the memory, and the data loading method also includes: obtaining indication information of the storage location of the original tensor in the memory, the indication information including the starting coordinates of the original tensor in the memory and the size values ​​in five dimensions, respectively represented as tensor_b1, tensor_b2, tensor_b3, tensor_b4, and tensor_b5.

[0010] According to some embodiments of the present disclosure, loading the acquired data into the cache includes: loading the acquired data from the memory into the cache according to multiple instructions.

[0011] According to some embodiments of the present disclosure, loading the acquired data into the cache area also includes: determining a first data storage format of the tensor to be processed in the cache area; placing the tensor to be processed in the cache area according to the first data storage format, wherein the first data storage format of the tensor to be processed in the cache area is the same as or different from the second data storage format of the original tensor in the memory.

[0012] According to some embodiments of the present disclosure, the data loading method also includes: obtaining boundary values ​​for the original tensor in at least a part of the five dimensions, the boundary values ​​being used to limit the boundary range of obtaining data from the original tensor, wherein the boundary values ​​are arbitrary values ​​compared to the size value of the original tensor in this dimension.

[0013] According to some embodiments of the present disclosure, sequentially obtaining data from the original tensor includes: for the data portion of the original tensor covered by the range defined by the boundary value, sequentially obtaining data from the range of the original tensor defined by the boundary value; and for the data portion of the original tensor not covered by the range defined by the boundary value, representing it as invalid data, and directly filling the invalid data into the tensor to be processed from the cache according to a predetermined value, where the predetermined value is equal to 0.

[0014] According to some embodiments of the present disclosure, the data loading method also includes: obtaining a data step value, wherein the data step value is used to specify the step size for sequentially obtaining data from the original tensor, wherein, for the data of the original tensor, taking the starting coordinate as the starting point, sequentially obtaining data from the original tensor includes: for the data of the original tensor, taking the starting coordinate as the starting point, sequentially obtaining data from the original tensor according to the data step value.

[0015] According to a second aspect of the present disclosure, a data loading method is provided, comprising: receiving a data loading instruction for executing a load instruction for loading a to-be-processed tensor from an original tensor in a memory into a cache area, wherein the original tensor is a 5-dimensional tensor, and its shape size is represented by 5 parameters b1, b2, b3, b4, and b5, b1, b2, b3, b4, and b5 respectively indicate the size of the original tensor in 5 dimensions and are all positive integers, and the 5 dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a channel number dimension; wherein, for the 5-dimensional tensor of the original tensor, the height dimension and the width dimension determine The data is represented as a pixel, and is accumulated step by step to a higher dimension, wherein the number of pixels is not counted in the channel number dimension; and after parsing the data loading instruction, the data loading instruction is executed using the execution unit, wherein the execution unit is used to execute the data loading instruction, including: obtaining the number of pixels included in the tensor to be processed, and the starting coordinates of the tensor to be processed in the coordinate system determined by the original tensor; for the data of the original tensor, starting from the starting coordinates, sequentially obtaining data from the original tensor until the number of data obtained is equal to the number of pixels included in the tensor to be processed; and loading the obtained data into the cache area.

[0016] According to a third aspect of the present disclosure, a data storage method is provided for obtaining a second tensor based on a first tensor in a cache and writing the second tensor to a memory, wherein the first tensor is a 5-dimensional tensor, and its shape and size are represented by 5 parameters b1, b2, b3, b4, and b5, b1, b2, b3, b4, and b5 respectively indicate the size of the first tensor in 5 dimensions and are all positive integers, and the 5 dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a channel number dimension; wherein, for the 5-dimensional tensor of the first tensor, the data determined by the height dimension and the width dimension are represented as a pixel, and are accumulated step by step to higher dimensions, wherein the number of pixels is not counted in the channel number dimension. The data storage method according to an embodiment of the present disclosure includes: obtaining the number of pixels included in the second tensor and the starting coordinates of the second tensor in the coordinate system determined by the first tensor; for the data of the first tensor, starting from the starting coordinates, sequentially obtaining data from the first tensor until the number of data obtained is equal to the number of pixels included in the second tensor; and storing the obtained data in the memory.

[0017] According to some embodiments of the present disclosure, a plurality of tensors are stored in a cache, a first tensor is one of the plurality of tensors, and a data storage format of the plurality of tensors in the cache may be any one of NDHWC or N(C / x)DHW(xC), the data storage format being used to indicate the storage order and dimensional arrangement of the tensors in the storage component, wherein, for the data storage format of NDHWC, N corresponds to b1, indicating a batch dimension, D corresponds to b2, indicating a depth dimension, H corresponds to b3, indicating a height dimension, W corresponds to b4, indicating a width dimension, and C corresponds to b5, indicating a channel number dimension; for the data storage format of N(C / x)DHW(xC), N corresponds to b1, indicating a batch dimension, (C / x) corresponds to b2, indicating a channel number dimension, D corresponds to b3, indicating a depth dimension, H corresponds to b4, indicating a height dimension, and W(xC) corresponds to b5, indicating a width dimension. In the data storage format of N(C / x)DHW(xC), x is a positive integer, and the number of channels of xC is bound to the width dimension.

[0018] According to some embodiments of the present disclosure, a plurality of tensors are stored in a cache. The data storage method may further include: obtaining indication information of a storage location of a first tensor in the cache, the indication information including a starting coordinate of the first tensor in the cache and size values ​​in five dimensions, respectively represented as tensor_b1, tensor_b2, tensor_b3, tensor_b4, and tensor_b5.

[0019] According to some embodiments of the present disclosure, storing the acquired data into the memory includes: storing the acquired data from the cache into the memory according to multiple instructions.

[0020] According to some embodiments of the present disclosure, storing the acquired data in the memory also includes: determining a first data storage format of the second tensor in the memory; placing the second tensor in the memory according to the first data storage format, wherein the first data storage format of the second tensor in the memory is the same as or different from the second data storage format of the first tensor in the cache area.

[0021] According to some embodiments of the present disclosure, the data storage method further includes: obtaining boundary values ​​for the first tensor in at least a portion of the five dimensions, the boundary values ​​being used to limit the boundary range for obtaining data from the first tensor, wherein the boundary values ​​are arbitrary values ​​compared to the size value of the first tensor in the dimension.

[0022] According to some embodiments of the present disclosure, sequentially obtaining data from a first tensor includes: for a data portion of the first tensor whose range is defined by a boundary value and covers the data portion of the first tensor, sequentially obtaining data from the range of the first tensor defined by the boundary value; and for a data portion of the first tensor that is not defined by the range defined by the boundary value, representing it as invalid data, and not storing the invalid data in the memory.

[0023] According to some embodiments of the present disclosure, the data storage method also includes: obtaining a data step value, wherein the data step value is used to specify the step size for sequentially obtaining data from the first tensor, wherein, for the data of the first tensor, taking the starting coordinates as the starting point, sequentially obtaining data from the first tensor includes: for the data of the first tensor, taking the starting coordinates as the starting point, sequentially obtaining data from the first tensor according to the data step value.

[0024] According to a fourth aspect of the present disclosure, a data storage method is provided, comprising: receiving a data storage instruction for executing a data storage instruction for obtaining a second tensor based on a first tensor in a cache area and writing the second tensor into a memory, wherein the first tensor is a 5-dimensional tensor, and its shape size is represented by 5 parameters b1, b2, b3, b4, and b5, b1, b2, b3, b4, and b5 respectively indicate the sizes of the first tensor in 5 dimensions and are all positive integers, and the 5 dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a channel number dimension; wherein, for the 5-dimensional tensor of the first tensor, the height dimension, the width dimension, and the channel number dimension are represented by 5 parameters b1, b2, b3, b4, and b5 respectively. The data determined by the degree dimension is represented as a pixel and accumulated step by step toward higher dimensions, wherein the number of pixels is not counted in the channel number dimension; and after parsing the data storage instruction, the data storage instruction is executed using the execution unit, wherein the data storage instruction is executed using the execution unit, including: obtaining the number of pixels included in the second tensor and the starting coordinates of the second tensor in the coordinate system determined by the first tensor; for the data of the first tensor, starting from the starting coordinates, sequentially obtaining data from the first tensor until the number of data obtained is equal to the number of pixels included in the second tensor; and storing the obtained data in the memory.

[0025] According to a fifth aspect of the present disclosure, a processor is provided, comprising an instruction parsing unit and an execution unit, wherein the instruction parsing unit is configured to: receive and parse a data loading instruction, wherein the data loading instruction instructs execution to load a to-be-processed tensor from an original tensor in a memory into a cache area, wherein the original tensor is a 5-dimensional tensor, and its shape size is represented by 5 parameters b1, b2, b3, b4, and b5, b1, b2, b3, b4, and b5 respectively indicate the size of the original tensor in 5 dimensions and are all positive integers, and the 5 dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a channel number dimension; wherein, for the original tensor A 5-dimensional tensor of the initial tensor, in which data determined by the height dimension and the width dimension are represented as a pixel, and are accumulated step by step toward higher dimensions, wherein the channel number dimension does not count the number of pixels; and the execution unit is configured to: execute a data loading instruction, wherein the execution unit executes the data loading instruction, including: obtaining the number of pixels included in the tensor to be processed, and the starting coordinates of the tensor to be processed in the coordinate system determined by the original tensor; for the data of the original tensor, taking the starting coordinates as the starting point, sequentially obtaining data from the original tensor until the number of data obtained is equal to the number of pixels included in the tensor to be processed; and loading the obtained data into the cache area.

[0026] According to a sixth aspect of the present disclosure, a processor is provided, comprising an instruction parsing unit and an execution unit, wherein the instruction parsing unit is configured to: receive and parse a data storage instruction, wherein the data storage instruction instructs execution to obtain a second tensor based on a first tensor in a cache area and write the second tensor to a memory, wherein the first tensor is a 5-dimensional tensor, and its shape size is represented by 5 parameters b1, b2, b3, b4, and b5, b1, b2, b3, b4, and b5 respectively indicate the sizes of the first tensor in 5 dimensions and are all positive integers, and the 5 dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a channel number dimension; Wherein, for the 5-dimensional tensor of the first tensor, the data determined by the height dimension and the width dimension is represented as a pixel, and is accumulated step by step toward higher dimensions, wherein the channel number dimension does not count the number of pixels; and the execution unit is configured to: execute a data storage instruction, wherein the execution unit executes the data storage instruction, including: obtaining the number of pixels included in the second tensor, and the starting coordinates of the second tensor in the coordinate system determined by the first tensor; for the data of the first tensor, starting from the starting coordinates, sequentially obtaining data from the first tensor until the number of data obtained is equal to the number of pixels included in the second tensor; and storing the obtained data to the memory.

[0027] According to the seventh aspect of the present disclosure, an electronic device is provided, comprising a processor and a memory connected to the processor, wherein the processor includes a cache area, wherein the processor is configured to run computer-executable instructions, and when the computer-executable instructions are run by the processor, the data loading method according to the embodiment of the present disclosure is implemented to load the tensor to be processed from the original tensor in the memory to the cache area, or the data storage method according to the embodiment of the present disclosure is implemented to obtain a second tensor based on the first tensor in the cache area and write the second tensor to the memory.

[0028] According to an eighth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the data loading method according to an embodiment of the present disclosure is implemented, or the data storage method according to an embodiment of the present disclosure is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0030] Figure 1 A schematic diagram of the structure of a general-purpose graphics processing unit (GPGPU) is shown;

[0031] Figure 2 A schematic structure of a tensor is shown;

[0032] Figure 3A Shows a schematic diagram of the data storage format of NDHWC;

[0033] Figure 3B A schematic diagram of the data storage format of N(C / x)DHW(xC) is shown, where x=32;

[0034] Figure 4 A schematic diagram of obtaining a tensor in related art is shown;

[0035] Figure 5 A schematic flow chart of a data loading method provided by at least one embodiment of the present disclosure is shown;

[0036] Figure 6 A schematic diagram of obtaining a tensor to be processed in a continuous manner according to an embodiment of the present disclosure is shown;

[0037] Figure 7A A schematic diagram illustrating a data storage format for NDHWC according to an embodiment of the present disclosure for obtaining tensors to be processed in a continuous manner is shown;

[0038] Figure 7B A schematic diagram illustrating a data storage format for N(C / x)DHW(xC) according to an embodiment of the present disclosure for obtaining tensors to be processed in a continuous manner is shown;

[0039] Figure 8A A schematic diagram of setting boundary values ​​for an original tensor according to an embodiment of the present disclosure is shown;

[0040] Figure 8B A schematic diagram of performing continuous data acquisition with setting boundaries for original tensors according to an embodiment of the present disclosure is shown;

[0041] Figure 9 A schematic diagram showing a method of continuously acquiring tensors to be processed according to a set step size according to an embodiment of the present disclosure is shown;

[0042] Figure 10 A schematic flow chart of a data loading method provided by at least one embodiment of the present disclosure is shown;

[0043] Figure 11 A schematic flow chart of a data storage method provided by at least one embodiment of the present disclosure is shown;

[0044] Figure 12 A schematic flow chart of a data storage method provided by at least one embodiment of the present disclosure is shown;

[0045] Figure 13 shows a schematic block diagram of a processor according to some embodiments of the present disclosure;

[0046] Figure 14 A schematic block diagram of an electronic device according to some embodiments of the present disclosure is shown;

[0047] Figure 15 A block diagram illustrating an example computing device implementing some embodiments of the present disclosure is shown;

[0048] Figure 16 A schematic block diagram of a computer-readable storage medium according to some embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0050] Unless otherwise defined, the technical or scientific terms used in this disclosure should have the usual meanings understood by people with ordinary skills in the field to which this disclosure belongs. The "first", "second" and similar words used in this disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits detailed descriptions of some known functions and known components.

[0051] Figure 1 A schematic diagram of the structure of a GPGPU is shown. Figure 1 As shown, GPGPU is actually an array of programmable multiprocessors. For example, the programmable multiprocessor can be a streaming processor cluster (SPC), including Figure 1 The stream processor clusters shown are 1, ..., and M, where M is a positive integer greater than 1. In a general-purpose graphics processor, one stream processor cluster processes one computing task, or multiple stream processor clusters process one computing task. For example, multiple stream processor clusters share data through a global cache or high-bandwidth memory (HBM).

[0052] like Figure 1 As shown, taking stream processor cluster 1 as an example, a stream processor cluster may include multiple computing units (CUs), such as Figure 1In the calculation unit 1, calculation unit 2, ..., calculation unit K, K is a positive integer. Each calculation unit is used to perform arithmetic and logical operations, such as accumulation, reduction, conventional addition, subtraction, multiplication, division, etc. A calculation unit can include multiple cores (also called calculation cores or calculation cores), each of which includes an arithmetic logic unit (ALU), a floating-point calculation unit, etc. The calculation core is used to perform specific calculation tasks. In addition, the calculation unit also includes registers (such as Figure 1 The register file (RFS) and shared memory in the stream processor cluster are used to hierarchically store source and destination data related to computing tasks. The shared memory in a computing unit is used to share data between the cores of that computing unit. In addition, the cache area can be understood as being used to share data between computing units within the stream processor cluster.

[0053] In parallel computing, computing tasks are generally performed by multiple threads. These threads are divided into multiple thread blocks before being executed in a general-purpose graphics processor (or parallel computing processor), and then distributed through the thread block distribution module ( Figure 1 (Not shown) Multiple thread blocks are distributed to various CUs. All threads in a thread block must be assigned to the same CU for execution. Thread blocks are also split into minimum execution warps (or simply warps). Each warp contains a fixed number (or less) of threads, for example, 32 threads. Multiple thread blocks can execute in the same CU or in different CUs.

[0054] In each computational unit, the warp scheduling / dispatching module ( Figure 1 (not shown) schedules and allocates thread warps so that the multiple compute cores of the compute unit can execute them. Depending on the number of compute cores in the compute unit, multiple warps in a thread block can execute simultaneously or in a time-sharing manner. Multiple threads in each warp execute the same instruction. Memory execution instructions are emitted to the shared memory within the compute unit or further emitted to the mid-level cache, global cache, or high-bandwidth memory for read and write operations.

[0055] like Figure 1 As shown, general computing operations, such as matrix calculations in the field of artificial intelligence, typically require large amounts of data. This data is typically stored in memory, such as high-bandwidth memory (HBM). When performing general computing operations, data must be loaded from memory (a load operation), and when computing results are obtained, data must be stored in memory (a store operation). The way data is stored in memory affects memory access bandwidth, which in turn affects the hardware utilization of the computing unit.

[0056] For example, general computing operations include general matrix multiplication (GEMM). For example, the data required for GEMM is represented as two 5-dimensional arrays, such as the matrix multiplication of two tensors A and B. Furthermore, general computing operations include convolution, which is a dot product of data. It is understood that the field of artificial intelligence involves other computing operations, which are not listed here. The data involved in these calculations is typically represented in the form of tensors.

[0057] For example, for a tensor A in the cache, its shape and size can be represented by a1, a2, a3, a4, and a5, where a1, a2, a3, a4, and a5 respectively indicate the size of the tensor data in five dimensions, and a1, a2, a3, a4, and a5 are positive integers. For example, the five dimensions include [N, D, H, W, C], where the N dimension represents the batch size, that is, N represents the batch dimension, that is, the number of data samples captured for one training session, the D dimension represents the depth dimension, the H dimension represents the height dimension of the input data, the W dimension represents the width dimension of the input data, and the C dimension represents the number of channels. For example, taking tensor A as an example, a1 can be the N dimension, a2 can be the D dimension, a3 can be the H dimension, a4 can be the W dimension, and a5 can be the C dimension. Of course, the present disclosure does not impose any specific restrictions on this.

[0058] As an example, Figure 2 Shows a schematic structure of a tensor. Figure 2 In the tensor shown, a1 is the N dimension and is equal to 1, a2 is the D dimension and is equal to 1, a3 is the H dimension and is equal to 5, a4 is the W dimension and is equal to 4, and a5 is the C dimension and is equal to 64. For example, Figure 2 The pixel elements of the tensor in are represented as 0, 1, 2, 3, ..., and so on.

[0059] Tensors can be placed in a variety of formats in storage (such as memory or cache), which are called data storage formats (layouts). Data storage formats are used to indicate the storage order and dimension arrangement of tensors in storage components. Figure 2 The tensors shown describe different data storage formats.

[0060] In related art, a data storage format may include NDHWC, also known as Linear mode. Figure 3A A schematic diagram showing the data storage format of NDHWC is shown.

[0061] For example, for the NDHWC linear model, Figure 3A As shown, from the first channel (a5=0, Figure 3A The first element of c0 in ( Figure 3A 0 in the , and then stores the second channel (a5=1, Figure 3A The first element of c1 in ( Figure 3A 20), and so on, until the first elements of all channels are laid out, for example, to the 64th channel (a5=63, Figure 3A The first element of c63) ( Figure 3A After the element 1260 in , select the first channel (a5=0, Figure 3A The second element of c0 in ( Figure 3A 1 in the array), and then store the second channel (a5=1, Figure 3A The second element of c1 in ( Figure 3A ), and so on, until the second elements of all channels are laid out, and so on.

[0062] In related technologies, the data storage format may also include N(C / x)DHW(xC), also known as an Interleave mode, where x may be set to 8, 16, 32, etc. as needed.

[0063] The N(C / x)DHW(xC) data storage format is similar to the NDHWC data storage format, with one key difference. In the N(C / x)DHW(xC) layout, the a5 channels are divided into a5 / x groups, with x channels in each group: the first group consists of channels a5=0 to a5=x-1, and the second group consists of channels a5=x to a5=2x-1, with each group arranged in the NDHWC format.

[0064] Figure 3B A schematic diagram of the data storage format of N(C / x)DHW(xC) is shown, where x=32.

[0065] like Figure 3B As shown, the 64 channels are divided into two groups, each with 32 channels. The first group consists of channel a5=0 ( Figure 3B c0) to a5=31 ( Figure 3B The first group consists of channels a5=32 to a5=63. Each group is then arranged in NDHWC format.

[0066] In memory, for example, a tensor B, similar to a tensor A in the cache, can be stored in memory according to one of the two data storage formats described above. The shape and size of the tensor can be similarly expressed as b1×b2×b3×b4×b5, where b1, b2, b3, b4, and b5 respectively indicate the sizes of tensor B in the five dimensions and are all positive integers.

[0067] It is understood that in related technologies and possible future developments, the data storage format of tensors is not limited to the two described above, the N(C / x)DHW(xC) data storage format and the NDHWC data storage format, and the method described in this disclosure does not impose such limitations. Furthermore, the memory and the cache can store multiple tensors generated during the calculation process. Generally, the storage space of the memory is much larger than the cache, but it is farther away from the computing unit, and the data transfer efficiency is lower than that of the cache.

[0068] In the related art, during the calculation process of the processing device, a large amount of calculation data will be generated, for example, in the form of tensors, which can be temporarily stored in a buffer area. For example, the buffer area here can refer to Figure 1 The buffer in the stream processor cluster shown in FIG. 1 , further, the data can also be transferred from the buffer or directly stored in the memory, for example, the memory can be Figure 1 The high-bandwidth memory HBM shown in FIG. The storage format of tensors in the memory and cache can be, for example, any of the N(C / x)DHW(xC) data storage formats and NDHWC data storage formats described above. Therefore, during the calculation process, depending on technical requirements and the storage characteristics of the cache and memory, a large amount of data transfer is required between the cache and memory. For example, tensors in the cache can be stored in the memory using a store instruction, or tensors in the memory can be loaded into the cache using a load instruction.

[0069] It can be understood that in this article, the data storage process of storing tensors in the cache area to the memory and the data loading process of loading tensors in the memory to the cache area can be implemented in a similar manner. Therefore, for the sake of convenience of description, in some embodiments or examples, only the data loading process is described as an example. Those skilled in the art can apply it similarly to the data storage process. The differences between the two will be described separately.

[0070] As an example, Figure 4 FIG. 1 shows a schematic diagram of obtaining tensors in related technologies. Figure 4 As shown, a tensor A is stored in the memory, which can be placed in the memory in the data storage format of N(C / x)DHW(xC) or NDHWC, that is, the tensor A is a 5-dimensional array. Figure 4 Only the three dimensions W, H and C are schematically shown in FIG. Figure 4 The left side shows the dimensional coordinate system of tensor A, where W dimension, H dimension and C dimension are respectively. During the data loading process, all or part of the data in tensor A can be loaded into the cache through the load instruction, for example, Figure 4The tensor B in the image is loaded as a whole into the cache. Specifically, the loading instruction can indicate the first starting point of the tensor A to be loaded (C=0, W=0, H=0), the second starting point of the tensor B in the coordinate system of tensor A (C=0, W=3, H=0), and the size of the tensor B in each dimension. Schematically, in Figure 4 In the 3D diagram shown, the tensor B is a cuboid defined by the second starting point and the dimensions of each dimension. Using information about the second starting point and the dimensions of each tensor to be loaded, the memory can load the tensor B into the cache.

[0071] The block-wise data loading method used in the aforementioned related technologies is applicable to general-purpose computations such as GEMM (matrix matrix multiplication) operations. However, it is not suitable for convolution operations, which are characterized by point-wise multiplications. This method limits the computational efficiency of these pixel-by-pixel convolution operations, increases data access time, and reduces overall processor performance, limiting the further development of efficient, general-purpose processors.

[0072] To address the aforementioned technical issues in related technologies, the present disclosure provides a data loading method, a data storage method, a processor, an electronic device, and a non-transitory computer-readable storage medium. This disclosure proposes a new data transfer model in which tensors are no longer acquired and transferred in bulk, but rather sequentially in pixel units. This model is suitable for computational operations such as convolution, improves data transfer efficiency for such operations, reduces memory access time, and enhances overall processor performance.

[0073] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. It should be understood that the present disclosure is not limited to these specific embodiments.

[0074] At least one embodiment of the present disclosure provides a data loading method for loading a to-be-processed tensor from an original tensor in memory into a buffer. The memory according to this embodiment may be, for example, High Bandwidth Memory (HBM), and the buffer may be, for example, a buffer in a streaming processor cluster, without limitation.

[0075] As an example, the original tensor in memory is a 5-dimensional tensor, and its shape and size are represented by five parameters b1, b2, b3, b4, and b5. b1, b2, b3, b4, and b5 respectively indicate the sizes of the original tensor in five dimensions and are all positive integers. The five dimensions include batch dimension, depth dimension, height dimension, width dimension, and channel number dimension. As an example, the data storage format of the original tensor in memory can be either NDHWC or N(C / x)DHW(xC). The data storage format is used to indicate the storage order and dimensional arrangement of the tensor in the storage component. For example, the storage component refers to the above-mentioned HBM or cache area.

[0076] For the data storage format of NDHWC, N corresponds to b1, which represents the batch dimension, D corresponds to b2, which represents the depth dimension, H corresponds to b3, which represents the height dimension, W corresponds to b4, which represents the width dimension, and C corresponds to b5, which represents the number of channels dimension.

[0077] For the data storage format of N(C / x)DHW(xC), N corresponds to b1, which represents the batch dimension, (C / x) corresponds to b2, which represents the number of channels dimension, D corresponds to b3, which represents the depth dimension, H corresponds to b4, which represents the height dimension, and W(xC) corresponds to b5, which represents the width dimension. In the data storage format of N(C / x)DHW(xC), x is a positive integer, and the number of channels of xC is bound to the width dimension. In general, x can be set to an integer multiple of 4.

[0078] Regarding the characteristics of the above two data storage formats, you can refer to the above combined Figure 3A-3B The description will not be repeated here.

[0079] Furthermore, for the 5-dimensional data of the original tensor, the data determined by the height dimension and the width dimension is represented as a pixel (or, it can also be called an element), and is accumulated step by step to higher dimensions. The number of channels does not count the number of pixels. As an example, Figure 4 As shown in , the values ​​of each W dimension and H dimension can determine a pixel. For example, W=0 and H=0 correspond to the first pixel (or element) in tensor A, and W=1 and H=0 correspond to the second pixel in tensor A. In the tensor, the C dimension does not affect the number of pixels.

[0080] Figure 5 This is a schematic flow chart of a data loading method provided by at least one embodiment of the present disclosure. Figure 5 As shown, the data loading method provided by at least one embodiment of the present disclosure includes at least steps S101-S103.

[0081] In step S101, the number of pixels included in the tensor to be processed and the starting coordinates of the tensor to be processed in the coordinate system determined by the original tensor are obtained. Then, in step S102, for the data of the original tensor, starting from the starting coordinates, data is sequentially obtained from the original tensor until the number of data obtained is equal to the number of pixels included in the tensor to be processed. In step S103, the obtained data is loaded into the cache. According to an embodiment of the present disclosure, the tensor to be processed can be used for convolution operations in a computing unit within a processor.

[0082] In the data loading method according to the embodiment of the present disclosure, in order to obtain the tensor to be processed from the original tensor in the memory, it is necessary to indicate the starting coordinates of the tensor to be processed in the coordinate system determined by the original tensor. For example, the starting coordinates can be combined with Figure 4 The second starting point described (C=0, W=3, H=0). Furthermore, in the data loading method according to the embodiment of the present disclosure, it is also necessary to indicate the number of pixels included in the tensor to be processed (for example, expressed as copy_pixel_num), that is, the total number of pixels in the tensor that you want to load into the cache. This is different from the block-type data loading method in the above-mentioned related technology (wherein the size of the tensor to be processed in each dimension needs to be indicated). According to the data loading method of the embodiment of the present disclosure, the range of the tensor to be processed is determined by the number of pixels to be obtained. Furthermore, it is not obtained from the original tensor in block form, but data is obtained sequentially from the starting coordinates until the number of data obtained is equal to the number of pixels included in the tensor to be processed. The specific implementation details of data loading based on copy_pixel_num and starting coordinates will be expanded in detail below.

[0083] As an example, Figure 6 FIG. 1 shows a schematic diagram of obtaining tensors to be processed in a continuous manner according to an embodiment of the present disclosure. Figure 6 In the tensor A, the original tensor in memory is represented. Figure 6 In the example, each pixel is shown as a square. The tensor A covers all squares, that is, both white squares and gray-shaded squares. The starting point of the original tensor is represented as (0, 0, 0), which means C=0, W=0, H=0, for example, to indicate the location of the original tensor in memory. Figure 6 In the example, tensor B represents the tensor to be processed, and its starting point is represented as (C=0, W=3, H=0), that is, data is obtained from the third pixel in the first row of tensor A. Then, Figure 6The figure schematically illustrates the process of acquiring the tensor to be processed in a sequential (or continuous) manner according to an embodiment of the present disclosure, that is, as shown by the dotted arrow, starting from the starting point (C=0, W=3, H=0), data is acquired from tensor A, for example, pixel by pixel, until the number of acquired pixels reaches copy_pixel_num. Figure 6 In the example, only the three dimensions H, W, and C are shown. It is understandable that if the data of these three dimensions still does not reach copy_pixel_num, higher-dimensional data can be further obtained. There is no limitation here. Figure 6 As shown in , the C dimension itself does not affect the number of pixels. Assume that Figure 6 The data storage format of the tensor A in memory is NDHWC format. Figure 6 In the example shown, the total number of pixels in the tensor to be processed is copy_pixel_num=27.

[0084] In an embodiment according to the present disclosure, in step S102, for the data of the original tensor, starting from the starting coordinate, sequentially obtaining data from the original tensor includes: removing the channel number dimension from the 5 dimensions of the original tensor, and sequentially obtaining data in order from low to high dimensions with the starting coordinate as the starting point for data acquisition, until the number of pixels obtained reaches copy_pixel_num. The reason why the channel number dimension is removed from the 5 dimensions is that for tensors, the channel number dimension does not count the number of pixels. That is, S102 may include taking the starting coordinate as the starting point for data acquisition from the original tensor, and obtaining pixel data in the order of width dimension, height dimension, depth dimension, and batch dimension until the number of pixels obtained reaches copy_pixel_num.

[0085] Compare Figure 4 and Figure 6 The data loading method provided by the embodiment of the present disclosure can realize pixel-by-pixel data loading, instead of Figure 4The whole-block acquisition in this data loading method is more conducive to the calculation process such as convolution operation, and is conducive to improving the calculation efficiency. Therefore, based on the method provided by the embodiment of the present disclosure, for the data to be subjected to the convolution operation next, the processor can, for example, instruct in the form of instructions to fetch this part of the data from the memory into the cache area according to the above-mentioned continuous data loading method for use in the convolution operation. It can be understood that the above-mentioned processor can be reasonably used according to the type of operation to be performed or the data processing characteristics, whether it is according to the continuous data loading method or the whole-block data loading method, that is, it can support the adaptive switching of these two loading methods, which will not be further expanded here. In addition, the memory or cache area may also include a corresponding identifier to indicate the specific acquisition method of this tensor.

[0086] According to some embodiments of the present disclosure, multiple tensors are stored in the memory, and the data loading method further includes: obtaining indication information of the storage location of the original tensor in the memory, the indication information including the starting coordinates of the original tensor in the memory and the size values ​​in five dimensions, respectively represented as tensor_b1, tensor_b2, tensor_b3, tensor_b4, and tensor_b5. It is understandable that the storage space of the memory is generally much larger than the cache area, which can store various tensor data generated during the calculation process. These tensor data can be placed in the memory according to any of the above-mentioned N(C / x)DHW(xC) or NDHWC data storage formats. On the basis of step S101 according to the embodiment of the present disclosure, in order to make the memory aware of the specific location of the tensor to be obtained in the memory, it is also possible to obtain indication information of the storage location of the original tensor in the memory, and the indication information includes the starting coordinates of the original tensor in the memory and the size values ​​in five dimensions, which are respectively represented as tensor_b1, tensor_b2, tensor_b3, tensor_b4, and tensor_b5. As an example, when the original tensor is in the data storage format of NDHWC, tensor_b1, tensor_b2, tensor_b3, tensor_b4, and tensor_b5 respectively represent the values ​​of the original tensor in the five dimensions of N, D, H, W, and C.

[0087] According to some embodiments of the present disclosure, loading the acquired data into the cache includes: loading the acquired data from the memory into the cache according to multiple instructions. Figure 6 The tensor B shown in can be loaded into the cache through multiple instructions, for example, by splitting it into multiple requests, without limitation.

[0088] According to some embodiments of the present disclosure, loading the acquired data into the cache further includes: determining a first data storage format of the tensor to be processed in the cache; and placing the tensor to be processed in the cache according to the first data storage format, wherein the first data storage format of the tensor to be processed in the cache is the same as or different from the second data storage format of the original tensor in the memory. Before storing tensor B in the cache, the method according to the embodiment of the present disclosure may also indicate the data placement method in the cache, that is, whether it is placed according to the data storage format of NDHWC or according to the data storage format of N(C / x)DHW(xC). In the method according to the embodiment of the present disclosure, there is no restriction on the placement method of the tensor to be processed in the cache. It can be NDHWC or N(C / x)DHW(xC), which can be the same as or different from the placement method of the original tensor in the memory. The data loading method of the embodiment of the present disclosure does not require the placement method of the data. The only difference is that different placement methods may affect the pixel reading order of tensor B and the specific content corresponding to the multiple instructions required when loading tensor B into the cache.

[0089] The impact of data storage format on the loading process of the tensor to be processed ("tensor B") will be discussed below. Figure 7A-7B Describe. Figure 7A and Figure 7B In the example shown, the data storage method of the tensor to be processed in the cache area is the same as the data placement method of the original tensor in the memory. It can be understood that the method according to the embodiment of the present disclosure can also be applied to these two different situations.

[0090] Figure 7A A schematic diagram showing a data storage format for NDHWC according to an embodiment of the present disclosure for obtaining tensors to be processed in a continuous manner is shown. Figure 7A The data storage format of the tensor shown in the memory is in the form of NDHWC, or is called the arrangement method according to NDHWC.

[0091] exist Figure 7A In the example shown, the original tensor corresponds to the data shown in the grid in the figure, where the size of the original tensor in the C dimension is 8 (the C dimension is not shown in the figure), the size of the W dimension is 4 (W_dim=4), the size of the H dimension is 4 (H_dim=4), the size of the D dimension is 2 (D_dim=2), and the size of the N dimension is 2 (N_dim=2). Taking the N dimension as an example, N_dim=2 corresponds to Figure 7A Based on the above parameters, N0 and N1 in the original tensor can be obtained as 64 pixels in total.

[0092] Then, in Figure 7AIn the example, the coordinates of the starting point of the tensor to be processed in the original tensor are w_coord_b=2, h_coord_b=2, d_coord_b=0, n_coord_b=0, c_coord_b=0, and the number of pixels included in the tensor to be processed is copy_pixel_num=40. Based on the above information, the starting position of the tensor to be processed in the original tensor is Figure 7A As shown in the figure, in the dimension N=0 and D=0, the pixel point where W=2 and H=2 are located, starting from this pixel point, according to the arrangement of the data, in the order from low dimension to high dimension (that is, in the order of W, H, D, N, as shown in the figure, Figure 7A ), sequentially acquire data until the number of acquired data is equal to the number of pixels included in the tensor to be processed. Figure 7A In the example shown in , starting from a determined starting point, data is obtained pixel by pixel in the order of arrows, so as to load this part of the data in the original tensor in the memory into the cache as a tensor to be processed for subsequent operations such as convolution calculations.

[0093] Figure 7B A schematic diagram showing a data storage format for N(C / x)DHW(xC) according to an embodiment of the present disclosure for obtaining tensors to be processed in a continuous manner is shown. Figure 7B The tensor shown in the figure has a data storage format in the memory of N(C / x)DHW(xC), or is arranged in the form of N(C / x)DHW(xC). Figure 7B In the example, x=8, each 8 C channels are grouped together and bounded to the W dimension. Figure 7A The data storage format of NDHWC shown in is ,for the data storage format of N(C / x)DHW(xC), the C dimension in the tensor is of a higher rank, located between the N dimension and the D dimension.

[0094] exist Figure 7B In the example shown, the original tensor corresponds to the data shown in the grid in the figure, wherein the size of the original tensor in the C dimension is 16 (shown as two 8*C dimensions in the figure), the size of the W dimension is 4 (W_dim=4), the size of the H dimension is 4 (H_dim=4), the size of the D dimension is 2 (D_dim=2), and the size of the N dimension is 2 (N_dim=2). Therefore, it can be obtained that the total number of pixels included in the original tensor is 64.

[0095] Then, in Figure 7BIn the example, the coordinates of the starting point of the tensor to be processed in the original tensor are w_coord_b=2, h_coord_b=2, d_coord_b=0, n_coord_b=0, c_coord_b=0, and the number of pixels included in the tensor to be processed is copy_pixel_num=40. Based on the above information, the starting position of the tensor to be processed in the original tensor is Figure 7B As shown in , where N=0 and D=0 and the first 8*C dimension are at the pixel where W=2 and H=2, starting from this pixel, according to the arrangement of the data, in the order from low dimension to high dimension (that is, in the order of W, H, D, C, N, as shown in Figure 7B As shown by the arrows in the data acquisition diagram, data is acquired sequentially until the number of acquired data is equal to the number of pixels included in the tensor to be processed. Figure 7A The acquisition order shown in , since the level of C dimension is higher, Figure 7B In the example, first, according to the starting point, the data of W, H, D and the first 8*C dimension are obtained. Then, according to the order of W, H, D, the data of the second 8*C dimension is obtained, and finally the data of N dimension is obtained. It can be understood that in Figure 7A and Figure 7B In the example, the number of pixels of the tensor to be processed is 40, and the difference is only due to the different arrangement of the C dimension. In addition, it should be noted that for Figure 7B The starting point of the second 8*C dimension shown in the W and H dimensions should be aligned with the starting point of the first 8*C dimension.

[0096] exist Figure 7B In the example shown in , starting from a determined starting point, data is obtained pixel by pixel in the order of arrows, so as to load this part of the data in the original tensor in the memory into the cache as a tensor to be processed for subsequent operations such as convolution calculations.

[0097] According to some embodiments of the present disclosure, the data loading method further includes: obtaining boundary values ​​for the original tensor in at least a portion of the five dimensions, the boundary values ​​being used to limit the boundary range of obtaining data from the original tensor, wherein the boundary value is an arbitrary value compared to the size value of the original tensor in the dimension. As an example, left and right boundary values ​​can be set for, for example, the C dimension, W dimension, H dimension, and D dimension, respectively, for example, as (L-tensor_C, R-tensor_C), (L-tensor_W, R-tensor_W), (L-tensor_H, R-tensor_H), (L-tensor_D, R-tensor_D).

[0098] In combination Figure 7A and Figure 7B In the described method, the range (i.e., boundary value) of the original tensor is not limited, that is, the tensor to be processed is obtained from the complete data range of the original tensor. In the method according to the embodiment of the present disclosure, it is also proposed that the range of the tensor to be processed can be defined by setting a boundary value. Furthermore, in the implementation process, the boundary value can be set to any value compared to the size value of the original tensor in this dimension. In other words, the boundary value can exceed the range of the original tensor itself.

[0099] As an example, Figure 8A FIG. 1 shows a schematic diagram of setting boundary values ​​for the original tensor according to an embodiment of the present disclosure. Figure 8A As shown, the rectangular box represents the range covered by the original tensor, which can be any dimension in the original tensor, such as C dimension, W dimension, H dimension or D dimension. Generally, no boundary value is set for N dimension. Figure 8A The 6 sub-pictures of show the left boundary value and the right boundary value respectively. Figure 8A The relationship between the size of the original tensor in that dimension is shown as "L" and "R" in the figure.

[0100] According to an embodiment of the present disclosure, in the case of setting a boundary value, sequentially obtaining data from the original tensor includes: for the data portion of the original tensor covered by the range defined by the boundary value, sequentially obtaining data from the range of the original tensor defined by the boundary value, for example, referring to Figure 7A and Figure 7B In contrast, the data portion of the original tensor that is not covered by the range defined by the boundary value is represented as invalid data. The invalid data is directly filled into the tensor to be processed from the buffer according to the predetermined value, where the predetermined value is equal to 0.

[0101] For example, in Figure 8A In the first sub-image of , the left boundary value L and the right boundary value R are both on the left side of the original tensor, that is, the data to be obtained in this dimension are all invalid data. In this case, for the invalid data, for example, this part of the data can be automatically filled by sending a zero-fill instruction to the buffer area. For another example, in Figure 8A In the second sub-image, the left boundary value L is to the left of the left boundary of the original tensor data range, and the right boundary value R is to the left of the right boundary of the original tensor. That is, the tensor to be processed is partly invalid data and partly valid data in the original tensor. Schematically, in Figure 8A In the figure, the data corresponding to the oblique shaded area is valid data, and the rest are invalid data. In this case, for valid data, such as reference Figure 7A and Figure 7BThe described order is performed, and for invalid data, this part of data can be automatically filled by, for example, sending a zero-fill instruction to the buffer area.

[0102] In the method according to the embodiment of the present disclosure, by setting a boundary value for the original tensor, the range of the tensor to be processed can be further delineated. In practical applications, this implementation method can adapt to the characteristics of operations such as convolution, for example, it is helpful to reduce the amount of calculation and greatly improve the flexibility of the data. As an example, assume that the original tensor corresponds to an intermediate tensor for feature extraction of an entire input image, and the input image includes a specific target, such as an object to be identified, and the object does not cover the entire image, that is, the image includes a background portion. In this case, by setting a boundary value, the range of the tensor to be processed to be obtained can be limited to the portion of the original tensor corresponding to the specific target, so as to reduce the amount of calculation of subsequent operations such as convolution and improve processing efficiency.

[0103] As an example, Figure 8B A schematic diagram of continuous data acquisition with boundaries set for the original tensor is shown. Figure 6 The situation shown in Figure 8B It can be understood as targeting Figure 6 The C, W, and H dimensions of tensor A in the are bounded, and the bounds are all within the size range of tensor A in the C, W, and H dimensions, which is equivalent to Figure 8A The fourth sub-image in the figure shows the boundary situation. Specifically, Figure 8B The outer box in the whole corresponds to the tensor A (corresponding to Figure 6 Tensor A in ), after setting the boundary, according to the data loading method of the embodiment of the present disclosure, the data will be sequentially acquired within the set boundary range with the starting point coordinates as the starting point until the number of acquired data is equal to the number of pixels included in the tensor to be processed. Figure 8B In the data part composed of squares, the data range is defined by the boundary values ​​set for the C dimension, W dimension, and H dimension, and data is sequentially acquired from the data range defined by the boundary. Data outside the data range defined by the boundary can be regarded as invalid data. Figure 8B The sequential data acquisition process can be referred to in conjunction with Figure 6 The description is not repeated here.

[0104] According to some embodiments of the present disclosure, the data loading method also includes: obtaining a data stride value, wherein the data stride value is used to specify the stride (Stride) for sequentially obtaining data from the original tensor, wherein, for the data of the original tensor, taking the starting coordinate as the starting point, sequentially obtaining data from the original tensor includes: for the data of the original tensor, taking the starting coordinate as the starting point, sequentially obtaining data from the original tensor according to the data stride value.

[0105] Figure 9 The figure shows a schematic diagram of obtaining the tensor to be processed in a continuous manner according to the set step size according to an embodiment of the present disclosure, wherein the step size is equal to 2, that is, one data is obtained every other pixel, that is, only the pixels shown in the shaded part are sequentially obtained, and the step size equal to 2 means skipping one pixel. Similarly, when the step size is equal to 3, it means skipping two pixels, and so on. It can be understood that when the set step size is equal to 1, the corresponding Figure 7A-7B The pixel-by-pixel data loading method shown in the figure. In practical applications, setting the step size can further reduce the amount of computation required for subsequent operations such as convolution, improve processing efficiency, and further increase the flexibility of data loading.

[0106] The data loading method provided by the embodiment of the present disclosure provides a new data acquisition mode that is different from the block-based data acquisition method, that is, it can realize pixel-by-pixel data loading according to the starting point and the number of pixels to be acquired (such as Figure 6 shown), instead of Figure 4 This data loading method is more conducive to the calculation process such as convolution operation, and is conducive to improving the calculation efficiency. Specifically, tensors are no longer obtained and transported in the form of whole blocks, but are sequentially obtained and transported in units of pixels, which is suitable for calculation operations such as convolution operations, improving the data transportation efficiency for such operations, reducing memory access time, and improving the overall performance of the processor.

[0107] According to some embodiments of the present disclosure, another data loading method is provided. Figure 10 FIG. 1 shows a schematic flow chart of a data loading method provided by at least one embodiment of the present disclosure. Figure 10 As shown, the data loading method according to the embodiment of the present disclosure includes step S201 and step S202.

[0108] In step S201 : a data loading instruction is received to load a tensor to be processed from an original tensor in a memory into a cache.

[0109] According to an embodiment of the present disclosure, the original tensor is a 5-dimensional tensor, and its shape and size are represented by 5 parameters b1, b2, b3, b4, and b5, where b1, b2, b3, b4, and b5 respectively indicate the sizes of the original tensor in 5 dimensions and are all positive integers. The 5 dimensions include batch dimension, depth dimension, height dimension, width dimension, and channel number dimension; wherein, for the 5-dimensional tensor of the original tensor, the data determined by the height dimension and the width dimension is represented as a pixel, and is accumulated step by step toward higher dimensions, wherein the number of pixels is not counted in the channel number dimension.

[0110] In step S202 , after parsing the data load instruction, the execution unit is used to execute the data load instruction.

[0111] like Figure 10 As shown, step S202 uses the execution unit to execute the data loading instruction, including:

[0112] S2021: Obtain the number of pixels included in the tensor to be processed and the starting coordinates of the tensor to be processed in the coordinate system determined by the original tensor;

[0113] S2022: For the data of the original tensor, starting from the starting coordinate, sequentially obtain data from the original tensor until the number of obtained data is equal to the number of pixels included in the tensor to be processed; and

[0114] S2023: Load the acquired data into the cache area.

[0115] For the relevant description of the tensors to be processed and the original tensors, as well as the specific implementation process of steps S2021-S2023, please refer to the relevant description of the aforementioned data loading method, and the repetitive parts will not be repeated here.

[0116] As an example, the data load instruction may be a machine instruction, or the data load instruction may be a microinstruction. For example, the data load instruction is implemented in the form of a Load instruction.

[0117] According to some embodiments of the present disclosure, a data storage method is further provided for obtaining a second tensor based on a first tensor in a cache area and writing the second tensor to a memory. It is understandable that the data storage method according to the embodiment of the present disclosure can be understood as the inverse process of the data loading method described above, that is, moving data from the cache area to the memory. The similarities between the implementation principle of the embodiment of the present disclosure and the above-mentioned data loading method will not be repeated, and only the differences will be described.

[0118] According to the data storage method of the embodiment of the present disclosure, the first tensor is a 5-dimensional tensor, and its shape and size are represented by 5 parameters b1, b2, b3, b4, and b5, b1, b2, b3, b4, and b5 respectively indicate the sizes of the first tensor in 5 dimensions and are all positive integers. The 5 dimensions include batch dimension, depth dimension, height dimension, width dimension, and channel number dimension; wherein, for the 5-dimensional tensor of the first tensor, the data determined by the height dimension and the width dimension is represented as a pixel, and is accumulated step by step toward higher dimensions, wherein the number of pixels is not counted in the channel number dimension. As an example, the first tensor may refer to a tensor stored in a cache area, which may be any of the data storage formats of NDHWC or N(C / x)DHW(xC), without limitation. In terms of understanding the implementation principle, it can be understood as the original tensor in the data loading method described above, that is, part or all of the data is obtained from the first tensor and transferred to the memory. This part of the tensor obtained from the first tensor is represented as the second tensor, which can be understood as the tensor to be processed in the data loading method described above.

[0119] Figure 11 A schematic flow chart of a data storage method provided by at least one embodiment of the present disclosure is shown. Figure 11 As shown, the data storage method includes steps S301-S303.

[0120] In step S301, the number of pixels included in the second tensor and the starting coordinates of the second tensor in the coordinate system determined by the first tensor are obtained. Next, in step S302, starting from the starting coordinates, data is sequentially obtained from the first tensor until the number of data obtained equals the number of pixels included in the second tensor. In step S303, the obtained data is stored in memory.

[0121] In the data storage method according to the embodiment of the present disclosure, the method of sequentially obtaining data from the first tensor can refer to the method described in conjunction with Figure 6 Compared with the block-type data storage method in the related art, the data storage method provided by the embodiment of the present disclosure can realize pixel-by-pixel data storage instead of Figure 4 This data storage method is more conducive to the calculation process such as convolution operation, and is conducive to improving the calculation efficiency. It is understandable that, for example, the processor can be used reasonably according to the type of operation to be performed or the data processing characteristics, whether it is a continuous data loading method or a block data loading method, that is, it can support adaptive switching of these two loading methods, which will not be further expanded here. In addition, the memory or cache area may also include a corresponding identifier to indicate the specific acquisition method of this tensor.

[0122] In practical applications, for example, for convolution operations in general computing operations, the operation process is multi-layered. For example, after the first layer of processing is completed, the data results need to be stored for use in the next layer of processing. As described above, for data that needs to be convolutional, the sequential data acquisition method provided by the embodiment of the present disclosure is adopted (refer to the above combined with Figure 5 、 Figure 6 、 Figure 7A 、 Figure 7B 、 Figure 8A and Figure 8B For example, 1x3 weight data can only be acquired in a sliding window format, so it can only be acquired row by row, unless the shape of the graph to be acquired is very regular, such as a cube, and the next layer does not require padding.

[0123] According to some embodiments of the present disclosure, a plurality of tensors are stored in a cache, a first tensor is one of the plurality of tensors, and a data storage format of the plurality of tensors in the cache may be any one of NDHWC or N(C / x)DHW(xC), the data storage format being used to indicate the storage order and dimensional arrangement of the tensors in the storage component, wherein, for the data storage format of NDHWC, N corresponds to b1, indicating a batch dimension, D corresponds to b2, indicating a depth dimension, H corresponds to b3, indicating a height dimension, W corresponds to b4, indicating a width dimension, and C corresponds to b5, indicating a channel number dimension; for the data storage format of N(C / x)DHW(xC), N corresponds to b1, indicating a batch dimension, (C / x) corresponds to b2, indicating a channel number dimension, D corresponds to b3, indicating a depth dimension, H corresponds to b4, indicating a height dimension, and W(xC) corresponds to b5, indicating a width dimension. In the data storage format of N(C / x)DHW(xC), x is a positive integer, and the number of channels of xC is bound to the width dimension.

[0124] According to some embodiments of the present disclosure, a plurality of tensors are stored in a cache. The data storage method may further include: obtaining indication information of a storage location of a first tensor in the cache, the indication information including a starting coordinate of the first tensor in the cache and size values ​​in five dimensions, respectively represented as tensor_b1, tensor_b2, tensor_b3, tensor_b4, and tensor_b5.

[0125] According to some embodiments of the present disclosure, storing the acquired data into the memory includes: storing the acquired data from the cache into the memory according to multiple instructions.

[0126] According to some embodiments of the present disclosure, storing the acquired data in the memory also includes: determining a first data storage format of the second tensor in the memory; placing the second tensor in the memory according to the first data storage format, wherein the first data storage format of the second tensor in the memory is the same as or different from the second data storage format of the first tensor in the cache area.

[0127] According to some embodiments of the present disclosure, the data storage method further includes: obtaining boundary values ​​for the first tensor in at least a portion of the five dimensions, the boundary values ​​being used to limit the boundary range for obtaining data from the first tensor, wherein the boundary values ​​are arbitrary values ​​compared to the size value of the first tensor in the dimension.

[0128] According to some embodiments of the present disclosure, sequentially obtaining data from a first tensor includes: for a data portion of the first tensor whose range is defined by a boundary value and covers the data portion of the first tensor, sequentially obtaining data from the range of the first tensor defined by the boundary value; and for a data portion of the first tensor that is not defined by the range defined by the boundary value, representing it as invalid data, and not storing the invalid data in the memory.

[0129] For invalid data that exceeds the range, the data storage method handles it differently from the data loading method according to the above-mentioned embodiment of the present disclosure. For the range data of the first tensor that exceeds the cache area, it can be understood that it is meaningless for the storage of the memory, and thus, this part of the data can be directly not stored. In comparison, in the data loading method according to the embodiment of the present disclosure, for invalid data that exceeds the boundary of the original tensor, it can be directly filled by the cache area, for example, filled with a predetermined value. As an example, the predetermined value can achieve a value of 0.

[0130] According to some embodiments of the present disclosure, the data storage method also includes: obtaining a data step value, wherein the data step value is used to specify the step size for sequentially obtaining data from the first tensor, wherein, for the data of the first tensor, taking the starting coordinates as the starting point, sequentially obtaining data from the first tensor includes: for the data of the first tensor, taking the starting coordinates as the starting point, sequentially obtaining data from the first tensor according to the data step value.

[0131] It can be understood that the data storage method according to the embodiment of the present disclosure can achieve a similar technical effect as the data loading method according to the embodiment of the present disclosure.

[0132] According to some embodiments of the present disclosure, another data storage method is provided. Figure 12 FIG. 1 shows a schematic flow chart of a data storage method provided by at least one embodiment of the present disclosure. Figure 12 As shown, the data storage method according to the embodiment of the present disclosure includes step S401 and step S402.

[0133] In step S401 , a data storage instruction is received to instruct execution of obtaining a second tensor based on a first tensor in a cache area and writing the second tensor into a memory.

[0134] According to an embodiment of the present disclosure, the first tensor is a 5-dimensional tensor, and its shape and size are represented by 5 parameters b1, b2, b3, b4, and b5, where b1, b2, b3, b4, and b5 respectively indicate the sizes of the first tensor in 5 dimensions and are all positive integers. The 5 dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a channel number dimension; wherein, for the 5-dimensional tensor of the first tensor, the data determined by the height dimension and the width dimension is represented as a pixel, and is accumulated step by step toward higher dimensions, wherein the number of pixels is not counted in the channel number dimension.

[0135] Step S402: After parsing the data storage instruction, the execution unit is used to execute the data storage instruction.

[0136] like Figure 12 As shown, step S402 uses the execution unit to execute the data storage instruction, including:

[0137] S4021: Obtain the number of pixels included in the second tensor and the starting coordinates of the second tensor in the coordinate system determined by the first tensor;

[0138] S4022: For the data of the first tensor, starting from the starting coordinate, sequentially obtain data from the first tensor until the number of obtained data is equal to the number of pixels included in the second tensor; and

[0139] S4023: Store the acquired data into the memory.

[0140] For the relevant description of the first tensor and the second tensor and the specific implementation process of steps S4021-S4023, please refer to the relevant description of the aforementioned data storage method, and the repeated parts will be omitted.

[0141] As an example, the data storage instruction may be a machine instruction, or the data storage instruction may be a microinstruction. For example, the data storage instruction is implemented in the form of a Store instruction.

[0142] According to some embodiments of the present disclosure, a processor is further provided, including an instruction parsing unit and an execution unit, wherein the instruction parsing unit is configured to: receive and parse a data loading instruction, wherein the data loading instruction instructs execution to load a to-be-processed tensor from an original tensor in a memory into a cache area, wherein the original tensor is a 5-dimensional tensor, and its shape size is represented by 5 parameters b1, b2, b3, b4, and b5, b1, b2, b3, b4, and b5 respectively indicate the size of the original tensor in 5 dimensions and are all positive integers, and the 5 dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a channel number dimension; wherein, for A 5-dimensional tensor of the original tensor, in which data determined by the height dimension and the width dimension are represented as a pixel, and are accumulated step by step toward higher dimensions, wherein the channel number dimension does not count the number of pixels; and an execution unit is configured to: execute a data loading instruction, wherein the execution unit executes the data loading instruction, including: obtaining the number of pixels included in the tensor to be processed and the starting coordinates of the tensor to be processed in the coordinate system determined by the original tensor; for the data of the original tensor, starting from the starting coordinates, sequentially obtaining data from the original tensor until the number of data obtained is equal to the number of pixels included in the tensor to be processed; and loading the obtained data into a cache area.

[0143] For example, multiple tensors are stored in the memory, and the original tensor is one of the multiple tensors. The data storage format of the multiple tensors in the memory can be any one of NDHWC or N(C / x)DHW(xC). The data storage format is used to indicate the storage order and dimensional arrangement of the tensors in the storage component. For the data storage format of NDHWC, N corresponds to b1, indicating the batch dimension, D corresponds to b2, indicating the depth dimension, H corresponds to b3, indicating the height dimension, W corresponds to b4, indicating the width dimension, and C corresponds to b5, indicating the number of channels. For the data storage format of N(C / x)DHW(xC), N corresponds to b1, indicating the batch dimension, (C / x) corresponds to b2, indicating the number of channels, D corresponds to b3, indicating the depth dimension, H corresponds to b4, indicating the height dimension, and W(xC) corresponds to b5, indicating the width dimension. In the data storage format of N(C / x)DHW(xC), x is a positive integer, and the number of channels of xC is bound to the width dimension. Generally, x can be set to an integer multiple of 4.

[0144] For example, the tensor to be processed is used for convolution operations in the computing unit within the processor. It is understandable that the above-mentioned processor can be used reasonably according to the type of operation to be performed or the data processing characteristics, whether it is a continuous data loading method or a block data loading method, that is, it can support adaptive switching of these two loading methods, which will not be further expanded here. In addition, the memory or cache area may also include a corresponding identifier to indicate the specific method of obtaining this tensor.

[0145] For example, there are multiple tensors stored in the memory, and the execution unit executes the data loading instruction further including: obtaining indication information of the storage location of the original tensor in the memory, the indication information including the starting coordinates of the original tensor in the memory and the size values ​​in five dimensions, which are respectively represented as tensor_b1, tensor_b2, tensor_b3, tensor_b4, and tensor_b5.

[0146] For example, the execution unit loading the acquired data into the cache includes: loading the acquired data from the memory into the cache according to a plurality of instructions.

[0147] For example, the execution unit loads the acquired data into the cache area and further includes: determining a first data storage format of the tensor to be processed in the cache area; and placing the tensor to be processed in the cache area according to the first data storage format, wherein the first data storage format of the tensor to be processed in the cache area is the same as or different from the second data storage format of the original tensor in the memory.

[0148] For example, the execution unit executing the data loading instruction also includes: obtaining boundary values ​​for the original tensor in at least a part of the five dimensions, the boundary values ​​are used to limit the boundary range of obtaining data from the original tensor, wherein the boundary values ​​are arbitrary values ​​compared to the size value of the original tensor in this dimension.

[0149] For example, the execution unit sequentially obtains data from the original tensor, including: for the data part of the original tensor covered by the range defined by the boundary value, sequentially obtaining data from the range of the original tensor defined by the boundary value; and for the data part of the original tensor not covered by the range defined by the boundary value, representing it as invalid data, the invalid data is directly filled into the tensor to be processed by the cache according to the predetermined value, where the predetermined value is equal to 0.

[0150] For example, the execution unit executing the data load instruction further includes: obtaining a data step value, wherein the data step value is used to specify a step size for sequentially obtaining data from the original tensor. Specifically, the execution unit executing the data of the original tensor, starting from the starting coordinate, sequentially obtaining data from the original tensor includes: sequentially obtaining data from the original tensor according to the data step value, starting from the starting coordinate.

[0151] According to some embodiments of the present disclosure, a processor is further provided, including an instruction parsing unit and an execution unit, wherein the instruction parsing unit is configured to: receive and parse a data storage instruction, wherein the data storage instruction instructs execution to obtain a second tensor based on a first tensor in a cache area and write the second tensor into a memory, wherein the first tensor is a 5-dimensional tensor, and its shape size is represented by 5 parameters b1, b2, b3, b4, and b5, b1, b2, b3, b4, and b5 respectively indicate the sizes of the first tensor in 5 dimensions and are all positive integers, and the 5 dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a channel number dimension ; wherein, for the 5-dimensional tensor of the first tensor, the data determined by the height dimension and the width dimension is represented as a pixel, and is accumulated step by step toward higher dimensions, wherein the number of channels dimension does not count the number of pixels; and the execution unit is configured to: execute a data storage instruction, wherein the execution unit executes the data storage instruction, including: obtaining the number of pixels included in the second tensor, and the starting coordinates of the second tensor in the coordinate system determined by the first tensor; for the data of the first tensor, starting from the starting coordinates, sequentially obtaining data from the first tensor until the number of data obtained is equal to the number of pixels included in the second tensor; and storing the obtained data to the memory.

[0152] For example, a plurality of tensors are stored in the cache, and the first tensor is one of the plurality of tensors. The data storage format of the plurality of tensors in the cache may be any one of NDHWC or N(C / x)DHW(xC). The data storage format is used to indicate the storage order and dimensional arrangement of the tensors in the storage component. For the data storage format of NDHWC, N corresponds to b1, indicating the batch dimension, D corresponds to b2, indicating the depth dimension, H corresponds to b3, indicating the height dimension, W corresponds to b4, indicating the width dimension, and C corresponds to b5, indicating the number of channels. For the data storage format of N(C / x)DHW(xC), N corresponds to b1, indicating the batch dimension, (C / x) corresponds to b2, indicating the number of channels, D corresponds to b3, indicating the depth dimension, H corresponds to b4, indicating the height dimension, and W(xC) corresponds to b5, indicating the width dimension. In the data storage format of N(C / x)DHW(xC), x is a positive integer, and the number of channels of xC is bound to the width dimension.

[0153] For example, multiple tensors are stored in the cache area, and the execution unit executes the data storage instruction also including: obtaining indication information of the storage location of the first tensor in the cache area, the indication information including the starting coordinates of the first tensor in the cache area and the size values ​​in five dimensions, respectively represented as tensor_b1, tensor_b2, tensor_b3, tensor_b4, and tensor_b5.

[0154] For example, the execution unit storing the acquired data into the memory includes: storing the acquired data from the cache into the memory according to the plurality of instructions.

[0155] For example, the execution unit storing the acquired data into the memory also includes: determining a first data storage format of the second tensor in the memory; and placing the second tensor in the memory according to the first data storage format, wherein the first data storage format of the second tensor in the memory is the same as or different from the second data storage format of the first tensor in the cache area.

[0156] For example, the execution unit executes the data storage instruction and also includes: obtaining boundary values ​​for the first tensor in at least a part of the five dimensions, the boundary values ​​are used to limit the boundary range of obtaining data from the first tensor, wherein the boundary values ​​are arbitrary values ​​compared to the size value of the first tensor in the dimension.

[0157] For example, the execution unit sequentially obtains data from the first tensor, including: for the data portion of the first tensor covered by the range defined by the boundary value, sequentially obtaining data from the range of the first tensor defined by the boundary value; and for the data portion of the first tensor not covered by the range defined by the boundary value, representing it as invalid data, and not storing the invalid data in the memory.

[0158] For example, the execution unit executing the data storage instruction further includes: obtaining a data step value, wherein the data step value is used to specify a step size for sequentially obtaining data from the first tensor. Specifically, the execution unit executing the data of the first tensor, starting from the starting coordinate, sequentially obtaining data from the first tensor includes: sequentially obtaining data from the first tensor according to the data step value, starting from the starting coordinate.

[0159] As an example, Figure 13 : shows a schematic block diagram of a processor according to some embodiments of the present disclosure, as shown in FIG. Figure 13 As shown, the processor 1000 may include an instruction parsing unit 1010 and an execution unit 1020. It is understood that the processor 1000 may be implemented to execute the data loading method according to an embodiment of the present disclosure to load a tensor to be processed from an original tensor in a memory to a cache area, or implement the data storage method according to an embodiment of the present disclosure to obtain a second tensor based on a first tensor in a cache area and write the second tensor to the memory.

[0160] Regarding the specific implementation process of the data storage method and the data loading method, please refer to the above description and will not be repeated here. The processor provided by at least one embodiment of the present disclosure can achieve similar technical effects as the aforementioned data loading method / data storage method, and the repeated parts will not be repeated here.

[0161] According to some embodiments of the present disclosure, an electronic device is also provided. Figure 14 1 shows a schematic block diagram of an electronic device according to some embodiments of the present disclosure. Figure 14 As shown, the electronic device 2000 may include a processor 2010 and a memory 2020 connected to the processor 2010. In addition, the processor 2010 may further include a cache area. According to an embodiment of the present disclosure, the memory 2020 may be implemented in the form of a high-bandwidth memory HBM, which is not limited. Specifically, according to an embodiment of the present disclosure, the processor 2010 is configured to run computer-executable instructions, which, when run by the processor 2010, implement the data loading method according to the embodiment of the present disclosure to load the tensor to be processed from the original tensor in the memory to the cache area, or implement the data storage method according to the embodiment of the present disclosure to obtain a second tensor based on the first tensor in the cache area and write the second tensor to the memory.

[0162] The processor 2010 can perform various actions and processes according to a program stored in a non-transitory memory. Specifically, the processor 2010 may refer to a processor chip capable of parallel computing, for example, a GPU (Graphics Processing Unit), a TPU (Tensor Processing Unit), an NPU (Neural Network Processing Unit), a DPU (Deep Learning Processing Unit), an APU (Accelerated Processing Unit), and a GPGPU (General-Purpose computing on Graphics Processing Unit). In addition, the processor 2010 may also be implemented as other conventional types of processors, which is not limited here.

[0163] Regarding the specific implementation process of the data storage method and the data loading method, please refer to the above description and will not be repeated here. The processor provided by at least one embodiment of the present disclosure can achieve similar technical effects as the aforementioned data loading method / data storage method, and the repeated parts will not be repeated here.

[0164] Figure 15 A block diagram of an example computing device implementing some embodiments of the present disclosure is shown. Figure 15 As shown, the computing device 3000 is suitable for implementing the data loading method or data storage method provided by the embodiment of the present disclosure. It should be noted that Figure 15The components of the computing device 3000 shown are merely exemplary and non-limiting. The computing device 3000 may also have other components according to actual application requirements.

[0165] like Figure 15 As shown, the computing device 3000 may include a processing device 3010 (eg, a central processing unit, a graphics processing unit, etc.), which may execute various appropriate actions and processes according to non-transitory computer-readable instructions stored in a memory to implement various functions.

[0166] For example, when the computer-readable instructions are executed by the processing device 3010, one or more steps of the data loading method described in any of the above embodiments, or one or more steps of the data storage method described in any of the above embodiments, can be executed. It should be noted that for a detailed description of the processing of the data loading method, reference can be made to the relevant description in the above embodiments of the data loading method, and for a detailed description of the processing of the data storage method, reference can be made to the relevant description in the above embodiments of the data storage method.

[0167] For example, the processing device 3010 , the read-only memory (ROM) 3020 , and the random access memory (RAM) 3030 are connected to each other via a bus 3040 . An input / output (I / O) interface 3050 is also connected to the bus 3040 .

[0168] For example, the memory may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) 3030 and / or cache memory. For example, computer-readable instructions may be loaded from storage device 3080 into RAM 3030 to execute the computer-readable instructions. Non-volatile memory may include, for example, read-only memory (ROM) 3020, a hard disk, erasable programmable read-only memory (EPROM), compact disc read-only memory (CD-ROM), flash memory, and the like. Various applications and data, as well as data used and / or generated by the applications, may also be stored in computer-readable storage media.

[0169] Typically, the following devices may be connected to the I / O interface 3050: an input device 3060 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 3070 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 3080 including, for example, a magnetic tape, a hard disk, a flash memory, etc.; and a communication device 3090. The communication device 3090 may allow the computing device 3000 to communicate with other electronic devices wirelessly or by wire to exchange data. Although Figure 15 A computing device 3000 is shown with various devices, but it should be understood that it is not required to implement or have all of the devices shown, and the computing device 3000 may alternatively implement or have more or fewer devices. For example, the processing device 3010 can control other components in the computing device 3000 to perform desired functions. The processing device 3010 can be a device with data processing capabilities and / or program execution capabilities, such as a central processing unit (CPU), a tensor processing unit (TPU), or a graphics processing unit (GPU). The GPU can be directly integrated into a system on a chip (SOC), directly integrated into a motherboard, or built into the motherboard's north bridge chip.

[0170] According to some embodiments of the present disclosure, a non-transitory computer-readable storage medium is also provided, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the data loading method according to the embodiment of the present disclosure is implemented, or the data storage method according to the embodiment of the present disclosure is implemented.

[0171] Figure 16 A schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. Figure 16 As shown, the computer-readable storage medium 4000 may be a non-transitory computer-readable storage medium, on which one or more computer-readable instructions 4010 may be non-transitory stored. For example, when the computer-readable instructions 4010 are executed by a processor, one or more steps of the data loading method described in any of the above embodiments, or one or more steps of the data storage method described in any of the above embodiments, may be executed. It should be noted that for a detailed description of the processing of the data loading method, reference may be made to the relevant description in the above embodiments of the data loading method, and for a detailed description of the processing of the data storage method, reference may be made to the relevant description in the above embodiments of the data storage method.

[0172] As an example, the storage medium 4000 can be applied to the electronic device 2000 and / or the computing device 3000. For example, the storage medium 4000 can be implemented as the storage device 3080 in the computing device 3000.

[0173] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0174] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit does not necessarily limit the unit itself.

[0175] The functions described above may be at least partially performed by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), Application Specific Standard Parts (ASSP), System on Chip (SOC), Complex Programmable Logic Device (CPLD), and the like.

[0176] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the present disclosure is not limited to technical solutions formed by specific combinations of the aforementioned technical features. It also encompasses other technical solutions formed by any combination of the aforementioned technical features or their equivalents, without departing from the scope of the above disclosure. For example, a technical solution formed by replacing the aforementioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0177] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0178] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

[0179] Regarding this disclosure, the following points need to be explained:

[0180] (1) The drawings of the embodiments of the present disclosure only relate to the structures related to the embodiments of the present disclosure. Other structures may refer to conventional designs.

[0181] (2) In the absence of conflict, the embodiments of the present disclosure and the features therein may be combined with each other to form new embodiments.

[0182] The above description is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. The protection scope of the present disclosure shall be based on the protection scope of the appended claims.

Claims

1. A data loading method for loading a tensor to be processed from an original tensor in memory into a buffer, wherein: The original tensor is a 5-dimensional tensor, and its shape and size are represented by 5 parameters b1, b2, b3, b4, and b5. b1, b2, b3, b4, and b5 respectively indicate the sizes of the original tensor in 5 dimensions and are all positive integers. The 5 dimensions include batch dimension, depth dimension, height dimension, width dimension, and channel number dimension. For the 5-dimensional tensor of the original tensor, the data determined by the height dimension and width dimension is represented as a pixel, and is accumulated step by step to higher dimensions. The channel number dimension does not count the number of pixels. The data loading method includes: Receiving, as input parameters of a data loading instruction, the number of pixels included in the tensor to be processed and the starting coordinates of the tensor to be processed in a coordinate system determined by the original tensor, wherein the starting coordinates are the coordinate values ​​of any tensor element in the original tensor that serves as a loading starting point in the coordinate system; For the data of the original tensor, starting from the starting coordinate, sequentially obtain data from the original tensor until the number of obtained data is equal to the number of pixels included in the tensor to be processed; and The acquired data is loaded into the cache area, wherein the acquired tensor to be processed is stored in the cache area in the form of a 5-dimensional tensor for data processing.

2. The method according to claim 1, wherein The memory stores multiple tensors, the original tensor is one of the multiple tensors, and the data storage format of the multiple tensors in the memory can be any one of NDHWC or N(C / x)DHW(xC). The data storage format is used to indicate the storage order and dimensional arrangement of the tensors in the storage component, wherein, for the data storage format of NDHWC, N corresponds to b1, indicating the batch dimension, D corresponds to b2, indicating the depth dimension, H corresponds to b3, indicating the height dimension, W corresponds to b4, indicating the width dimension, and C corresponds to b5, indicating the number of channels dimension; for the data storage format of N(C / x)DHW(xC), N corresponds to b1, indicating the batch dimension, (C / x) corresponds to b2, indicating the number of channels dimension, D corresponds to b3, indicating the depth dimension, H corresponds to b4, indicating the height dimension, and W(xC) corresponds to b5, indicating the width dimension. In the data storage format of N(C / x)DHW(xC), x is a positive integer, and the number of channels of xC is bound to the width dimension.

3. The method according to claim 1, wherein The tensor to be processed is used for a convolution operation in a computing unit within the processor.

4. The method according to claim 1, wherein The memory stores a plurality of tensors, and the data loading method further includes: Obtain indication information of the storage location of the original tensor in the memory, where the indication information includes the starting coordinates of the original tensor in the memory and size values ​​in five dimensions, which are represented as tensor_b1, tensor_b2, tensor_b3, tensor_b4, and tensor_b5, respectively.

5. The method according to claim 1, wherein The loading of the acquired data into the buffer area comprises: The acquired data is loaded from the memory into the cache area according to multiple instructions.

6. The method according to claim 5, wherein: The step of loading the acquired data into the buffer area further comprises: Determine a first data storage format of the tensor to be processed in the buffer area; and The tensor to be processed is placed in the cache area according to the first data storage format, wherein the first data storage format of the tensor to be processed in the cache area is the same as or different from the second data storage format of the original tensor in the memory.

7. The method according to claim 4, further comprising: Obtain boundary values ​​for the original tensor in at least a portion of the five dimensions, respectively, wherein the boundary values ​​are used to limit the boundary range of obtaining data from the original tensor, wherein the boundary values ​​are arbitrary values ​​compared to the size value of the original tensor in this dimension.

8. The method according to claim 7, wherein: Sequentially acquiring data from the original tensor includes: For a data portion of the original tensor whose range defined by the boundary value covers the original tensor, sequentially acquiring data from the range of the original tensor defined by the boundary value; and The data portion of the original tensor that is not covered by the range defined by the boundary value is represented as invalid data. The invalid data is directly filled into the tensor to be processed by the cache area according to a predetermined value, wherein the predetermined value is equal to 0.

9. The method according to claim 1, further comprising: Obtaining a data step value, wherein the data step value is used to specify a step size for sequentially obtaining data from the original tensor, wherein, for the data of the original tensor, starting from the starting coordinate, sequentially obtaining data from the original tensor includes: For the data of the original tensor, starting from the starting coordinate, data is sequentially obtained from the original tensor according to the data step value.

10. A data loading method, comprising: Receive a data loading instruction that instructs execution of loading a to-be-processed tensor from an original tensor in memory into a cache, wherein the original tensor is a 5-dimensional tensor whose shape and size are represented by 5 parameters b1, b2, b3, b4, and b5, b1, b2, b3, b4, and b5 respectively indicate the sizes of the original tensor in 5 dimensions and are all positive integers, and the 5 dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a channel number dimension; wherein, for the 5-dimensional tensor of the original tensor, the data determined by the height dimension and the width dimension is represented as a pixel, and is accumulated step by step toward higher dimensions, wherein the channel number dimension does not calculate the number of pixels, and the data loading instruction includes as input parameters the number of pixels included in the to-be-processed tensor and the starting coordinates of the to-be-processed tensor in a coordinate system determined by the original tensor, wherein the starting coordinates are the coordinate values ​​of any tensor element in the original tensor that serves as the loading starting point in the coordinate system; and After parsing the data load instruction, executing the data load instruction using an execution unit, The step of using the execution unit to execute the data loading instruction includes: For the data of the original tensor, starting from the starting coordinate, sequentially obtain data from the original tensor until the number of obtained data is equal to the number of pixels included in the tensor to be processed; and The acquired data is loaded into the cache area, wherein the acquired tensor to be processed is stored in the cache area in the form of a 5-dimensional tensor for data processing.

11. A data storage method for obtaining a second tensor based on a first tensor in a cache and writing the second tensor into a memory, wherein: The first tensor is a 5-dimensional tensor, and its shape and size are represented by 5 parameters b1, b2, b3, b4, and b5, where b1, b2, b3, b4, and b5 respectively indicate the sizes of the first tensor in 5 dimensions and are all positive integers. The 5 dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a channel number dimension. For the 5-dimensional tensor of the first tensor, the data determined by the height dimension and the width dimension is represented as a pixel, and is accumulated step by step toward higher dimensions. The channel number dimension does not count the number of pixels. The data storage method includes: Receiving, as input parameters of a data storage instruction, the number of pixels included in the second tensor and the starting coordinates of the second tensor in a coordinate system determined by the first tensor, wherein the starting coordinates are the coordinate values ​​of any tensor element in the first tensor that is used as a loading starting point in the coordinate system; For the data of the first tensor, starting from the starting coordinate, sequentially acquire data from the first tensor until the number of acquired data is equal to the number of pixels included in the second tensor; and The acquired data is stored in the memory, wherein the acquired second tensor is stored in the memory in the form of a 5-dimensional tensor for data processing.

12. The method according to claim 11, wherein The cache area stores multiple tensors, the first tensor is one of the multiple tensors, and the data storage format of the multiple tensors in the cache area can be any one of NDHWC or N(C / x)DHW(xC), and the data storage format is used to indicate the storage order and dimensional arrangement of the tensors in the storage component, wherein, for the data storage format of NDHWC, N corresponds to b1, indicating the batch dimension, D corresponds to b2, indicating the depth dimension, H corresponds to b3, indicating the height dimension, W corresponds to b4, indicating the width dimension, and C corresponds to b5, indicating the number of channels dimension; for the data storage format of N(C / x)DHW(xC), N corresponds to b1, indicating the batch dimension, (C / x) corresponds to b2, indicating the number of channels dimension, D corresponds to b3, indicating the depth dimension, H corresponds to b4, indicating the height dimension, and W(xC) corresponds to b5, indicating the width dimension. In the data storage format of N(C / x)DHW(xC), x is a positive integer, and the number of channels of xC is bound to the width dimension.

13. The method according to claim 11, wherein The buffer area stores a plurality of tensors, and the data storage method further includes: Obtain indication information of a storage location of the first tensor in the cache, where the indication information includes a starting coordinate of the first tensor in the cache and size values ​​in five dimensions, represented as tensor_b1, tensor_b2, tensor_b3, tensor_b4, and tensor_b5, respectively.

14. The method according to claim 11, wherein The storing the acquired data into the memory comprises: The acquired data is stored from the buffer area to the memory according to a plurality of instructions.

15. The method according to claim 14, wherein The storing the acquired data into the memory further comprises: Determine a first data storage format of the second tensor in the memory; The second tensor is placed in the memory according to the first data storage format, wherein the first data storage format of the second tensor in the memory is the same as or different from the second data storage format of the first tensor in the cache area.

16. The method according to claim 13, further comprising: Obtain boundary values ​​for the first tensor in at least a portion of the five dimensions, respectively, where the boundary values ​​are used to limit a boundary range for obtaining data from the first tensor, wherein the boundary values ​​are arbitrary values ​​compared to the size values ​​of the first tensor in the dimension.

17. The method according to claim 16, wherein Sequentially acquiring data from the first tensor includes: For a data portion of the first tensor whose range defined by the boundary value covers the data portion of the first tensor, sequentially acquiring data from the range of the first tensor defined by the boundary value; and The data portion of the first tensor that is not covered by the range defined by the boundary value is represented as invalid data, and the invalid data is not stored in the memory.

18. The method according to claim 11, further comprising: Obtaining a data step value, wherein the data step value is used to specify a step size for sequentially obtaining data from the first tensor, wherein, for the data of the first tensor, starting from the starting coordinate, sequentially obtaining data from the first tensor includes: For the data of the first tensor, starting from the starting coordinates, data is sequentially obtained from the first tensor according to the data step value.

19. A data storage method, comprising: Receive a data storage instruction that instructs execution of obtaining a second tensor based on a first tensor in a cache and writing the second tensor into a memory, wherein the first tensor is a 5-dimensional tensor, and its shape and size are represented by 5 parameters b1, b2, b3, b4, and b5, b1, b2, b3, b4, and b5 respectively indicate the sizes of the first tensor in 5 dimensions and are all positive integers, and the 5 dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a channel number dimension; wherein, for the 5-dimensional tensor of the first tensor, the data determined by the height dimension and the width dimension is represented as a pixel, and is accumulated step by step toward higher dimensions, wherein the channel number dimension does not calculate the number of pixels, and the data storage instruction includes as input parameters the number of pixels included in the second tensor and the starting coordinates of the second tensor in the coordinate system determined by the first tensor, wherein the starting coordinates are the coordinate values ​​of any tensor element in the first tensor that serves as the loading starting point in the coordinate system; and After parsing the data storage instruction, use the execution unit to execute the data storage instruction, The step of executing the data storage instruction using the execution unit includes: For the data of the first tensor, starting from the starting coordinate, sequentially acquire data from the first tensor until the number of acquired data is equal to the number of pixels included in the second tensor; and The acquired data is stored in the memory, wherein the acquired second tensor is stored in the memory in the form of a 5-dimensional tensor for data processing.

20. A processor comprising an instruction parsing unit and an execution unit, wherein: The instruction parsing unit is configured to: receive and parse a data loading instruction, wherein the data loading instruction instructs execution to load a to-be-processed tensor from an original tensor in a memory into a cache area, wherein the original tensor is a 5-dimensional tensor, and its shape and size are represented by 5 parameters b1, b2, b3, b4, and b5, b1, b2, b3, b4, and b5 respectively indicate the sizes of the original tensor in 5 dimensions and are all positive integers, and the 5 dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a channel number dimension; wherein, for the 5-dimensional tensor of the original tensor, the data determined by the height dimension and the width dimension is represented as a pixel, and is accumulated step by step toward higher dimensions, wherein the channel number dimension does not calculate the number of pixels, and the data loading instruction includes as input parameters the number of pixels included in the to-be-processed tensor and the starting coordinates of the to-be-processed tensor in a coordinate system determined by the original tensor, wherein the starting coordinates are the coordinate values ​​of any tensor element in the original tensor in the coordinate system; and The execution unit is configured to: execute the data loading instruction, The execution unit executes the data loading instruction, including: For the data of the original tensor, starting from the starting coordinate, sequentially obtain data from the original tensor until the number of obtained data is equal to the number of pixels included in the tensor to be processed; and The acquired data is loaded into the cache area, wherein the acquired tensor to be processed is stored in the cache area in the form of a 5-dimensional tensor for data processing.

21. A processor comprising an instruction parsing unit and an execution unit, wherein: The instruction parsing unit is configured to: receive and parse a data storage instruction, wherein the data storage instruction instructs execution to obtain a second tensor based on a first tensor in a cache and write the second tensor into a memory, wherein the first tensor is a 5-dimensional tensor, and its shape and size are represented by 5 parameters b1, b2, b3, b4, and b5, b1, b2, b3, b4, and b5 respectively indicate the sizes of the first tensor in 5 dimensions and are all positive integers, and the 5 dimensions include a batch dimension, a depth dimension, a height dimension, a width dimension, and a channel number dimension; wherein, for the 5-dimensional tensor of the first tensor, the data determined by the height dimension and the width dimension is represented as a pixel, and is accumulated step by step toward higher dimensions, wherein the channel number dimension does not calculate the number of pixels, and the data storage instruction includes the number of pixels included in the second tensor as input parameters, and the starting coordinates of the second tensor in the coordinate system determined by the first tensor, wherein the starting coordinates are the coordinate values ​​of any tensor element in the first tensor as the loading starting point in the coordinate system; and The execution unit is configured to: execute the data storage instruction, The execution unit executes the data storage instruction, including: For the data of the first tensor, starting from the starting coordinate, sequentially acquire data from the first tensor until the number of acquired data is equal to the number of pixels included in the second tensor; and The acquired data is stored in the memory, wherein the acquired second tensor is stored in the memory in the form of a 5-dimensional tensor for data processing.

22. An electronic device comprising a processor and a memory connected to the processor, wherein: The processor includes a cache area, wherein The processor is configured to run computer-executable instructions, which, when run by the processor, implement the data loading method according to any one of claims 1-10 to load the tensor to be processed from the original tensor in the memory to the cache area, or implement the data storage method according to any one of claims 11-19 to obtain the second tensor based on the first tensor in the cache area and write the second tensor to the memory.

23. A non-transitory computer-readable storage medium, wherein: The non-transitory computer-readable storage medium stores computer-executable instructions, When the computer executable instructions are executed by a processor, the data loading method according to any one of claims 1 to 10 is implemented, or the data storage method according to any one of claims 11 to 19 is implemented.

Images