Data processing method and device, electronic device, and storage medium

Abstract

A data processing method and device, electronic equipment and storage medium. The data processing method comprises: obtaining a first original tensor form of first to-be-processed data, the first to-be-processed data being stored in a storage unit; obtaining a mapping relationship between the first original tensor and a first target tensor, wherein the granularity of a single operation of the first original tensor is a plurality of tensor elements; calculating index information of the first target tensor according to the mapping relationship between the first target tensor and the first original tensor; and performing a transformation operation on the first original tensor according to the calculated index information of the first target tensor, so as to transform the first original tensor into the first target tensor. The data processing method can reduce the number of tensor transformations, reduce resource consumption, and improve system performance.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to a data processing method and apparatus, an electronic device, and a storage medium. BACKGROUND

[0002] In the process of neural network calculation, the provision and storage of input and output data of the calculation unit are various. The current artificial intelligence training framework generally uses the format of a tensor to store data. For example, in a deep learning framework such as pytorch, a high-dimensional matrix or vector can be created using the data layout format of a tensor, so as to achieve efficient calculation. In various neural network architectures, tensor transformation is often involved, such as transforming a 4-dimensional tensor into a 3-dimensional tensor, or performing a transpose operation on a 3-dimensional tensor, and the like. Since the tensor transformation is a memory operation, the performance is usually poor, and if multiple tensor transformations are involved, the performance of the system will be significantly reduced. SUMMARY

[0003] To solve the problem of performance reduction of the system caused by multiple tensor transformations, embodiments of the present disclosure provide a data processing method and apparatus, an electronic device, and a storage medium.

[0004] At least one embodiment of the present disclosure provides a data processing method, comprising: obtaining a first original tensor form of first to-be-processed data, the first to-be-processed data being stored in a storage unit; obtaining a mapping relationship between the first original tensor and a first target tensor, wherein the granularity of a single operation of the first original tensor is a plurality of tensor elements; calculating index information of the first target tensor according to the mapping relationship between the first target tensor and the first original tensor; and performing a transformation operation on the first original tensor according to the calculated index information of the first target tensor, so as to transform the first original tensor into the first target tensor.

[0005] For example, in the data processing method provided by at least one embodiment of the present disclosure, the plurality of tensor elements in the single operation are loaded or stored in the format of one-dimensional data, two-dimensional data, or three-dimensional data.

[0006] For example, in the data processing method provided by at least one embodiment of the present disclosure, the obtaining of the mapping relationship between the first original tensor and the first target tensor comprises: representing the first original tensor as an equivalent second original tensor, wherein the granularity of a single operation of the second original tensor is a single tensor element; obtaining a basic transformation operation in an equivalent process of representing the first original tensor as the second original tensor, a process of converting the second original tensor into a second target tensor, and an equivalent process of representing the second target tensor as the first target tensor; and obtaining the mapping relationship between the first target tensor and the first original tensor according to the basic transformation operation.

[0007] For example, in the data processing method provided by at least one embodiment of the present disclosure, the index information of the first target tensor is calculated according to the mapping relationship between the first target tensor and the first original tensor, comprising: obtaining the index information of the first original tensor, wherein the index information of the first original tensor comprises the dimension size, the storage step, the element coordinate and the storage address of the first original tensor; and calculating the coordinate of the first target tensor according to the mapping relationship between the first target tensor and the first original tensor and the index information of the first original tensor.

[0008] For example, in the data processing method provided by at least one embodiment of the present disclosure, the first original tensor is transformed into the first target tensor by performing a transformation operation on the first original tensor according to the calculated index information of the first target tensor, comprising: performing a coordinate conversion operation on the element coordinate of the first original tensor according to the calculated index information of the first target tensor, so as to transform the first original tensor into the first target tensor.

[0009] For example, the data processing method provided by at least one embodiment of the present disclosure further comprises: obtaining the form of a second original tensor of second to-be-processed data, wherein the second to-be-processed data is stored in the storage unit; obtaining the mapping relationship between the second original tensor and a second target tensor, wherein the granularity of single operation of the second original tensor is a single tensor element; calculating the index information of the second target tensor according to the mapping relationship between the second target tensor and the second original tensor; and performing the transformation operation on the second original tensor according to the calculated index information of the second target tensor, so as to transform the second original tensor into the second target tensor.

[0010] For example, in the data processing method provided by at least one embodiment of the present disclosure, the mapping relationship between the second original tensor and the second target tensor is obtained, comprising: obtaining a basic transformation operation in the process of converting the second original tensor into the second target tensor; and obtaining the mapping relationship between the second target tensor and the second original tensor according to the basic transformation operation.

[0011] For example, in the data processing method provided by at least one embodiment of the present disclosure, the index information of the second target tensor is calculated according to the mapping relationship between the second target tensor and the second original tensor, comprising: obtaining the index information of the second target tensor, wherein the index information of the second target tensor comprises the dimension size, the storage step, the element coordinate and the storage address of the second target tensor; and calculating the coordinate of the second target tensor according to the mapping relationship between the second target tensor and the second original tensor and the index information of the second target tensor.

[0012] For example, in the data processing method provided in at least one of the embodiments of the present disclosure, the performing, according to the calculated index information of the second target tensor, the transformation operation on the second original tensor to transform the second original tensor into the second target tensor comprises: performing a coordinate conversion operation once on the element coordinates of the second original tensor according to the calculated index information of the second target tensor to transform the second original tensor into the second target tensor.

[0013] For example, in the data processing method provided in at least one of the embodiments of the present disclosure, the basic transformation operation comprises dimension splitting, dimension merging, dimension rearrangement and addressing operation, wherein the dimension splitting is used to expand dimensions, the dimension merging is used to compress dimensions, the dimension rearrangement is used to exchange at least two dimensions, and the addressing operation is used to rewrite the address of data in a storage unit.

[0014] For example, the data processing method provided in at least one of the embodiments of the present disclosure further comprises: in response to the presence of the dimension rearrangement in the basic transformation operation, determining whether the addressing operation needs to be performed according to the basic transformation operation after the dimension rearrangement; and if the dimension splitting or the dimension merging exists after the dimension rearrangement, determining that the addressing operation needs to be performed so that the address is continuous for the dimension splitting or the dimension merging.

[0015] At least one of the embodiments of the present disclosure further provides a data processing apparatus, which comprises: a storage unit configured to store the first to-be-processed data; an acquisition unit configured to acquire a first original tensor form of the first to-be-processed data and a mapping relationship between the first original tensor and a first target tensor, wherein the granularity of a single operation of the first original tensor is a plurality of tensor elements; a calculation unit configured to calculate index information of the first target tensor according to the mapping relationship between the first target tensor and the first original tensor; and a transformation unit configured to perform a transformation operation on the first original tensor according to the calculated index information of the first target tensor to transform the first original tensor into the first target tensor.

[0016] For example, in the data processing apparatus provided in at least one of the embodiments of the present disclosure, the storage unit is further configured to store the first to-be-processed data in the form of the first target tensor.

[0017] For example, in the data processing apparatus provided by at least one embodiment of the present disclosure, the storage unit is further configured to store second to-be-processed data; the obtaining unit is further configured to obtain a second original tensor form of the second to-be-processed data and a mapping relationship between the second original tensor and a second target tensor, wherein a granularity of a single operation of the second original tensor is a single tensor element; the calculation unit is further configured to calculate index information of the second target tensor according to the mapping relationship between the second target tensor and the second original tensor; and the transformation unit is further configured to perform a transformation operation on the second original tensor according to the calculated index information of the second target tensor, so as to transform the second original tensor into the second target tensor.

[0018] For example, in the data processing apparatus provided by at least one embodiment of the present disclosure, the storage unit is further configured to store the second to-be-processed data in the form of the second target tensor.

[0019] An electronic device is provided by at least one embodiment of the present disclosure, including: a memory, which non-transiently stores computer executable instructions; and a processor, which is configured to run the computer executable instructions, wherein the computer executable instructions, when run by the processor, implement the data processing method in any embodiment of the present disclosure.

[0020] A non-transitory computer readable storage medium is provided by at least one embodiment of the present disclosure, which stores computer executable instructions, and the computer executable instructions, when executed by a processor, implement the data processing method in any embodiment of the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, but not limit the present disclosure.

[0022] Figure 1 A schematic diagram of exemplary tensor dimension merging and tensor dimension splitting is provided for at least one embodiment of the present disclosure;

[0023] Figure 2 A schematic diagram of exemplary tensor dimension rearrangement is provided for at least one embodiment of the present disclosure;

[0024] Figure 3 A schematic diagram of exemplary addressing operation is provided for at least one embodiment of the present disclosure;

[0025] Figure 4 A schematic diagram of a data processing method is provided for at least one embodiment of the present disclosure;

[0026] Figure 5a schematic diagram of a tensor transformation process;

[0027] Figure 6 a flow chart of a data processing method provided by at least one embodiment of the present disclosure;

[0028] Figure 7 a schematic diagram of a tensor transformation process provided by at least one embodiment of the present disclosure;

[0029] Figure 8 a schematic block diagram of a data processing apparatus provided by at least one embodiment of the present disclosure;

[0030] Figure 9 a schematic block diagram of an electronic device provided by at least one embodiment of the present disclosure.

[0031] Figure 10 a schematic diagram of a non-transitory computer-readable storage medium provided by at least one embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions of the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are some but not all of the embodiments of the present disclosure. Based on the described embodiments of the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative effort belong to the scope of the present disclosure.

[0033] Unless otherwise defined, technical terms or scientific terms used in the present disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which the present disclosure pertains. The terms “first”, “second” and similar terms used in the present disclosure do not denote any order, quantity or importance, but are used to distinguish different components. The terms “include” or “contain” and similar terms mean that the elements or objects before the terms encompass the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects. The terms “connect” or “connected” and similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “upper”, “lower”, “left”, “right” and the like are used only to represent relative positional relationships, and when the absolute positions of the described objects are changed, the relative positional relationships can also be changed accordingly.

[0034] The present disclosure will be described below through several specific embodiments. In order to keep the following description of the embodiments of the present disclosure clear and brief, the detailed description of known functions and known components can be omitted. When any component of the embodiments of the present disclosure appears in more than one drawing, the component is denoted by the same or similar reference numeral in each drawing.

[0035] In many applications of neural network computing, it is often required to obtain certain form of data for specific processing. These data to be processed can be image data, audio data, text data, or other data to be processed in application fields such as medical diagnosis, transportation, etc.

[0036] Tensor, as a basic data structure for storing a collection of numbers, has been widely used in current neural network, machine learning systems. For example, in a neural network, the data to be processed can be represented as a tensor, and then substituted into a mathematical formula for calculation. The mathematical formula can be visually represented as a computation graph, which has multiple operators (Op) for representing different mathematical operations, and the multiple operators can be connected by lines, which can represent the dependency relationship between the calculations of the operators.

[0037] Tensors can have different dimensions, such as zero-dimensional tensors, one-dimensional tensors (e.g., vectors), two-dimensional tensors (e.g., matrices), three-dimensional tensors, or even higher-dimensional tensors, so tensors are also called multidimensional arrays. The specific numerical values in a tensor can be application data, such as image data, natural language data, etc. For example, a specific numerical value in a two-dimensional tensor can represent the grayscale value of a pixel in the image data corresponding to a monochrome image.

[0038] For example, a zero-dimensional tensor can be understood as a scalar, such as the numerical values 2, 3, 5, etc. Zero-dimensional tensors can be mainly used for the representation of model loss and various measurement indicators, such as accuracy, precision, etc.

[0039] For example, a one-dimensional tensor can be understood as a vector. For example, a one-dimensional tensor with a shape of 3 includes the numerical values 0, 3, 20, and can be represented as [0, 3, 20].

[0040] For example, a two-dimensional tensor can be understood as a matrix. For example, a two-dimensional tensor with a shape of 2x2 includes the numerical values 2, 3, 1, 5, and can be represented as a 2x2 matrix [[2, 3], [1, 5]]. For example, as shown in Figure 1 For example, a three-dimensional tensor can be understood as a copy of multiple two-dimensional matrices, where the copy refers to the copy of the shape, not the numerical values, i.e., the copied matrices have the same shape, and the numerical values in the matrices can be different. For example, as shown in

[0041] For example, a three-dimensional tensor can be understood as a copy of multiple two-dimensional matrices, where the copy refers to the copy of the shape, not the numerical values, i.e., the copied matrices have the same shape, and the numerical values in the matrices can be different. For example, as shown inFigure 1 As shown, the tensor A is a three-dimensional tensor with shape 2x3x4, and the tensor A can be understood as a combination of 2 matrices with the same shape of 2 rows and 3 columns, for example, the two matrices respectively include the values 1, 2, 3, 4, 5, 6 and the values 7, 8, 9, 10, 11, 12, and the tensor A can be represented as [[[1, 2, 3], [4, 5, 6]], [[7, 8, 9], [10, 11, 12]]].

[0042] The tensor can also have more dimensions, such as a four-dimensional tensor, a five-dimensional tensor, a seven-dimensional tensor, and the like. As described above, each dimension of the tensor is increased by copying the part before the dimension is increased, so a four-dimensional tensor can be understood as copying the shape of a three-dimensional tensor in the fourth dimension one or more times. A four-dimensional tensor can be mainly used to represent image data, for example, a four-dimensional tensor has four dimensions B, C, H, W, and the sizes of each dimension are 2, 3, 4, and 4 respectively, so that the four-dimensional tensor with shape 2x3x4x4 can be understood as 2 pictures, each picture has RGB three colors, that is, has 3 channels, and each channel of the picture has 4 rows and 4 columns of pixel points, that is, the batch, channel, height, and width of the picture can correspond to the four dimensions B, C, H, W of the four-dimensional tensor respectively. The description of the four-dimensional tensor above is not repeated here.

[0043] The operation of tensor transformation can generalize the tensor to any dimension, for example, in the case of the same data, a high-dimensional tensor can be converted into a low-dimensional tensor, and a low-dimensional tensor can also be converted into a high-dimensional tensor, and the value size in the tensor will not change before and after the conversion.

[0044] Figure 1 Two basic tensor transformation operations are shown: dimension merging and dimension splitting.

[0045] Dimension merging can convert a high-dimensional tensor into a low-dimensional tensor, for example, convert a three-dimensional tensor A into a two-dimensional tensor B, or convert a three-dimensional tensor A into a one-dimensional tensor [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], etc. In pytorch, dimension merging can be implemented using the function view or squeeze (delete the dimension with size 1).

[0046] Dimension splitting can convert a low-dimensional tensor into a high-dimensional tensor, for example, convert a two-dimensional tensor B into a three-dimensional tensor A, or convert a one-dimensional tensor [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12] into a two-dimensional tensor B or a three-dimensional tensor A, etc. In pytorch, dimension splitting can be implemented using the function view or unsqueeze (adding a dimension with size 1 at the specified position).

[0047] Figure 2 Another basic tensor transformation operation is shown: dimension rearrangement. Dimension rearrangement can exchange the dimensions of a tensor, and in pytorch, dimension rearrangement operation can be implemented by functions transpose or permute. For example, the transpose function can transpose a tensor, i.e., exchange two dimensions of the tensor. For example, as shown in Figure 2 , the transpose function can rearrange the second dimension and the third dimension of tensor A, i.e., can transpose the tensor A with shape 2x2x3 into tensor C with shape 2x3x2. For example, the permute function can operate on several dimensions of a tensor at the same time, i.e., exchange multiple dimensions of the tensor, for example, permute(2, 0, 1) can transform the tensor with shape 1x2x3 into a tensor with shape 3x1x2.

[0048] Since a tensor is only a data structure for storing a collection of numbers, the data in the tensor will not change when the shape of the tensor is changed by the above basic transformation. The data in the tensor is actually stored in the storage unit of the chip. When the data is stored in the memory of the storage unit, it is generally stored continuously, i.e., the storage of the data in the memory has one-dimensionality, and the storage of the data in the memory is independent of any dimension of the tensor. For example, as shown in Figure 3 , the values 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 in tensor A are written in sequence in the memory (e.g., storage area 1) in a flat manner, and the values 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 in tensor A have addresses (e.g., hexadecimal) in storage area 1, for example, 00000001, 00000002, 00000003, 00000004, 00000005, 00000006, 00000007, 00000008, 00000009, 0000000a, 0000000b, 0000000c, respectively.

[0049] Data is written into memory according to the dimension order of the initial tensor when the data is created. For example, the tensor A has three dimensions of batch, row and column (for example, the dimension sizes of the three dimensions of batch, row and column are 2, 2 and 3 respectively), and the data is loaded according to the data structure of the tensor A, and the data can be written into the storage area 1 according to the following order: first, 3 columns of data 1, 2 and 3 in the first row of the first batch of the tensor A are written in sequence, then 3 columns of data 4, 5 and 6 in the second row of the first batch are written in sequence, after the data in the first batch is written into the storage area 1, the data in the second batch of the tensor A is written in the same order, that is, 3 columns of data 7, 8 and 9 in the first row of the second batch of the tensor A are written in sequence, and then 3 columns of data 10, 11 and 12 in the second row of the second batch are written in sequence. It can be understood that the data of the tensor A is arranged in the storage area 1 according to the dimension order from right to left, starting from the rightmost dimension (column) of the three dimensions of batch, row and column, and from the view of the tensor, the data of the tensor A is arranged in the storage area 1 along the row direction first.

[0050] When it is necessary to access the data in the tensor, an index can be used to locate the storage address of the data in the memory. For example, the index of a certain element of the tensor is calculated according to the dimension size of the tensor, so that the data value is read from the corresponding storage address according to the index. The index of each element of the tensor depends on the element coordinates and the storage step, and the index of each element satisfies the following formula:

[0051] index(E(e1,e2,…,en))=s1*e1+s2*e2+…+sn*en

[0052] Wherein, E(e1,e2,…,en) represents an element in the tensor view, e1, e2,…,en respectively represent the coordinate values of the element in n dimensions, and s1, s2,…,sn respectively represent the sizes of the storage steps of the n dimensions of the tensor.

[0053] The coordinates of an element refer to the position of the element in the tensor, and the position of the element in different dimensions of the tensor can be determined according to the coordinates of the element. The element coordinates are related to the shape of the tensor, and the coordinate values of the element usually start from 0, and if the size of a dimension of the tensor is n, the coordinate values of the element in the dimension range from 0 to n-1. For example, the tensor A has three dimensions of batch, row and column, and the sizes of the three dimensions of batch, row and column are 2, 2 and 3 respectively, and the coordinate values of the element in the three dimensions range from 0 to 1, 0 to 1 and 0 to 2 respectively.

[0054] For the convenience of description, the following will use the data value corresponding to the tensor element to represent the serial number of the tensor element when introducing the coordinate of the element, unless otherwise specified. For example, the 1st element of tensor A is located at the 1st column of the 1st row of the first batch, and the coordinate of the 1st element is (0, 0, 0); the 2nd element of tensor A is located at the 2nd column of the 1st row of the first batch, and the coordinate of the 2nd element is (0, 0, 1); the 3rd element of tensor A is located at the 3rd column of the 1st row of the first batch, and the coordinate of the 3rd element is (0, 0, 2); the 4th element of tensor A is located at the 1st column of the 2nd row of the first batch, and the coordinate of the 4th element is (0, 1, 0); the 5th element of tensor A is located at the 2nd column of the 2nd row of the first batch, and the coordinate of the 5th element is (0, 1, 1); the 6th element of tensor A is located at the 3rd column of the 2nd row of the first batch, and the coordinate of the 6th element is (0, 1, 2); the 7th element of tensor A is located at the 1st column of the 1st row of the second batch, and the coordinate of the 7th element is (1, 0, 0); the 8th element of tensor A is located at the 2nd column of the 1st row of the second batch, and the coordinate of the 8th element is (1, 0, 1); the 9th element of tensor A is located at the 3rd column of the 1st row of the second batch, and the coordinate of the 9th element is (1, 0, 2); the 10th element of tensor A is located at the 1st column of the 2nd row of the second batch, and the coordinate of the 10th element is (1, 1, 0); the 11th element of tensor A is located at the 2nd column of the 2nd row of the second batch, and the coordinate of the 11th element is (1, 1, 1); the 12th element of tensor A is located at the 3rd column of the 2nd row of the second batch, and the coordinate of the 12th element is (1, 1, 2). The 1st-12th elements of tensor A can be represented as E(0, 0, 0), E(0, 0, 1), E(0, 0, 2), E(0, 1, 0), E(0, 1, 1), E(0, 1, 2), E(1, 0, 0), E(1, 0, 1), E(1, 0, 2), E(1, 1, 0), E(1, 1, 1) and E(1, 1, 2) respectively.

[0055] The storage step can be understood as the number of elements in the storage area that need to be skipped when the index is increased by 1 in each dimension of the tensor. As described above, the data is written into the memory according to the dimension order of the initial tensor when it is created, for example, the data of the tensor A is arranged into the storage area 1 according to the dimension order of column, row, batch, therefore the storage step of each dimension is related to the above dimension order. The size of the storage step of each dimension depends on the dimension order, the dimension size of the previous dimension in the dimension order and the storage step of the previous dimension, usually the storage step of each dimension is equal to the dimension size of the previous dimension multiplied by the storage step of the previous dimension. Taking the tensor A as an example, the storage step of the “column” dimension of the tensor A is 1, the storage step of the “row” dimension is equal to the dimension size of the “column” dimension multiplied by the storage step of the “column” dimension, that is, the storage step of the “row” dimension is 3x1=3; the storage step of the “batch” dimension is equal to the dimension size of the “row” dimension multiplied by the storage step of the “row” dimension, that is, the storage step of the “batch” dimension is 2x3=6.

[0056] In combination Figure 2 The storage area 1 shown in combination with the tensor A and Figure 3 The storage step in the index can be more intuitively understood by the storage area 1 shown in combination with the tensor A. For example, in an example, the data can be read from the storage area 1 according to the index of the tensor A. For example, since only the “column” dimension changes from the 1st element to the 2nd element of the tensor A, the storage step of the “column” dimension is 1, therefore when the 2nd element is indexed, it is equivalent to jumping only once from the storage location (storage address is 00000001) of the 1st element to the storage location (storage address is 00000002) of the 2nd element. For example, since only the “row” dimension changes from the 1st element to the 4th element of the tensor A, the storage step of the “row” dimension is 3, therefore when the 4th element is indexed, it is equivalent to jumping 3 times from the storage location (storage address is 00000001) of the 1st element to the storage location (storage address is 00000004) of the 4th element. For example, since only the “batch” dimension changes from the 1st element to the 7th element of the tensor A, the storage step of the “batch” dimension is 6, therefore when the 7th element is indexed, it is equivalent to jumping 6 times from the storage location (storage address is 00000001) of the 1st element to the storage location (storage address is 00000007) of the 7th element.

[0057] Specifically, if the data of the 1st element of the tensor A is to be read, the index of the 1st element can be calculated according to the above formula. For example, the coordinates of the 1st element are (0, 0, 0), so e1 = 0, e2 = 0, e3 = 0, the storage steps of the batch, row and column dimensions of the tensor A are 6, 3 and 1 respectively, so s1 = 6, s2 = 3, s3 = 1, and the index of the 1st element can be determined as index(E(0, 0, 0)) = 0*6 + 0*3 + 0*1 = 0 according to the above formula. For example, if the data of the 8th element of the tensor A is to be read, since the coordinates of the 8th element are (1, 0, 1), so e1 = 1, e2 = 0, e3 = 1, the storage steps of the batch, row and column dimensions of the tensor A are 6, 3 and 1 respectively, so s1 = 6, s2 = 3, s3 = 1, and the index of the 8th element can be determined as index(E(1, 0, 1)) = 1*6 + 0*3 + 1*1 = 7 according to the above formula. Therefore, if the data corresponding to the 1st element (with a value of 1) is stored at the storage address 00000001 in the storage area 1, the data corresponding to the 8th element (with a value of 8) is stored at the storage address 00000008 in the storage area 1.

[0058] Of course, in some cases, the data of the first element of a tensor can not be stored from the starting position of a storage area, in which case a storage offset s0 should be added in the above formula. For example, if the 1st element of the tensor A is stored from the 10th position of the storage area 1, then the storage offset s0 = 10, the index of the 1st element is 10, and the index of the 8th element is 17, therefore, the storage address of the 1st element is 0000000b based on the initial address 00000001, and the storage address of the 8th element is 00000012 based on the initial address 00000001.

[0059] It should be noted that the two basic operations of dimension merging and dimension splitting have requirements for the continuity of the address, the dimension rearrangement operation has no requirement for the continuity of the address, but will destroy the continuity of the address. That is, dimension merging requires that all the dimensions to be merged are continuous in the memory address, dimension splitting requires that all the dimensions to be split are continuous in the memory address, and the memory address of all the dimensions after dimension merging and dimension splitting is unchanged. Dimension rearrangement does not require that all the dimensions to be rearranged are continuous in the memory address, but will destroy the original continuity of the dimensions in the memory address. That is, if the two dimensions to be rearranged are originally discontinuous in the memory address, the two dimensions can still be rearranged; if the two dimensions before rearrangement are originally continuous in the memory address, the two dimensions after rearrangement will become discontinuous in the memory address.

[0060] All dimensions before and after dimension merging and splitting are continuous in memory address. Figure 1 and Figure 3 As shown in the figure, the shape of tensor A is 2×2×3, with 12 elements. When performing dimension merging operation on tensor A, it can be understood as first arranging the data of the 12 elements of tensor A as follows Figure 3 The data is then sequentially filled into a new 3×4 tensor B in the form of a column of data in storage area 1. At this point, a new index can be given to each element of the new shape of tensor B, for example, still looping from the last dimension (the rightmost dimension) to the first dimension (the leftmost dimension), in ascending order within each dimension. That is, tensor B also has the same 12 data as tensor A, and when these 12 data of tensor B are arranged in storage area 1, they are also arranged along the row direction of tensor B. Although the element coordinates and storage stride of the 12 data of tensor B have changed, the indexes of the 12 data in tensor B remain unchanged. That is, the storage strides of the rows and columns of tensor B are 4 and 1 respectively, and the coordinates of the 12 elements of tensor B are (0,0), (0,1), (0,2), (0,3), (1,0), (1,1), (1,2), (1,3), (2,0), (2,1), (2,2), (2,3), (2,3), but the indexes of the 12 elements of tensor B are still 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, so the storage addresses of the 12 elements of tensor B in storage area 1 are still continuous and unchanged.

[0061] The dimensions after dimension reordering are not continuous in memory address. For example, Figure 2 and Figure 3As shown, the shape of tensor A is 2x2x3, with 12 elements. After the dimension rearrangement operation on the second and third dimensions of tensor A, each element is given a new index according to the new shape of tensor C. For example, the shape of tensor C is 2x3x2, still from the last dimension (the rightmost dimension) to the first dimension (the leftmost dimension) in each dimension, and sorted within each dimension, but since the shape of tensor C is different from that of tensor A, although tensor C also has the same 12 data as tensor A, the element coordinates and dimension order of the 12 data of tensor C have changed, so the order of the storage steps and the index of tensor C have also changed. For example, the storage steps of the batch, row, and column of tensor A are 6, 3, and 1, respectively, while the storage steps of the batch, row, and column of tensor C become 6, 2, and 1, respectively, and the coordinates of the 12 elements of tensor C have also changed, that is, the coordinates of the element (the first element) corresponding to data 1 of tensor C are (0, 0, 0), the coordinates of the element (the third element) corresponding to data 2 are (0, 1, 0), the coordinates of the element (the fifth element) corresponding to data 3 are (0, 2, 0), the coordinates of the element (the second element) corresponding to data 4 are (0, 0, 1), the coordinates of the element (the fourth element) corresponding to data 5 are (0, 1, 1), the coordinates of the element (the sixth element) corresponding to data 6 are (0, 2, 1), the coordinates of the element (the seventh element) corresponding to data 7 are (1, 0, 0), the coordinates of the element (the ninth element) corresponding to data 8 are (1, 1, 0), the coordinates of the element (the eleventh element) corresponding to data 9 are (1, 2, 0), the coordinates of the element (the eighth element) corresponding to data 10 are (1, 0, 1), the coordinates of the element (the tenth element) corresponding to data 11 are (1, 1, 1), and the coordinates of the element (the twelfth element) corresponding to data 12 are (1, 2, 1).

[0062] Therefore, after the dimension reordering, the dimensional order of the second and third dimensions of tensor C has changed, and the storage steps of the three dimensions of tensor C have also changed to 6, 1, and 3. Since the coordinates of the 1st to 12th elements of tensor C are (0,0,0), (0,0,1), (0,0,2), (0,1,0), (0,1,1), (0,1,2), (1,0,0), (1,0,1), (1,0,2), (1,1,0), (1,1,1), and (1,1,2), Therefore, the calculated indices of the 1st to 12th elements of tensor C in storage area 1 are: 0, 3, 1, 4, 2, 5, 6, 9, 7, 10, 8, 11, and the storage addresses are: 00000001, 00000004, 00000002, 00000005, 00000003, 00000006, 00000007, 0000000a, 00000008, 0000000b, 00000009, 0000000c. In other words, the storage addresses of the 12 elements of tensor C have changed in storage area 1 and are no longer continuous.

[0063] Since the dimensions of tensor C are not continuous in storage area 1, if we continue to merge the second and third dimensions of tensor C, the requirement of dimension merging for address continuity will not be met. Therefore, we need to perform addressing operations first to make the dimensions of tensor C continuous in memory address. For example, Figure 3 As shown, the addressing operation is performed on the tensor C, and the data of the tensor C is written into the new storage area 2 according to the dimension order of the tensor C and the storage stride corresponding to the dimension, so that the elements of the tensor C are continuous in the storage area 2. That is, the storage stride of the tensor C is changed from 6, 1, 3 after the dimension reordering to 6, 2, 1 (determined as 6, 2, 1 according to the shape of the tensor C 2×3×2), so that the indexes of the 1st to 12th elements of the tensor C (corresponding to data values: 1, 4, 2, 5, 3, 6, 7, 10, 8, 11, 9, 12) in the storage area 2 are: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, the storage addresses of the 1st to 12th elements of tensor C in storage area 2 are: 00000001, 00000002, 00000003, 00000004, 00000005, 00000006, 00000007, 00000008, 00000009, 0000000a, 0000000b, 0000000c. After the addressing operation, the three dimensions of tensor C are continuous in memory address, meeting the address continuity requirement. Therefore, tensor C can be merged or split into dimensions.

[0064] Currently, when loading or storing a tensor, a data layout format with a single operation granularity of a single tensor element is usually adopted, which is referred to as a regular format tensor (or a strided tensor) in the present disclosure. For a regular format tensor, a user can specify a coordinate on each dimension of the tensor, and then access data of the coordinate from a corresponding memory. As described above, the index of the element can be determined using the storage stride and the element coordinate of the regular format tensor, so as to calculate the storage address of the data corresponding to the element, that is, the storage address of the data is the multiplication accumulation of the element coordinate and the storage stride.

[0065] When processing data, transformation of a tensor is often involved. As shown in Figure 4 , if a tensor processed by an operator OP0 is regarded as an original tensor, and an object of an operator OP1 is a target tensor, then the original tensor needs to be transformed into the target tensor by single or multiple transformations. Figure 4 As shown in FIG. 2, the second original tensor and the second target tensor are both regular format tensors. In actual operation, for example, in an example, the second original tensor of three dimensions needs to be transformed into a four-dimensional tensor, then the two dimensions of the four-dimensional tensor are transposed, and finally the dimension with a size of 1 in the four-dimensional tensor is removed to obtain the second target tensor, which undergoes multiple transformations in the process.

[0066] The transformation process of the regular format tensor from the original tensor to the target tensor will be described in detail below. Figure 5

[0067] Figure 5 FIG. 1 is a schematic diagram of a conventional tensor transformation process. In order to facilitate understanding and description, the dimension size, storage stride, element coordinate, and address continuity of the tensor and other information are arranged in a table as shown in Figure 5 , in which the information of the first dimension, the second dimension, …, and the Nth dimension of the tensor is represented from top to bottom, and the dimension framed by a dashed circle in the table is the object of the next transformation operation.

[0068] For example, the sizes of the four dimensions of the original tensor D are B, C, H, and W, the storage strides of the four dimensions are CHW, HW, W, and 1 (as shown in the table, the storage stride of each dimension is equal to the size of the dimension below it multiplied by the storage stride), and the element coordinates of the four dimensions are represented by P b , P c , P h , and P w , respectively, with the value range of P b being 0-B-1, the value range of P c being 0-C-1, the value range of P h being 0-H-1, and the value range of P w ​The value range is 0 to W-1.

[0069] Similarly, the sizes of the three dimensions of the target tensor K are BHW / 49 (assuming BHW is divisible by 49), 49, and C. The storage steps of these three dimensions are 49C, C, and 1 respectively. The element coordinates of these three dimensions are represented by P b’ 、P h7w7 、P c Indicates that P b’ The value range is 0~BHW / 49-1, P h7w7 The value range is 0 to 48, P c The value range is 0 to C-1.

[0070] like Figure 5 As shown in the figure, if you want to convert the four-dimensional original tensor D into the three-dimensional target tensor K, you need to go through multiple tensor transformation operations. There are many ways to transform the original tensor D into the target tensor K. The type, number and order of transformation operations used in each transformation method can be different. Figure 5 The transformation method shown is used as an example to explain in detail how to transform the original tensor into the target tensor in the traditional method.

[0071] For example, through the dimension merging (first transformation) operation, the third dimension (H) and the fourth dimension (W) of the tensor D are merged, thereby transforming into a tensor E, and the size HW of the third dimension of the tensor E is equal to the product of the size H of the third dimension of the tensor D and the size W of the fourth dimension. The storage steps of the three dimensions of the tensor E become CHW, HW, 1, and the coordinate range of the third dimension of the tensor E becomes P hw , so the coordinates of the elements of tensor E also change accordingly. The specific changes in the coordinates of the elements can be found in the above description and will not be repeated here.

[0072] For example, considering that the size of the first dimension of the target tensor K (BHW / 49) is related to the size of the third dimension (H) and the size of the fourth dimension (W) of the original tensor D, a dimension rearrangement (second transformation) operation is performed on the second and third dimensions of tensor E, so that HW and B are adjacent. After the dimension rearrangement, the three dimensions of tensor F are discontinuous in the memory address. It should be pointed out that the second dimension HW of tensor F is obtained by merging H and W, so the address continuity is still maintained in the second dimension. The coordinate range of tensor F has not changed, and the element coordinates have not changed, but because the order of the dimensions has changed, the order of the storage steps of the dimensions has also changed. That is, the storage steps of the three dimensions of tensor F are CHW, 1, and HW, respectively.

[0073] For example, a dimension splitting (the third transformation) operation is performed on the tensor F next, so as to split the second dimension HW of the tensor F into two dimensions, and a tensor G is obtained. The storage steps of the four dimensions of the tensor G become CHW, W, 1, and HW, and the coordinate ranges of the four dimensions are P b , P h , P w , P c .

[0074] For example, considering that the size of the second dimension of the target tensor K is 49, a dimension splitting (the fourth transformation) operation can be performed on the tensor G, so as to split the second dimension and the third dimension of the tensor G into two dimensions respectively, and the size of one of the two dimensions is 7. That is, the sizes of the six dimensions of the tensor G are B, H / 7, 7, W / 7, 7, and C respectively, the storage steps are CHW, 7W, W, 7, 1, and HW respectively, and the coordinates are P b , P h / 7 , P h7 , P w / 7 , P w7 , P c .

[0075] For example, in order to combine the two dimensions with the size of 7 into a dimension with the size of 49, a dimension rearrangement (the fifth transformation) operation needs to be performed first, so as to rearrange the dimension order of the tensor H into the dimension order of the tensor I, so as to facilitate subsequent operations. It needs to be noted that since the dimension rearrangement does not need to consider the address continuity, even if each dimension in the tensor H is not continuous in the memory address, the dimension rearrangement operation can also be performed.

[0076] For example, the combination operation can be performed on the two dimensions with the size of 7 after the two dimensions are adjacent, but since the two dimensions are not continuous in the memory address at this time, an addressing operation (the sixth transformation) needs to be performed first, so as to make the six dimensions of the tensor J continuous in the memory address. The related description of the addressing operation can be referred to the foregoing description, and will not be described herein again.

[0077] For example, a dimension combination (the seventh transformation) operation is performed finally, so as to combine the first dimension, the second dimension, and the third dimension of the tensor J into the first dimension of the target tensor K, and combine the fourth dimension and the fifth dimension of the tensor J into the second dimension of the target tensor K, and thus the final target tensor K is obtained.

[0078] It needs to be noted that the type, number, and order of the transformation operations from the original tensor D to the target tensor K can be selected according to actual needs. For example, in another example, the first transformation, the second transformation, and the third transformation in the foregoing transformation process can also be replaced by a dimension rearrangement operation, that is, the tensor D can also be converted into the tensor K by only five transformation operations.

[0079] For the above transformation method, on the one hand, depending on the type of transformation operation, explicit memory data transfer may be required for tensors. Since the time overhead of explicit memory data transfer increases with the number of transformations, if multiple tensor transformations are involved, the system performance will be significantly reduced. On the other hand, this transformation method is even less cost-effective for block-format tensors, which have higher performance. Block-format tensors must first undergo one or more transformations to regular-format tensors before the above transformation operations can be applied. The additional conversion between block-format tensors and regular-format tensors results in greater overhead.

[0080] Block format tensors are a data layout format that is different from conventional format tensors. Block format tensors refer to tensors that are loaded or stored with a single operation granularity of multiple tensor elements. A single operation here refers to loading or storing data once. For example, a block format tensor can load or store multiple tensor elements at once in a single operation, that is, a block format tensor can load or store data in a "block" manner, which can significantly reduce the number of times data is loaded or stored, reduce the number of times data is moved in memory, and improve system performance.

[0081] Since block format tensors and regular format tensors have different memory layouts, block format tensors cannot be directly transformed in the same way as regular format tensors. Currently, it is usually necessary to convert block format tensors to regular format tensors first. Figure 4 As shown, to achieve the conversion from a first original tensor in a block format layout to a first target tensor in a block format layout, the first original tensor in a block format layout needs to be converted to a first original tensor in a regular format layout through an additional transformation operation 41, then multiple transformations are performed according to the transformation scheme of the regular format tensor to obtain a second target tensor, and finally, the second target tensor in the regular format layout is converted to the first target tensor in a block format layout through another additional transformation operation 42. Therefore, in addition to the original multiple transformation operations, the two additional transformation operations 41 and 42 make the overall performance of the block format tensor worse.

[0082] At least one embodiment of the present disclosure provides a data processing method and apparatus, an electronic device and a storage medium. The data processing method comprises: obtaining a first original tensor form of first to-be-processed data, the first to-be-processed data being stored in a storage unit; obtaining a mapping relationship between the first original tensor and a first target tensor, wherein the granularity of a single operation of the first original tensor is a plurality of tensor elements; calculating index information of the first target tensor according to the mapping relationship between the first target tensor and the first original tensor; and performing a transformation operation on the first original tensor according to the calculated index information of the first target tensor, so as to transform the first original tensor into the first target tensor.

[0083] The data processing method and apparatus provided by the above-mentioned embodiments of the present disclosure calculate the index information of the target tensor by obtaining the mapping relationship between the original tensor and the target tensor, so as to convert the original tensor into the target tensor by one coordinate transformation operation according to the index information. Compared with the scheme of multiple tensor transformations, the data processing method and apparatus can significantly reduce the number of transformations, improve the data processing efficiency, and improve the system performance. In at least one embodiment, the data processing method and apparatus can also avoid additional conversion between tensors of two layout formats, thereby reducing the additional data moving overhead.

[0084] Figure 6 A flowchart of a data processing method provided by at least one embodiment of the present disclosure is shown in FIG. 1. As shown in FIG. 1, the data processing method provided by at least one embodiment of the present disclosure comprises steps S100-S400. Figure 6

[0085] Step S100: Obtain a first original tensor form of first to-be-processed data, the first to-be-processed data being stored in a storage unit.

[0086] Step S200: Obtain a mapping relationship between the first original tensor and a first target tensor, wherein the granularity of a single operation of the first original tensor is a plurality of tensor elements.

[0087] Step S300: Calculate index information of the first target tensor according to the mapping relationship between the first target tensor and the first original tensor.

[0088] Step S400: Perform a transformation operation on the first original tensor according to the calculated index information of the first target tensor, so as to transform the first original tensor into the first target tensor.

[0089] For example, the first to-be-processed data can be image data, audio data, text data or other data that needs to be processed in application fields such as medical diagnosis and transportation, and the embodiments of the present disclosure do not limit this.

[0090] ​For example, in step S100, the first to-be-processed data can be acquired from the storage unit, and a first original tensor form of the first to-be-processed data is obtained. For example, the first to-be-processed data 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, and 12 can be acquired from the storage area 1 as shown in FIG. 1, and a tensor A (or a tensor B, or other original tensors) form of the first to-be-processed data is obtained. Figure 3

[0091] For example, the granularity of the single-time operation of the first original tensor is a plurality of tensor elements, such as 2 elements, 3 elements, 4 elements, 8 elements, or a larger number of elements. That is, the first original tensor is a block format tensor, and a plurality of tensor elements can be loaded or stored at a time in a single-time operation. The first original tensor can be loaded or stored in a "block" manner, so that the number of loading or storing times can be significantly reduced, and the system performance can be improved.

[0092] For example, in at least one embodiment of the present disclosure, the plurality of tensor elements in the single-time operation are loaded or stored in a one-dimensional data, two-dimensional data, or three-dimensional data format. For example, the one-dimensional data format can be understood as being stored in the storage unit at a time in a column or a row data unit, and the two-dimensional data format and the three-dimensional data format can be understood as being stored in the storage unit at a time in a "surface" or a "block" unit. For example, the format of the loaded data or the stored data can be one-dimensional, two-dimensional, or three-dimensional according to actual needs.

[0093] For example, in step S200, the mapping relationship between the first original tensor and the first target tensor can be acquired by acquiring the types and the number of the transformation operations that need to be performed in the conventional transformation mode described above. It should be noted that, although the related information of the transformation operations that can be used needs to be acquired in step S200, these transformation operations are not actually performed, but the transformation operations that need to be experienced in the mapping process from the first original tensor to the first target tensor are acquired (or demonstrated) in form.

[0094] When the basic transformation operation is selected in the demonstration process, it needs to be noted that the dimension merging and the dimension splitting should meet the requirement of the address continuity, and the influence of the dimension rearrangement operation on the address continuity needs to be noted.

[0095] For example, in at least one embodiment of the present disclosure, the data processing method further includes: in response to the dimension rearrangement existing in the basic transformation operation, determining whether the addressing operation needs to be performed according to the basic transformation operation after the dimension rearrangement; if the dimension splitting or the dimension merging exists after the dimension rearrangement, it is determined that the addressing operation needs to be performed, so that the address is continuous for the dimension splitting or the dimension merging. The specific description of this step can be referred to the related description in the foregoing, and will not be described here.

[0096] Figure 7 ​A schematic diagram of a tensor transformation process provided in at least one embodiment of the present disclosure is provided. Figure 7 Index information of the first original tensor and the first target tensor, such as dimension size, storage stride, element coordinates, and storage address continuity, is shown.

[0097] For example, in at least one example of the embodiments of the present disclosure, the above-mentioned step S200, i.e., obtaining the mapping relationship between the first original tensor and the first target tensor, includes: expressing the first original tensor as an equivalent second original tensor, wherein the granularity of a single operation of the second original tensor is a single tensor element; obtaining basic transformation operations in the equivalent process of expressing the first original tensor as the second original tensor, the process of converting the second original tensor into the second target tensor, and the process of expressing the second target tensor as the equivalent process of the first target tensor; and obtaining the mapping relationship between the first target tensor and the first original tensor according to the basic transformation operations.

[0098] For example, in at least one example of the embodiments of the present disclosure, basic transformation operations include dimension splitting, dimension merging, dimension rearrangement, and addressing operations. For example, dimension splitting is used to expand a dimension, dimension merging is used to compress a dimension, dimension rearrangement is used to exchange at least two dimensions, and addressing operations are used to rewrite the address of data in a storage unit. Detailed descriptions of basic transformation operations such as dimension splitting, dimension merging, dimension rearrangement, and addressing operations can be found in the relevant descriptions of the embodiments above and are not repeated here.

[0099] For example, Figure 7 As shown, the first original tensor and the first target tensor are block format tensors. For example, the single operation granularity of the first original tensor is 128. For ease of understanding, the first original tensor can be expressed as a high-dimensional tensor form of a conventional format tensor. For example, the first original tensor can be expressed as a tensor with a single operation granularity of 1 in 7 dimensions, and its data volume is the same as the data volume of the second original tensor (for example, tensor D) with a conventional format tensor. However, for the access operation of tensor D, it is necessary to loop B×C×H×W times, while for the first original tensor, it only needs to be executed B×C / 4×H / 4×W / 8 times, that is, compared with the tensor D with a conventional format, the first original tensor with a block format only requires 1 / 128 operations, which can greatly reduce the number of operations and improve system performance.

[0100] For example, in the above example, for a more complex mapping relationship, the first original tensor can be first equivalently expressed as a second original tensor with a simpler form and a regular format. After obtaining the mapping relationship between the second original tensor and the second target tensor, the second target tensor can be equivalently expressed as the first target tensor. The demonstration process of the mapping relationship is simplified by obtaining the basic transformation operations in each process step by step.

[0101] For example, as shown in Figure 7 , the first original tensor can be represented as an equivalent second original tensor (e.g., tensor D), and the basic transformation operations involved in the equivalent process are obtained, such as the dimension rearrangement, addressing operation and dimension merging involved in the view conversion from the first original tensor to tensor D. Then, the basic transformation operations in the conversion process from the second original tensor to the second target tensor are obtained, such as Figure 7 , the dashed arrow between tensor D and tensor K indicates that the basic transformation operations in the conversion process from tensor D to tensor K are obtained, and the basic transformation operations involved in the process can refer to the conversion process as shown in Figure 5 , as described above, the conversion process can involve seven basic transformation operations (or five basic transformation operations). Finally, as shown in Figure 7 , the tensor K is represented as an equivalent first target tensor, and the basic transformation operations involved in the equivalent process are obtained, such as the dimension split and dimension rearrangement operations involved in the view conversion from tensor K to the first target tensor.

[0102] For example, as shown in Figure 7 , the mapping relationship between the first original tensor and the first target tensor can be composed of the basic transformation operations represented by the dashed arrows in the dashed box, which include the change relationship of the dimension size, storage step, element coordinate and storage address of the tensor (e.g., which can be represented as a mapping formula), so that after obtaining the mapping relationship between the first original tensor and the first target tensor, step S300 can be executed, that is, the index information of the first target tensor can be calculated according to the mapping relationship between the first target tensor and the first original tensor.

[0103] For example, in at least one example of the embodiments of the present disclosure, the above step S300, i.e., calculating the index information of the first target tensor according to the mapping relationship between the first target tensor and the first original tensor, includes: obtaining the index information of the first original tensor, the index information of the first original tensor including the dimension size, storage step, element coordinate and storage address of the first original tensor; calculating the coordinate of the first target tensor according to the mapping relationship between the first target tensor and the first original tensor and the index information of the first original tensor.

[0104] For example, as shown in Figure 7 , the equivalent representation tensor of the first original tensor has 7 dimensions, and the sizes of the 7 dimensions are B, C / 4, H / 4, W / 8, 4, 4 and 8, respectively, and the storage steps are CHW, 4WH, 16W, 128, 32, 8 and 1, respectively. The element coordinate of each element is the combination of the coordinates of the dimensions where each element is located, and the coordinates of each dimension are P b , P c’ , P h’ , P w’、P c4 、P h4 、P w8 , the value ranges of these coordinates are 0~B-1, 0~C / 4-1, 0~H / 4-1, 0~W / 8-1, 0~3, 0~3, and 0~8, respectively. Based on the above index information of the first original tensor and the change relationship between the dimension size, storage stride, element coordinates, and storage address included in the basic transformation operation described above, the index information of the first target tensor can be calculated. For example, the above index information of the first original tensor is substituted into the mapping formula described above to calculate the element coordinates of the first target tensor and the new storage address corresponding to the data.

[0105] For example, in at least one example of the embodiments of the present disclosure, the above-mentioned step S400, that is, performing a transformation operation on the first original tensor according to the index information of the calculated first target tensor to transform the first original tensor into the first target tensor, includes: performing a coordinate conversion operation on the element coordinates of the first original tensor according to the index information of the calculated first target tensor to transform the first original tensor into the first target tensor.

[0106] like Figure 7 As shown, the data processing method provided by at least one embodiment of the present disclosure can calculate a mapping formula through the mapping relationship between the first original tensor and the first target tensor, thereby directly calculating the index information of the first target tensor based on the mapping formula and the index information of the first original tensor. Therefore, in actual hardware operations, the conversion from the first original tensor to the first target tensor only requires one coordinate conversion operation, that is, multiple transformation operations in the traditional transformation method are integrated into one transformation operation, thereby significantly reducing the number of transformations, reducing data movement overhead, and thereby improving data processing efficiency and enhancing system performance.

[0107] The data processing method provided by at least one embodiment of the present disclosure can also be used for conventional format tensors with a single operation granularity of a single tensor element, or a conventional format tensor can be viewed as a low-dimensional representation of a block format tensor. Similarly, the data processing method of the present disclosure can merge multiple transformation operations in conventional format tensor transformations into a single transformation operation, thereby significantly reducing the number of transformations and data movement overhead, thereby improving data processing efficiency and enhancing system performance.

[0108] For example, the data processing method provided by at least one embodiment of the present disclosure further includes steps S500 to S800.

[0109] Step S500: obtaining the second original tensor form of the second data to be processed, the second data to be processed being stored in a storage unit.

[0110] Step S600: Obtain a mapping relationship between the second original tensor and the second target tensor, wherein a granularity of a single operation of the second original tensor is a single tensor element.

[0111] Step S700: Calculate index information of the second target tensor according to the mapping relationship between the second target tensor and the second original tensor.

[0112] Step S800: Perform a transformation operation on the second original tensor according to the calculated index information of the second target tensor, so as to transform the second original tensor into the second target tensor.

[0113] For example, in at least one of the examples of the embodiments of the present disclosure, the step S600, i.e., obtaining the mapping relationship between the second original tensor and the second target tensor, comprises: obtaining a basic transformation operation in the process of converting the second original tensor into the second target tensor; and obtaining the mapping relationship between the second target tensor and the second original tensor according to the basic transformation operation.

[0114] For example, in at least one of the examples of the embodiments of the present disclosure, the step S700, i.e., calculating the index information of the second target tensor according to the mapping relationship between the second target tensor and the second original tensor, comprises: obtaining the index information of the second target tensor, the index information of the second target tensor comprising a dimension size, a storage step, an element coordinate and a storage address of the second target tensor; and calculating a coordinate of the second target tensor according to the mapping relationship between the second target tensor and the second original tensor and the index information of the second target tensor.

[0115] For example, in at least one of the examples of the embodiments of the present disclosure, the step S800, i.e., performing the transformation operation on the second original tensor according to the calculated index information of the second target tensor, so as to transform the second original tensor into the second target tensor, comprises: performing a coordinate conversion operation on the element coordinate of the second original tensor according to the calculated index information of the second target tensor, so as to transform the second original tensor into the second target tensor.

[0116] The detailed description of the above steps can refer to the steps S100-S400 in the foregoing, which will not be described here.

[0117] The present disclosure at least one embodiment further provides a data processing apparatus 800. Figure 8 As shown in the schematic block diagram of the data processing apparatus provided by at least one embodiment of the present disclosure, Figure 8 The data processing apparatus 800 comprises a storage unit 801, an obtaining unit 802, a calculating unit 803 and a transforming unit 804.

[0118] For example, the storage unit 801 is configured to store the first data to be processed; the acquisition unit 802 is configured to obtain the first original tensor form of the first data to be processed and the mapping relationship between the first original tensor and the first target tensor, wherein the granularity of a single operation of the first original tensor is multiple tensor elements; the calculation unit 803 is configured to calculate the index information of the first target tensor based on the mapping relationship between the first target tensor and the first original tensor; the transformation unit 804 is configured to perform a transformation operation on the first original tensor based on the calculated index information of the first target tensor to transform the first original tensor into the first target tensor.

[0119] For example, in the data processing device provided in at least one embodiment of the present disclosure, the storage unit 801 is further configured to store the first data to be processed in the form of a first target tensor, for example, the first data to be processed with a rewritten address is stored in a storage area of ​​the storage unit 801.

[0120] For example, in the data processing device provided by at least one embodiment of the present disclosure, the storage unit 801 is also configured to store the second data to be processed; the acquisition unit 802 is also configured to obtain the second original tensor form of the second data to be processed and the mapping relationship between the second original tensor and the second target tensor, wherein the granularity of a single operation of the second original tensor is a single tensor element; the calculation unit 803 is also configured to calculate the index information of the second target tensor based on the mapping relationship between the second target tensor and the second original tensor; the transformation unit 804 is also configured to perform a transformation operation on the second original tensor based on the calculated index information of the second target tensor to transform the second original tensor into a second target tensor.

[0121] For example, in the data processing device provided in at least one embodiment of the present disclosure, the storage unit 801 is further configured to store the second data to be processed in the form of a second target tensor, for example, the second data to be processed with the rewritten address is stored in a storage area of ​​the storage unit 801.

[0122] At least one embodiment of the present disclosure further provides an electronic device, Figure 9 A schematic block diagram of an electronic device provided in accordance with at least one embodiment of the present disclosure.

[0123] For example, Figure 9 As shown, the electronic device includes a processor 1001, a communication interface 1002, a memory 1003, and a communication bus 1004. The processor 1001, the communication interface 1002, and the memory 1003 communicate with each other via the communication bus 1004. The processor 1001, the communication interface 1002, the memory 1003, and other components can also communicate with each other via a network connection.

[0124] For example, the memory 1003 is configured to store computer-executable instructions non-transitorily. When the processor 1001 executes the computer-executable instructions, the computer-executable instructions are implemented to realize the data processing method according to any of the above embodiments. For the specific implementation of each step of the data processing method and related explanations, please refer to the above description, which will not be repeated here.

[0125] For example, the processor 1001 can control other components in the electronic device to perform desired functions. The processor 1001 can be a central processing unit (CPU), a network processing unit (NP), etc., and can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic component, a discrete hardware component. The central processing unit (CPU) can be X86 or ARM architecture, etc. For example, the processor 1001 executes the program stored on the memory 1003 to realize the implementation of the data processing method, which is the same as the above embodiment, and will not be repeated here.

[0126] For example, the communication bus 1004 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is shown in the figure, but it does not mean that there is only one bus or only one type of bus. For example, the communication interface 1002 is configured to realize the communication between the electronic device and other devices.

[0127] For example, the memory 1003 can include one or more computer program products in any combination, which can include various forms of computer readable storage media, such as volatile memory and / or non-volatile memory. For example, the volatile memory can include random access memory (RAM), cache memory, etc. The non-volatile memory can include read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer-executable instructions can be stored on the computer-readable storage medium, and the processor 1001 can execute the computer-executable instructions to realize various functions of the electronic device. Various application programs and various data, etc. can also be stored in the storage medium. For the detailed description of the process of the electronic device executing the data processing method, please refer to the related description in the above embodiment of the data processing method, and the repeated parts will not be repeated here.

[0128] Figure 10 A schematic diagram of a non-transitory computer-readable storage medium provided by at least one embodiment of the present disclosure is shown. For example, as shown in Figure 10As shown, one or more computer-executable instructions 1101 can be non-transitorily stored in the storage medium 1100. For example, the computer-executable instructions 1101 can perform one or more steps of the data processing method according to the above description when executed by a processor.

[0129] For example, the storage medium 1100 can be applied in the electronic device 800 described above. For example, the storage medium 1100 can include the memory 1003 in the electronic device. The description of the storage medium 1100 can refer to the description of the memory 1003 in the embodiments of the electronic device, and the repeated parts will not be described herein.

[0130] Although the present disclosure has been described in detail with general description and specific embodiments, some modifications or improvements can be made to the present disclosure on the basis of the embodiments of the present disclosure, which is obvious to those skilled in the art. Therefore, these modifications or improvements made on the basis of the spirit of the present disclosure are within the scope of the present disclosure.

[0131] For the present disclosure, the following points need to be explained:

[0132] (1) The drawings of the embodiments of the present disclosure only involve the structures involved in the embodiments of the present disclosure, and other structures can refer to the general design.

[0133] (2) For the purpose of clarity, the thickness of the layer or region is exaggerated or reduced in the drawings used to describe the embodiments of the present disclosure, that is, the drawings are not drawn according to the actual proportion.

[0134] (3) In the case of no conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0135] The above is only a specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto, and the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A data processing method, comprising: Acquire a first original tensor form of first data to be processed, where the first data to be processed is stored in a storage unit; Obtaining a mapping relationship between the first original tensor and the first target tensor, wherein a granularity of a single operation of the first original tensor is a plurality of tensor elements; Obtaining index information of the first original tensor, wherein the index information of the first original tensor includes a dimension size, a storage stride, element coordinates, and a storage address of the first original tensor; Calculate the index information of the first target tensor according to the mapping relationship between the first target tensor and the first original tensor and the index information of the first original tensor; A transformation operation is performed on the first original tensor according to the calculated index information of the first target tensor to transform the first original tensor into the first target tensor.

2. The data processing method according to claim 1, wherein: The plurality of tensor elements in the single operation are loaded or stored in a format of one-dimensional data, two-dimensional data, or three-dimensional data.

3. The data processing method according to claim 1, wherein: The obtaining a mapping relationship between the first original tensor and the first target tensor includes: Representing the first raw tensor as an equivalent second raw tensor, wherein the granularity of a single operation on the second raw tensor is a single tensor element; Obtaining basic transformation operations in an equivalent process of expressing the first original tensor as the second original tensor, a process of converting the second original tensor into a second target tensor, and an equivalent process of expressing the second target tensor as the first target tensor; A mapping relationship between the first target tensor and the first original tensor is obtained according to the basic transformation operation.

4. The data processing method according to claim 1, wherein: The performing a transformation operation on the first original tensor according to the calculated index information of the first target tensor to transform the first original tensor into the first target tensor includes: A coordinate conversion operation is performed on the element coordinates of the first original tensor according to the calculated index information of the first target tensor to transform the first original tensor into the first target tensor.

5. The data processing method according to claim 1, further comprising: obtaining a second original tensor form of second data to be processed, where the second data to be processed is stored in the storage unit; Obtaining a mapping relationship between the second original tensor and the second target tensor, wherein the granularity of a single operation of the second original tensor is a single tensor element; Calculate index information of the second target tensor according to a mapping relationship between the second target tensor and the second original tensor; The transformation operation is performed on the second original tensor according to the calculated index information of the second target tensor to transform the second original tensor into the second target tensor.

6. The data processing method according to claim 5, wherein: The obtaining a mapping relationship between the second original tensor and the second target tensor includes: Obtaining a basic transformation operation in a process of converting the second original tensor into the second target tensor; A mapping relationship between the second target tensor and the second original tensor is obtained according to the basic transformation operation.

7. The data processing method according to claim 5 or 6, wherein: The calculating the index information of the second target tensor according to the mapping relationship between the second target tensor and the second original tensor includes: Obtain index information of the second target tensor, where the index information of the second target tensor includes a dimension size, a storage step size, an element coordinate, and a storage address of the second target tensor; The coordinates of the second target tensor are calculated according to a mapping relationship between the second target tensor and the second original tensor and index information of the second target tensor.

8. The data processing method according to claim 7, wherein: The performing the transformation operation on the second original tensor according to the calculated index information of the second target tensor to transform the second original tensor into the second target tensor includes: A coordinate conversion operation is performed on the element coordinates of the second original tensor according to the calculated index information of the second target tensor to transform the second original tensor into the second target tensor.

9. The data processing method according to claim 3 or 6, wherein: The basic transformation operations include dimension splitting, dimension merging, dimension rearrangement and addressing operations. The dimension splitting is used to expand the dimension, the dimension merging is used to compress the dimension, the dimension rearrangement is used to exchange at least two dimensions, and the addressing operation is used to rewrite the address of data in the storage unit.

10. The data processing method according to claim 9, further comprising: In response to the dimension rearrangement in the basic transformation operation, determining whether the addressing operation needs to be performed according to the basic transformation operation after the dimension rearrangement; If the dimension splitting or the dimension merging exists after the dimension rearrangement, it is determined that the addressing operation needs to be performed so that the addresses are continuous for the dimension splitting or the dimension merging.

11. A data processing device comprising: a storage unit configured to store first data to be processed; an acquiring unit, configured to acquire a first original tensor form of the first to-be-processed data; Obtaining a mapping relationship between the first original tensor and the first target tensor, wherein the granularity of a single operation of the first original tensor is multiple tensor elements; obtaining index information of the first original tensor, wherein the index information of the first original tensor includes a dimension size, a storage stride, element coordinates, and a storage address of the first original tensor; a computing unit configured to calculate index information of the first target tensor according to a mapping relationship between the first target tensor and the first original tensor and the index information of the first original tensor; A transformation unit is configured to perform a transformation operation on the first original tensor according to the calculated index information of the first target tensor, so as to transform the first original tensor into the first target tensor.

12. The data processing apparatus according to claim 11, wherein: The storage unit is further configured to store the first to-be-processed data in the form of the first target tensor.

13. The data processing apparatus according to claim 12, wherein: The storage unit is further configured to store second data to be processed; The acquisition unit is further configured to acquire a second original tensor form of the second to-be-processed data and a mapping relationship between the second original tensor and the second target tensor, wherein the granularity of a single operation of the second original tensor is a single tensor element; The computing unit is further configured to calculate index information of the second target tensor according to a mapping relationship between the second target tensor and the second original tensor; The transformation unit is further configured to perform a transformation operation on the second original tensor according to the calculated index information of the second target tensor to transform the second original tensor into the second target tensor.

14. The data processing apparatus according to claim 13, wherein: The storage unit is further configured to store the second to-be-processed data in the form of the second target tensor.

15. An electronic device comprising: a memory that non-transitorily stores computer-executable instructions; a processor configured to execute the computer-executable instructions, Wherein, when the computer executable instructions are executed by the processor, the data processing method according to any one of claims 1-10 is implemented.

16. A non-transitory computer-readable storage medium, wherein: The non-transitory computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the data processing method according to any one of claims 1 to 10 is implemented.

Images