Data conversion method and apparatus, electronic device and storage medium

By transposing the first and third dimensions of data combinations in the neural network accelerator, the problem of difficulty in converting data layout formats between different neural network frameworks is solved, efficient data conversion is achieved, and equipment burden is reduced.

WO2024230730A1PCT designated stage expired Publication Date: 2024-11-14BEIJING YOUZHUJU NETWORK TECH CO LTD +1
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2024/091702
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-09
Filing Date
2024-05-08
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

Existing neural network accelerators are difficult to support the conversion between data layout formats between different neural network frameworks, resulting in a large delay in data conversion and increasing the burden on the device.

Method used

Provide a data conversion method and device, by reading elements in the data combination, performing transposition of the first dimension and the third dimension, and writing it to memory, supporting data arrangement format conversion between different neural network frameworks , reduce the burden on accelerators and other equipment.

Benefits of technology

It reduces the delay in data conversion and reduces the burden on devices such as accelerators, achieving efficient conversion between data layout formats between different neural network frameworks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024091702_14112024_PF_FP_ABST
    Figure CN2024091702_14112024_PF_FP_ABST
Patent Text Reader

Abstract

A data conversion method and apparatus, an electronic device and a storage medium, for use in converting the dimension of a first data combination. The first data combination comprises at least one batch, and dimensions of data corresponding to each batch comprise a first dimension, a second dimension and a third dimension. The data conversion method comprises: reading n elements in a first data combination according to a first dimension direction to obtain a first processing group, wherein a first element to an n-th element in the first processing group are arranged in the first dimension direction, and n is a positive integer; transposing a first dimension and a third dimension of the first processing group to obtain a second processing group, wherein a first element to an n-th element in the second processing group are arranged in a third dimension direction; and writing the first element to the n-th element in the second processing group into a first memory. According to the data conversion method, the delay of data conversion is reduced, and the device burden is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Data conversion method and device, electronic device and storage medium

[0001] This disclosure claims priority to Chinese patent application No. 202310514473.2 filed on May 9, 2023, and the contents of the above-mentioned Chinese patent application disclosure are hereby incorporated by reference in their entirety as part of this disclosure. Technical Field

[0002] Embodiments of the present disclosure relate to a data conversion method, a data conversion device, an electronic device, and a storage medium. Background Art

[0003] The field of deep learning in artificial intelligence (AI) involves a large number of operations such as convolutional neural networks (CNNs) and matrix multiplication. In current computing devices, data processing integrated circuits such as coprocessors and accelerators can execute programs to perform various AI computing functions. An accelerator for deep learning can be a configurable fixed-function hardware accelerator that can be used for training and inference operations in deep learning applications. It provides complete hardware acceleration for convolutional neural networks by exposing the operations associated with each convolutional layer (e.g., convolution, pooling, full connection, activation, aggregation, local response normalization, etc.).

[0004] Summary of the Invention

[0005] At least one embodiment of the present disclosure provides a data conversion method for converting the dimensions of a first data combination, wherein the first data combination includes at least one batch, and the dimensions of data corresponding to each batch in the at least one batch include a first dimension, a second dimension, and a third dimension. The data conversion method includes: reading n elements in the first data combination along the first dimension direction to obtain a first processing group, wherein the first to nth elements in the first processing group are arranged along the first dimension direction, and n is a positive integer; performing transposition on the first dimension and the third dimension of the first processing group to obtain a second processing group, wherein the first to nth elements in the second processing group are arranged along the third dimension direction; and writing the first to nth elements in the second processing group into a first memory.

[0006] At least one embodiment of the present disclosure further provides a data conversion device for converting the dimensions of a first data combination, wherein the first data combination includes at least one batch, and the dimensions of the data corresponding to each batch in the at least one batch include a first dimension, a second dimension, and a third dimension. The data conversion device includes: a reading module, configured to read n elements in the first data combination in the direction of the first dimension to obtain a first processing group, wherein the first element to the nth element in the first processing group are arranged in the direction of the first dimension, and n is a positive integer; a transposition module, configured to perform transposition on the first dimension and the third dimension of the first processing group to obtain a second processing group, wherein the first element to the nth element in the second processing group are arranged in the direction of the third dimension; and a writing module, configured to write the first element to the nth element in the second processing group into a first memory.

[0007] At least one embodiment of the present disclosure further provides a data conversion device. The data conversion device includes: a processor; a memory including one or more computer program modules; wherein the one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules are used to implement the data conversion method provided in any embodiment of the present disclosure.

[0008] At least one embodiment of the present disclosure further provides an electronic device comprising: the data conversion device provided in any embodiment of the present disclosure; and a data processing device configured to process the first data combination.

[0009] At least one embodiment of the present disclosure further provides a storage medium storing non-transitory computer-readable instructions, which, when executed by a computer, implements the data conversion method provided by any embodiment of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, rather than limiting the present disclosure.

[0011] FIG1 is a schematic diagram of an example of dimensions of a data combination;

[0012] FIG2 is a schematic diagram of a data conversion device;

[0013] FIG3 is an exemplary flow chart of a data conversion method provided by at least one embodiment of the present disclosure;

[0014] FIG4 is a schematic diagram of an example of a data conversion method provided by at least one embodiment of the present disclosure;

[0015] FIG5 is an exemplary flow chart of an example of step S120 in FIG3 ;

[0016] FIG6 is a schematic diagram of another example of a data conversion method provided by at least one embodiment of the present disclosure;

[0017] FIG7 is a schematic block diagram of a data conversion device provided by at least one embodiment of the present disclosure;

[0018] FIG8 is a schematic block diagram of another data conversion device provided by at least one embodiment of the present disclosure;

[0019] FIG9 is a schematic block diagram of an electronic device provided by at least one embodiment of the present disclosure;

[0020] FIG10 is a schematic block diagram of another electronic device provided by at least one embodiment of the present disclosure; and

[0021] FIG11 is a schematic diagram of a storage medium provided by at least one embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] To make the purpose, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0023] Unless otherwise defined, the technical or scientific terms used in this disclosure should have the usual meanings understood by people with ordinary skills in the field to which this disclosure belongs. The words "first", "second" and similar words used in this disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one", "an" or "the" do not indicate a quantity limitation, but rather indicate the existence of at least one. Words such as "include" or "comprise" mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0024] The present disclosure is described below using several specific embodiments. To keep the following description of the embodiments of the present disclosure clear and concise, detailed descriptions of known functions and components may be omitted. When any component of an embodiment of the present disclosure appears in more than one drawing, the component is represented by the same or similar reference numeral in each drawing.

[0025] For operations like convolutional neural networks and matrix multiplication and addition, existing processors, such as coprocessors and accelerators, implement different hardware structures in different ways, resulting in different requirements for data layout formats for each hardware structure. For example, for convolutional neural network operations, the data combination to be processed has multiple dimensions, and the data combination includes at least one batch, and the data corresponding to each batch further includes multiple dimensions. For A×B matrix multiplication operations (both the A matrix and the B matrix are two-dimensional data combinations), the data layout format in memory is typically row-major (height × width), while the ideal data layout format for hardware calculations is row-major (height × width) for the A matrix and column-major (width × height) for the B matrix.

[0026] FIG1 is a schematic diagram of an example of dimensions of a data combination.

[0027] For example, as shown in Figure 1, a data combination has multiple dimensions [N, C, H, W]. The data combination includes at least one batch, where N represents the number of batches. The data corresponding to each batch includes multiple channels, where C represents the number of channels. For example, in each channel, multiple data elements are arranged two-dimensionally to form an array. The vertical height of the data in each channel is H, which represents the number of data elements in the array along the column direction. The horizontal width of the data in each channel is W, which represents the number of data elements in the array along the row direction.

[0028] For example, the data combination in Figure 1 includes 2 batches, that is, the number of batches N=2; each batch includes 16 channels, that is, the number of channels in each batch C=16; the number of data elements in the data element array along the column direction in each channel is 5, that is, the height H=5; the number of data elements in the data element array along the row direction in each channel is 4, that is, the width W=4.

[0029] For example, data combinations in different neural network frameworks have different data arrangement formats, that is, when processing data combinations, the order of reading and writing memory is different according to the dimensions of the data combinations. For example, data arrangement formats include NHWC, NCHW, and some private data arrangement formats in accelerators (such as NCHWC'). For example, when using some neural network accelerators to process data combinations, it is necessary to use the data conversion module in the neural network accelerator to convert the data arrangement format.

[0030] FIG2 is a schematic diagram of a data conversion device.

[0031] For example, as shown in FIG2 , the data conversion device 110 includes a buffer 113 and direct memory access (DMA) controllers for reading and writing, including a read DMA 111 and a write DMA 112. For example, the data conversion device 110 may be a module for data format conversion in a neural network accelerator, and the data conversion device 110 is connected to other data processing devices in the accelerator via a bus interface 120. For example, the bus interface 120 may be an Advanced eXtensible Interface (AXI) bus protocol interface, or an AXI Coherency Extensions (ACE) bus protocol interface.

[0032] For example, the read DMA 111 reads part of the data to be processed from the memory into the buffer 113 via the bus interface 120. After the data is rearranged in the buffer 113, the write DMA 112 writes the rearranged data to the memory via the bus interface 120. For example, the buffer 113 may be composed of multiple static random-access memories (SRAMs).

[0033] For example, the data conversion device in the neural network accelerator only supports the conversion between a specific data arrangement format (such as NCHW) in a specific neural network framework (such as the Pytorch framework) and a private data arrangement format (such as NCHWC'), and it is difficult to support the conversion between the data arrangement format (such as NHWC) in other neural network frameworks (such as the TensorFlow framework). Therefore, for the conversion between different data arrangement formats corresponding to different neural network frameworks (for example, the conversion between NCHW and NHWC, etc.), it is usually necessary to perform the corresponding data conversion task in the processing equipment such as the accelerator, which leads to a large delay in data conversion and increases the burden on the accelerator and other equipment.

[0034] At least one embodiment of the present disclosure provides a data conversion method for converting the dimensions of a first data combination. The first data combination includes at least one batch, and the dimensions of the data corresponding to each batch include a first dimension, a second dimension, and a third dimension. The data conversion method includes: reading n elements in the first data combination in the direction of the first dimension to obtain a first processing group, wherein the 1st element to the nth element in the first processing group are arranged in the direction of the first dimension, and n is a positive integer; performing a transposition on the first dimension and the third dimension of the first processing group to obtain a second processing group, wherein the 1st element to the nth element in the second processing group are arranged in the direction of the third dimension; and writing the 1st element to the nth element in the second processing group into a first memory.

[0035] At least one embodiment of the present disclosure further provides at least one data conversion device, electronic device, and storage medium.

[0036] The methods, apparatuses, devices, and storage media provided by at least one embodiment of the present disclosure support conversion between different data arrangement formats corresponding to different neural network frameworks, and can offload corresponding data conversion tasks from processing devices such as accelerators, thereby reducing data conversion delays and alleviating the burden on devices including accelerators.

[0037] Hereinafter, at least one embodiment of the present disclosure will be described in detail with reference to the accompanying drawings. It should be noted that the same reference numerals in different drawings will be used to refer to the same elements described above.

[0038] FIG3 is an exemplary flowchart of a data conversion method provided by at least one embodiment of the present disclosure.

[0039] For example, as shown in Figure 3, at least one embodiment of the present disclosure provides a data conversion method for converting the dimensions of a first data set. For example, the first data set includes at least one batch, and the dimensions of the data corresponding to each batch include a first dimension, a second dimension, and a third dimension. For example, the data conversion method includes the following steps S110 to S130.

[0040] Step S110: reading n elements in the first data combination along the first dimension direction to obtain a first processing group, where n is a positive integer;

[0041] Step S120: performing transposition on the first dimension and the third dimension of the first processing group to obtain a second processing group;

[0042] Step S130: writing the 1st to nth elements in the second processing group into the first memory.

[0043] For example, the first dimension, the second dimension, and the third dimension are different from each other; in some examples, the first dimension can be the number of channels, the second dimension can be the height, and the third dimension can be the width. For example, the first data combination includes N batches, and each batch corresponds to a plurality of channels of data, and the first dimension is the number of channels C; for example, in each channel, a plurality of elements are arranged in two dimensions to form an array (i.e., an H×W plane), the second dimension is the height H of the data in each channel in the vertical dimension (i.e., the number of elements in the array along the column direction), and the third dimension is the width W of the data in each channel in the horizontal dimension (i.e., the number of elements in the array along the row direction).

[0044] It should be noted that the first dimension, the second dimension and the third dimension are not limited to the above description, and other selections can also be made according to the specific format of the first data combination and actual needs. The embodiments of the present disclosure do not limit this.

[0045] In some examples, multiple elements in the first data set are stored in the first memory in a first order, where the first order is a priority arrangement order in the third dimension, the second dimension, the first dimension, and the batch direction. For example, when the first dimension is the number of channels C, the second dimension is the height H, and the third dimension is the width W, the data arrangement format of the multiple elements in the first data set in the first memory is NCHW.

[0046] For example, for the first data combination arranged as NCHW in the first order, in the first memory, starting from the starting element, the elements in the row where the starting element is located are arranged in sequence along the W direction, and then move to the next row along the H direction to arrange the elements of the row in sequence along the W direction... After the elements in the H×W plane where the starting element is located are arranged, if multiple elements in the first data combination have not been arranged, move to the next H×W plane along the C direction to arrange the multiple elements in the H×W plane in sequence in the aforementioned manner... After all elements in the batch N where the starting reading position is located are arranged, if multiple elements in the first data combination have not been arranged, move to the next batch along the N direction to arrange the multiple elements in the batch in sequence in the aforementioned manner.

[0047] It should be noted that the arrangement of the first data combination in the first memory is not limited to the above description, and other options can be made according to actual needs. The embodiments of the present disclosure do not limit this.

[0048] For example, in step S110, when reading n elements in the first data combination along the first dimension, the starting reading position (e.g., the position of the starting element) is first determined, and then the n elements are read sequentially along the first dimension (e.g., direction C) starting from the starting reading position. For example, the n elements read along the first dimension can be used as the first processing group, and the first element to the nth element in the first processing group are arranged along the first dimension. For example, the first processing group can include only the first dimension, the second dimension, and the third dimension, where the first dimension can be the number of channels C, the second dimension can be the height H, and the third dimension can be the width W.

[0049] In some examples, the first dimension of the first processing group may be the number n of elements contained in the first processing group, the second dimension may be 1, and the third dimension may be the number m of sub-elements contained in each element, where m is a positive integer; that is, the first dimension of each element in the first processing group may be 1, the second dimension of each element in the first processing group may be 1, and the third dimension of each element in the first processing group may be m. In some examples, m=n.

[0050] For example, in step S120, when the first dimension of the first processing group is n and the third dimension W is the number of sub-elements contained in each element, after transposing the first dimension and the third dimension of the first processing group, the first dimension of the second processing group obtained is the number m of sub-elements contained in each element, and the third dimension is the number n of elements contained in the second processing group, that is, the 1st element to the nth element in the second processing group are arranged in the direction of the third dimension, and the first dimension and the third dimension of each element in the second processing group are also transposed compared with each element in the first processing group.

[0051] In some examples, the first dimension of the second processing group can be the number m of sub-elements contained in each element, the second dimension can be 1, and the third dimension can be the number n of elements contained in the second processing group; that is, the first dimension of each element in the second processing group is m, the second dimension of each element in the second processing group is 1, and the third dimension of each element in the second processing group is 1. In some examples, m=n.

[0052] For example, in step S130, n elements in the second processing group are written to corresponding storage locations in the first memory. In some examples, the storage locations of the n elements in the first memory are the same as the arrangement locations of the n elements in the second processing group.

[0053] In some examples, the data conversion method may further include step S140:

[0054] Step S140 : reading the first data combination multiple times to obtain multiple first processing groups.

[0055] For example, in step S140, the first data combination can be read multiple times in the manner of step S110, with n elements read each time, thereby obtaining multiple first processing groups, wherein each first processing group includes n elements arranged in the direction of the first dimension; in some examples, after reading and obtaining the first first processing group, the second first processing group can be read in the first order, and then the third first processing group can be read in the first order... until all first data combinations are read. For example, further, the first and third dimensions can be transposed on each of the multiple first processing groups in the manner of step S120 to obtain multiple corresponding second processing groups, wherein each second processing group includes n elements arranged in the direction of the third dimension.

[0056] In some examples, the data conversion method may further include step S150:

[0057] Step S150: writing the plurality of second processing groups into the first memory to obtain a second data combination.

[0058] For example, in step S150, multiple second processing groups can be written into the first memory multiple times in the manner of step S130 to obtain a second data combination. For example, when the first data combination is read multiple times in the manner of step S110, if all elements in the first data combination have been read, that is, the multiple first processing groups include all elements in the first data combination, then the multiple second processing groups obtained by transposing in the manner of step S120 include transposed elements of all elements in the first data combination; therefore, after the multiple second processing groups are written into the first memory, the number of elements included in the obtained second data combination is the same as the number of elements included in the first data combination.

[0059] In some examples, the multiple elements in the second data set are stored in the first memory in a second order. The second order can be a priority arrangement order in the first dimension, the third dimension, the second dimension, and the batch direction. For example, when the first dimension is the number of channels C, the second dimension is the height H, and the third dimension is the width W, the data arrangement format of the multiple elements in the second data set in the first memory is NHWC.

[0060] For example, for the second data combination arranged as NHWC in the second order, in the first memory, starting from the starting element, the elements in the row where the starting element is located are arranged in sequence along the C direction, and then move to the next row along the W direction to arrange the elements in the row in sequence along the C direction... After the C×W plane where the starting write position is located is filled, if multiple elements in the second data combination have not been arranged, move to the next C×W plane along the H direction to continue to arrange the multiple elements in the C×W plane in sequence in the aforementioned manner... After all elements in the batch N where the starting write position is located are arranged, if multiple elements in the second data combination have not been arranged, move to the next batch along the N direction to arrange the multiple elements in the batch in sequence in the aforementioned manner.

[0061] It should be noted that the arrangement of the second data combination in the first memory is not limited to the above description, and other options can be made according to actual needs. The embodiments of the present disclosure are not limited to this.

[0062] In some examples, the data arrangement format of the first data combination is NCHW, and after the conversion in steps S110 to S150, the data arrangement format of the second data combination obtained is NHWC; in other examples, the data arrangement format of the first data combination and the data arrangement format of the second data combination obtained after the conversion can also be other types, which can be selected according to actual needs, and the embodiments of the present disclosure are not limited to this.

[0063] For example, the data conversion method shown in FIG3 can be implemented by the data conversion device 110 shown in FIG2 . For example, a first data group to be read can be stored in a first memory (e.g., a memory). In step S110 , the read DMA 111 reads n elements of the first data group from the first memory into the buffer 113 via the bus interface 120 to obtain a first processing group. In step S120 , the second memory (e.g., the buffer 113 in FIG2 ) transposes the first and third dimensions of the first processing group to obtain a second processing group. In step S130 , the write DMA 112 writes the first to nth elements of the second processing group into the first memory via the bus interface 120.

[0064] For example, further, the data conversion device 110 shown in Figure 2 can also implement steps S140 and S150; in step S140, the read DMA 111 can read the first data combination stored in the first memory multiple times, and read n elements in the first data combination from the first memory into the buffer 113 multiple times to obtain multiple first processing groups; in step S150, the write DMA 112 writes multiple second processing groups into the first memory, thereby obtaining the converted second data combination.

[0065] It should be noted that the first memory is a memory, and the second memory is a buffer (for example, a ping-pang buffer, which includes two cache units, each used for alternating reading and writing). The first memory and the second memory can also be selected from other types according to actual conditions, and the embodiments of the present disclosure are not limited to this.

[0066] It should be noted that in addition to using the data conversion device 110 shown in Figure 2, other types of devices or equipment can also be used to execute the data conversion method provided by at least one embodiment of the present disclosure. The specific selection can be made according to actual needs, and the embodiments of the present disclosure do not limit this.

[0067] FIG4 is a schematic diagram of an example of a data conversion method provided by at least one embodiment of the present disclosure. For example, FIG4 is a specific example of the data conversion method in FIG3.

[0068] For example, the first data combination is shown in the lower left corner of Figure 4. The first data combination to be read is stored in a first memory (e.g., a memory); the number of batches included in the first data combination is N, and the data corresponding to each batch includes multiple channels, and the first dimension is the number of channels C; in each channel, multiple elements are arranged in two dimensions to form an array (i.e., an H×W plane), the second dimension is the height H of the data in each channel in the vertical dimension (i.e., the number of elements in the array along the column direction), and the third dimension is the width W of the data in each channel in the horizontal dimension (i.e., the number of elements in the array along the row direction). For example, in the example of Figure 4, N=1, C=64, H=4, and W=2.

[0069] For example, as shown in the lower left corner of FIG4 , the first data combination includes 512 elements, and the number on each element represents the storage position of the element in the first memory. That is, the data arrangement format of the 512 elements in the first data combination in the first memory is NCHW. For example, specifically, according to the priority arrangement order of the third dimension W direction, the second dimension H direction, the first dimension C direction, and the batch N direction (i.e., the first order), starting from the starting element (i.e., storage position 1), the elements of the row where the starting element is located are arranged in sequence along the W direction (i.e., storage position 1, storage position 2), then moving to the next row along the H direction to arrange the elements of the row in sequence along the W direction (i.e., storage position 3, storage position 4)... After the elements in the H×W plane where the starting element is located (i.e., storage position 1 to storage position 8) are arranged, moving to the next H×W plane along the C direction to arrange multiple elements in the array in the above manner (i.e., storage position 9 to storage position 16)... until all 512 elements are arranged.

[0070] For example, as shown in FIG4 , the number of sub-elements contained in each element is 32 (i.e., m=32), and the multiple sub-elements are arranged along the direction of the third dimension W'; that is, the first dimension C' and the second dimension H' of each element are both 1, and the third dimension W' is 32 (i.e., C'×H'×W'=1×1×32).

[0071] For example, according to step S110 of FIG3 , 32 elements (i.e., n=32) in the first data combination can be read along the first dimension C, that is, as shown in FIG4 , storage locations 1, 9, 17…249 in the first memory are read sequentially. For example, the 32 elements (i.e., elements 1 to 32) read in the first order can serve as the first processing group; the first processing group is shown in the upper left corner of FIG4 (i.e., the shaded portion), and the first to 32 elements in the first processing group are arranged along the first dimension C. For example, the first processing group shown in FIG4 can include only the first dimension C1, the second dimension H1, and the third dimension W1, and the first dimension C1 of the first processing group can be 32 elements (i.e., n=32) contained in the first processing group, the second dimension H1 can be 1, and the third dimension W1 can be 32 sub-elements contained in each element (i.e., m=32), that is, C1×H1×W1=32×1×32. For example, as shown in FIG4 , the number of elements included in the first processing group is n=32, and the number of sub-elements contained in each element is m=32, that is, m=n.

[0072] For example, according to step S120 of FIG3 , the first dimension C1 and the third dimension W1 of the first processing group are transposed to obtain a second processing group. For example, the second processing group is shown in the upper right corner of FIG4 (i.e., the shaded portion), and the first to 32 elements in the second processing group are arranged in the direction of the third dimension W, and the first dimension C2 of the second processing group is the number of sub-elements contained in each element, 32 (i.e., m=32), the second dimension H2 can be 1, and the third dimension W2 is the number of elements contained in the second processing group, 32 (i.e., n=32), that is, C2×H2×W2=32×1×32.

[0073] For example, as shown in Figure 4, the first dimension C" of each element in the second processing group is 32, and the second dimension H" and the third dimension W" are both 1 (i.e., C"×H"×W"=32×1×1), that is, the first dimension C" and the third dimension W" of each element in the second processing group are also transposed compared to the first dimension C' and the third dimension W' of each element in the first processing group.

[0074] For example, according to step S130 of FIG3 , elements 1 to 32 in the second processing group are written to corresponding storage locations in the first memory; the second data combination obtained after writing multiple second processing groups to the first memory is shown in the lower right corner of FIG4 , where the number on each element of the second data combination represents the storage location of the element in the first memory. Specifically, as shown in FIG4 , according to step S130 of FIG3 , the 32 elements are sequentially written to storage locations 1, 3, 5, 7…63 in the first memory, that is, the storage locations of the 32 elements in the first memory are the same as the arrangement locations in the second processing group.

[0075] For example, as shown in the upper left corner of Figure 4, according to step S140, after elements 1 to 32 in the first data combination are converted according to steps S110 to S130 in Figure 3, the conversion can be continued on the next group of elements (i.e., elements 33 to 64) in the manner of steps S110 to S130 in Figure 3... until all elements in the first data combination are converted, thereby obtaining the converted second data combination as shown in the lower right corner of Figure 4.

[0076] For example, specifically, in step S140, the first data combination can be read multiple times in the manner of step S110. After the first first processing group is read, the second first processing group can be read in the first order, and then the third first processing group can be read in the first order... until the first data combination is read; 32 elements are read each time, thereby obtaining 16 first processing groups, wherein each first processing group includes 32 elements arranged in the direction of the first dimension C.

[0077] For example, further, the first dimension and the third dimension transposition can be performed on the 32 first processing groups respectively according to the method in step S120 to obtain 32 corresponding second processing groups, wherein each second processing group includes 32 elements arranged in the direction of the third dimension W.

[0078] For example, further in step S150, multiple second processing groups can be written into the first memory multiple times in the manner of step S130 to obtain a second data combination. For example, as shown in the lower right corner of Figure 4, the number of elements included in the obtained second data combination is the same as the number of elements included in the first data combination.

[0079] For example, as shown in FIG4 , the data arrangement format of the first data combination in the first memory is NCHW. After the conversion in steps S110 to S150 , the data arrangement format of the second data combination in the first memory is NHWC.

[0080] For example, as shown in the lower right corner of Figure 4, in the first memory, for the second data combination in the NHWC format, according to the priority arrangement order of the first dimension C direction, the second dimension W direction, the third dimension H direction and the batch N direction (i.e., the second order), starting from the starting element (i.e., storage position 1), the elements in the row where the starting element is located are arranged in sequence along the C direction (i.e., storage position 1, storage position 2), and then move to the next row along the W direction to arrange the elements of the row in sequence along the C direction (i.e., storage position 3, storage position 4)... After the C×W plane where the starting write position is located (i.e., storage position 1 to storage position 128) is filled, move along the H direction to the next C×W plane to continue to arrange multiple elements in the C×W plane in sequence in the above manner (i.e., storage position 129 to storage position 256)... until 512 elements are arranged.

[0081] For example, for the example of FIG4 , each time a request is made to read 32 elements in the first data combination, the requested read address RD_addr can be expressed by the following formula (1):

[0082] RD_addr=datain_addr+C×IN_PLANAR_STRIDE+H×

[0083] IN_LINE_STRIDE+W×2(1)

[0084] Wherein, datain_addr represents the starting address of the data requested to be read (e.g., the address of element 1 in the lower left corner of FIG4 ), IN_PLANAR_STRIDE represents the spacing of each H×W plane in the first memory in the C direction, and IN_LINE_STRIDE represents the spacing of each line of data in the first memory in the W direction. For example, as shown in formula (1), the read address RD_addr is equal to the sum of the starting address datain_addr, the product of C and the H×W plane spacing (planar-stride), the product of H and the W direction line spacing (line-stride), and W and the space occupied by each element (e.g., 2 bytes for int16 data precision).

[0085] For example, for the example of FIG4 , each time a request is made to write 32 elements in the second processing group into the first memory, the requested write address WR_addr can be expressed by the following formula (2):

[0086] WR_addr=dataout_addr+W×OUT_PLANAR_STRIDE+H×

[0087] OUT_LINE_STRIDE+C×2(2)

[0088] Where dataout_addr represents the starting address of the data requested to be written (e.g., the address of element 1 in the lower right corner of Figure 4 ), OUT_PLANAR_STRIDE represents the spacing of each H×C plane in the first memory in the W direction, and OUT_LINE_STRIDE represents the spacing of each line of data in the first memory in the C direction. For example, as shown in formula (2), the write address WR_addr is equal to the sum of the starting address dataout_addr, the product of W and the H×C plane spacing (planar-stride), the product of H and the C direction line spacing (line-stride), and C and the space occupied by each element (e.g., 2 bytes for int16 data precision).

[0089] It should be noted that the example shown in Figure 4 is based on the data precision of int16. The data conversion method provided in the embodiment of the present disclosure can also be used for other data precisions (for example, int8, fp64, fp32, fp16, etc.) according to actual needs. The embodiment of the present disclosure does not limit this.

[0090] The data conversion method provided by at least one embodiment of the present disclosure supports conversion between different data arrangement formats corresponding to different neural network frameworks (i.e., conversion between dimensions of the first data combination), and can offload the corresponding data conversion tasks from processing devices such as accelerators, thereby reducing the delay of data conversion and alleviating the burden on devices such as accelerators.

[0091] FIG. 5 is an exemplary flow chart of an example of step S120 in FIG. 3 .

[0092] For example, in step S120, the first processing group is written into the second memory, and the second memory includes n cache lines; as shown in FIG5, in order to perform transposition on the first dimension and the third dimension of the first processing group, step S120 in FIG3 may include the following steps S121 to S122.

[0093] Step S121: shifting the first element to the nth element in the first processing group sequentially to write the first cache line to the nth cache line in n cache lines to obtain a cache data group, wherein the cache data group includes a plurality of diagonal lines;

[0094] Step S122: Read multiple diagonal rows in sequence to obtain a second processing group.

[0095] For example, in step S121, taking the first processing group as including n elements and m sub-elements contained in each element as an example, the first dimension of the first processing group is n, the second dimension is 1, and the third dimension is m. In this case, for example, the n×m sub-elements of the first processing group are written row by row into n cache lines, each cache line includes m columns, and the n×m sub-elements written into the second memory are arranged two-dimensionally to form an n-row and m-column sub-element array, which serves as a cache data group. For example, when m=n, the cache data group includes 2n-1 diagonal rows.

[0096] For example, in step S121, taking the case where each element in the first processing group includes n sub-elements (i.e., m=n), first, the n sub-elements in the first element in the first processing group are written into the first cache line of the n cache lines, wherein the first sub-element to the nth sub-element in the first element are sequentially written into the first column to the nth column of the first cache line; then, the n sub-elements in the kth element in the first processing group are written into the kth cache line of the n cache lines, wherein the kth sub-element to the nth sub-element in the kth element are sequentially written into the kth cache line of the n cache lines. Write the first column to the n-k+1th column of the kth cache row, and the first sub-element to the k-1th sub-element of the kth element are written to the n-k+2th column to the nth column of the kth cache row, where k = 2, 3, ..., n-1; finally, write the n sub-elements of the nth element in the first processing group to the nth cache row of n cache rows, wherein the nth sub-element of the nth element is written to the first column of the nth cache row, and the first sub-element to the n-1th sub-element of the nth element are written to the second column to the nth column of the nth cache row. For example, through the above process, a cache data group can be obtained, which includes n×n sub-elements in n rows and n columns, and the multiple diagonal rows of the cache data group include the first diagonal row to the 2n-1th diagonal row.

[0097] For example, in step S122, again taking the example of each element in the first processing group including n sub-elements (i.e., m=n), first, k sub-elements in the k-th diagonal row among the plurality of diagonal rows and n-k sub-elements in the n+k-th diagonal row among the plurality of diagonal rows are read as the k-th element in the second processing group, where k=1, 2, ..., n-1; then, n sub-elements in the n-th diagonal row among the plurality of diagonal rows are read as the n-th element in the second processing group. For example, through the above process, a second processing group is obtained, where the first dimension of the second processing group is m, the second dimension is 1, and the third dimension is n.

[0098] For example, since the 1st to nth elements in the first processing group are arranged along the first dimension, correspondingly, the 1st to mth sub-elements in each element are arranged along the third dimension. The process of sequentially shifting and writing the n elements in the first processing group into n cache lines is the process of sequentially shifting and writing into n cache lines along the first dimension, and the process of sequentially reading multiple diagonal rows is the process of reading and obtaining the n elements of the second processing group along the third dimension. Therefore, the 1st to nth elements in the read second processing group are arranged along the third dimension, while the 1st to mth sub-elements in each element are arranged along the first dimension. That is, the first and third dimensions of the second processing group are transposed compared to those of the first processing group.

[0099] Figure 6 is a schematic diagram of another example of a data conversion method provided by at least one embodiment of the present disclosure. For example, Figure 6 is a specific example of the data conversion method shown in Figure 3 or Figure 5.

[0100] For example, as shown in Figure 6, the first data combination is stored in the first memory. In step S110 of Figure 3, the 1st to nth elements in the first data combination are read out in the first order as the first processing group, and the n elements in the first processing group are written into the second memory respectively; in step S120 of Figure 3, the second memory performs a transposition on the first dimension and the third dimension of the first processing group to obtain a second processing group. For example, the first dimension of the first processing group is, for example, the number of channels C1, and the third dimension is, for example, the width W1. After processing in step S120, the first dimension C2 and the third dimension W2 of the second processing group are transposed compared to the first dimension C1 and the third dimension W1 of the first processing group (i.e., C1W1=>W2C2); in step S130 of Figure 3, the 1st to nth elements in the second processing group are written into the first memory in the second order.

[0101] For example, in the example of FIG6 , the first memory may be a memory, and the second memory may be a buffer (e.g., a ping-pong buffer). For example, as shown in FIG6 , the second memory may be composed of a plurality of SRAMs, and each column of the second memory is an SRAM, and the SRAM may independently control the read and write addresses of each position thereof. Since the SRAM occupies a smaller space, compared to registers with a larger area overhead, the SRAM is more suitable for format conversion of large-scale data, thereby saving space and improving data conversion efficiency.

[0102] For example, since for each SRAM, only the sub-elements at one position can be read at a time, the sub-elements in each column of the second memory (that is, the sub-elements at all positions of an SRAM) cannot be read simultaneously; therefore, the transposition between rows and columns can be completed by circular shift writing and oblique reading. Steps S121 to S122 in Figure 5 are an example of circular shift writing and oblique reading; when writing by row, the addresses of the sub-elements in each cache row are the same, and when reading obliquely, the addresses of the sub-elements in each diagonal row are different.

[0103] For example, as shown in Figure 6, multiple elements in the first processing group are transposed in the first dimension and the third dimension by circular shift writing and oblique reading. For example, in the example of Figure 6, the number of elements included in the first processing group is 16 (i.e., n=16), and the number of sub-elements contained in each element is also 16 (m=16), that is, the first processing group contains 16×16 sub-elements, and the first dimension C1 of the first processing group is 16, the second dimension H1 is 1, and the third dimension W1 is 16.

[0104] For example, as shown in FIG6 , according to step S121 of FIG5 , the 1st to 16th elements in the first processing group are sequentially shifted and written to the 1st to 16th cache lines to obtain a cache data group; that is, the 16×16 sub-elements of the first processing group are written row by row to the 16 cache lines of the second memory, with each row shifted by an additional sub-element length compared to the previous row. For example, each cache line includes 16 columns, and the 16×16 sub-elements after being written to the second memory are arranged two-dimensionally to form a 16-row, 16-column sub-element array as a cache data group, which includes 31 diagonal rows. For example, further, according to step S122 of FIG5 , the 31 diagonal rows are read sequentially to obtain a second processing group.

[0105] For example, as shown in FIG6 , according to step S121 of FIG5 , the specific process of cyclic shift writing is:

[0106] (1) For the first element in the first processing group (e.g., the example element read from the first memory in FIG6 , the first cache row is shown as the first row of the second memory on the left side of FIG6 ), its 16 sub-elements are sequentially written into the first to the sixteenth columns of the first cache row, i.e., no shifting is required when writing the first element;

[0107] (2) For the kth element in the first processing group (the kth cache row is shown as the kth row of the second memory on the left side of FIG6 ), where k=2, 3, …, 15, its kth to 16th sub-elements are sequentially written into the 1st column to the 17-kth column, and its 1st to k-1th sub-elements are written into the 18-kth to 16th columns of the kth cache row (taking the 3rd element in the first processing group as an example, its 3rd to 16th sub-elements are sequentially written into the 1st column to the 14th column, and its 1st to 2nd sub-elements are written into the 15th to 16th columns of the 3rd cache row);

[0108] (3) For the 16th element in the first processing group (the 16th cache row is shown as the last row of the second memory on the left side of Figure 6), its 16th sub-element is written to the 1st column of the 16th cache row, and its 1st to 15th sub-elements are written to the 2nd to 16th columns of the 16th cache row.

[0109] For example, as shown in FIG6 , the above process is equivalent to shifting the sub-elements in the dashed portion on the left side of each row of the second memory to the position indicated by the arrow on the right side. For example, the cache data group obtained by the above process is shown in the solid line portion of the second memory in FIG6 . This cache data group includes 16 rows and 16 columns of 16×16 sub-elements, and the multiple diagonal rows of the cache data group include the first diagonal row to the 31st diagonal row.

[0110] For example, as shown in FIG6 , according to step S122 of FIG5 , the specific process of the exemplary oblique reading is as follows:

[0111] (1) Read the k sub-elements in the k-th diagonal row and the 16-k sub-elements in the 16+k-th diagonal row as the k-th element in the second processing group, where k = 1, 2, ..., 15 (taking the sub-elements marked by the shaded portion in the second memory on the right side of FIG6 as an example, read the 14 sub-elements in the 14th diagonal row and the 2 sub-elements in the 30th diagonal row as the 14th element in the second processing group);

[0112] (2) Read the 16 sub-elements in the 16th diagonal row as the 16th element in the second processing group.

[0113] For example, through the above process, a transposed second processing group is obtained. For example, after the 16 elements in the first processing group are shifted and written into 16 cache lines in sequence, the addresses of the sub-elements in each cache line are the same, while the addresses of the sub-elements in each diagonal row are different when read diagonally. For example, since the 1st to 16th elements in the first processing group are arranged in the direction of the first dimension, correspondingly, the 1st to 16th sub-elements in each element are arranged in the direction of the third dimension, according to the address correspondence between the written and read sub-elements, the 1st to 16th elements in the second processing group obtained by reading are arranged in the direction of the third dimension, while the 1st to 16th sub-elements in each element are arranged in the direction of the first dimension. Therefore, the first dimension and the third dimension of the second processing group are transposed compared to the first processing group.

[0114] In the data conversion method provided by the embodiments of the present disclosure, the embodiment shown in Figure 5 or Figure 6 can not only complete step S120 of Figure 3, but also be used in other application scenarios that require two-dimensional data transposition. Therefore, the data conversion method provided by at least one embodiment of the present disclosure, in addition to supporting the conversion between different data arrangement formats corresponding to different neural network frameworks (for example, NCHW=>NHWC), can also be used to achieve the transposition of two-dimensionally arranged data, thereby further being used for the transposition of row-major order and column-major order in matrix multiplication operations (for example, height×width=>width×height) or other application scenarios, and the embodiments of the present disclosure are not limited to this.

[0115] It should be noted that the data arrangement format (type of dimension or arrangement method), the number of elements included, and the corresponding conversion method of the first data combination, the first processing group, the second processing group, and the second data combination described in Figures 3 to 6 are only exemplary. Other arrangement forms or corresponding conversion methods can also be selected according to actual needs. The embodiments of the present disclosure do not limit this.

[0116] FIG7 is a schematic block diagram of a data conversion device provided by at least one embodiment of the present disclosure.

[0117] For example, as shown in FIG7 , the data conversion device 200 is used to convert the dimensions of a first data set. For example, the data conversion device 200 includes a reading module 210, a transposition module 220, and a writing module 230. For example, the first data set includes at least one batch, and the dimensions of the data corresponding to each batch include a first dimension, a second dimension, and a third dimension. In some examples, the first dimension is the number of channels, the second dimension is the height, and the third dimension is the width.

[0118] For example, the reading module 210 is configured to read n elements in the first data group in a first order to obtain a first processing group. For example, the first to nth elements in the first processing group are arranged along a first dimension, where n is a positive integer. In other words, the acquisition module 210 can be configured to execute, for example, step S110 shown in FIG. 3 .

[0119] For example, the transposition module 220 is configured to perform a transposition on the first and third dimensions of the first processing group to obtain a second processing group. For example, the first to nth elements in the second processing group are arranged along the third dimension. In other words, the processing module 220 can be configured to perform step S120 shown in Figure 3, for example.

[0120] For example, the writing module 230 is configured to write the 1st to nth elements in the second processing group into the first memory. In other words, the calculating module 230 can be configured to execute step S130 shown in FIG3 .

[0121] For example, the reading module 210 is further configured to read the first data combination multiple times to obtain multiple first processing groups. In other words, the reading module 210 can be configured to execute step S140.

[0122] For example, the writing module 230 is further configured to write the plurality of second processing groups into the first memory to obtain a second data combination. For example, the number of elements included in the second data combination is the same as the number of elements included in the first data combination. In other words, the writing module 230 can be configured to execute step S150.

[0123] In some examples, multiple elements in the first data set are stored in the first memory in a first order, where the first order is a priority order of the third dimension, the second dimension, the first dimension, and the batch direction. In some examples, multiple elements in the second data set are stored in the first memory in a second order, where the second order is a priority order of the first dimension, the third dimension, the second dimension, and the batch direction.

[0124] In some examples, the first dimension of each element in the first processing group is 1, the second dimension of each element in the first processing group is 1, and the third dimension of each element in the first processing group is m; the first dimension of each element in the second processing group is m, the second dimension of each element in the second processing group is 1, and the third dimension of each element in the second processing group is 1, where m is a positive integer. In some examples, m=n.

[0125] In some examples, the first processing group is written to a second memory, where the second memory includes n cache lines. For example, the transposition module 220 can also be configured to sequentially shift elements 1 through n in the first processing group and write them to cache lines 1 through n in the n cache lines, thereby obtaining a cache data group, e.g., the cache data group includes multiple diagonal lines; and sequentially read the multiple diagonal lines to obtain a second processing group. In some examples, the first memory is a memory, and the second memory is a buffer.

[0126] In some examples, each element in the first processing group includes n sub-elements; the transposition module 220 can also be configured to: write the n sub-elements of the first element in the first processing group to the first cache line of the n cache lines, wherein the first sub-element to the nth sub-element of the first element are written to the first column to the nth column of the first cache line in sequence; write the n sub-elements of the kth element in the first processing group to the kth cache line of the n cache lines, wherein the kth sub-element to the nth sub-element of the kth element are written to the kth cache line of the n cache lines. The 1st column to the n-k+1th column of the kth cache row and the 1st sub-element to the k-1th sub-element of the kth element are written to the n-k+2th column to the nth column of the kth cache row, where k = 2, 3, ..., n-1; the n sub-elements of the nth element in the first processing group are written to the nth cache row of n cache rows, where the nth sub-element of the nth element is written to the 1st column of the nth cache row, and the 1st sub-element to the n-1th sub-element of the nth element are written to the 2nd column to the nth column of the nth cache row.

[0127] In some examples, each element in the second processing group includes n sub-elements, the cache data group includes n×n sub-elements of n rows and n columns, and the multiple diagonal rows include the 1st diagonal row to the 2n-1th diagonal row; the transposition module 220 can also be configured to: read the k sub-elements in the kth diagonal row in the multiple diagonal rows and the nk sub-elements in the n+kth diagonal row in the multiple diagonal rows as the kth element in the second processing group, where k=1,2,…,n-1; read the n sub-elements in the nth diagonal row in the multiple diagonal rows as the nth element in the second processing group.

[0128] Since the details of the operations of the data conversion device 200 have been introduced in the above description of the data conversion method shown in Figure 3, they will not be repeated here for the sake of brevity. For relevant details, please refer to the above description of Figures 1 to 6.

[0129] It should be noted that the above-mentioned modules in the data conversion device 200 shown in Figure 7 can be configured as software, hardware, firmware or any combination of the above items to perform specific functions. For example, these modules can correspond to dedicated integrated circuits, can also correspond to pure software code, and can also correspond to modules that combine software and hardware. As an example, the device described with reference to Figure 7 can be a PC computer, a tablet device, a personal digital assistant, a smart phone, a web application or other device that can execute program instructions, but is not limited to this.

[0130] In addition, although it is divided into the module that is used to carry out corresponding processing respectively when describing data conversion device 200 above, yet, it is clear to those skilled in the art that the processing that each module is carried out also can not carry out under the situation of clear demarcation between any concrete module division or each module in device.In addition, the data conversion device 200 described above with reference to Fig. 7 is not limited to comprise the module described above, but can also increase some other modules (for example, storage module, control module etc.) as required, and perhaps above module also can be combined.

[0131] At least one embodiment of the present disclosure also provides another data conversion device, which includes a processor and a memory; the memory includes one or more computer program modules; the one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules include a method for implementing the data conversion method provided by the embodiment of the present disclosure described above.

[0132] FIG8 is a schematic block diagram of another data conversion device provided by at least one embodiment of the present disclosure.

[0133] For example, as shown in FIG8 , the data conversion device 300 includes a processor 310 and a memory 320. For example, the memory 320 is used to store non-transitory computer-readable instructions (e.g., one or more computer program modules). The processor 310 is used to execute the non-transitory computer-readable instructions. When the non-transitory computer-readable instructions are executed by the processor 310, one or more steps of the data conversion method described above can be executed. The memory 320 and the processor 310 can be interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0134] For example, the processor 310 may be a digital signal processor (DSP) or other processing unit with data conversion capability and / or program execution capability, such as an X86, ARM architecture, or a field programmable gate array (FPGA). The processor 310 may be a general-purpose processor or a dedicated processor, and may control other components in the data conversion device 300 to perform desired functions.

[0135] For example, the memory 320 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, a flash memory, etc. One or more computer program modules may be stored on the computer-readable storage medium, and the processor 310 may execute one or more computer program modules to implement various functions of the data conversion device 300. The computer-readable storage medium may also store various applications and various data, as well as various data used and / or generated by the applications.

[0136] It should be noted that, in the embodiments of the present disclosure, the specific functions and technical effects of the data conversion device 300 can be referred to the above description of the data conversion method provided in at least one embodiment of the present disclosure, and will not be repeated here.

[0137] At least one embodiment of the present disclosure further provides an electronic device, which includes a data processing device and the data conversion device provided by any embodiment of the present disclosure.

[0138] FIG9 is a schematic block diagram of an electronic device provided by at least one embodiment of the present disclosure.

[0139] For example, as shown in FIG9 , the electronic device 400 includes a data conversion device 410 and a data processing device 420. For example, the electronic device 400 may be a neural network accelerator or other AI accelerator, or any device such as a mobile phone, tablet computer, laptop computer, e-book, game console, television, digital photo frame, or navigation system, or any combination of electronic devices and hardware, which is not limited in the embodiments of the present disclosure.

[0140] For example, the data conversion device 410 may be the data conversion device provided in any embodiment of the present disclosure, and the data processing device 420 may be a module in a neural network accelerator for data processing (e.g., data transmission, convolution calculation, etc.), including but not limited to a processor, a controller, a memory, a bus, or other electronic devices for data processing.

[0141] FIG10 is a schematic block diagram of another electronic device provided by at least one embodiment of the present disclosure.

[0142] For example, as shown in Figure 10, the electronic device 500 is suitable for implementing the data conversion method provided by the embodiment of the present disclosure. It should be noted that the electronic device 500 shown in Figure 10 is only an example and does not bring any limitation to the functions and scope of use of the embodiment of the present disclosure.

[0143] For example, as shown in FIG10 , electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 51, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 52 or programs loaded from a storage device 58 into a random access memory (RAM) 53. RAM 53 also stores various programs and data required for the operation of electronic device 500. Processing device 51, ROM 52, and RAM 53 are interconnected via a bus 54. An input / output (I / O) interface 55 is also connected to bus 54. Typically, the following devices may be connected to I / O interface 55: input device 56, such as a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output device 57, such as a liquid crystal display (LCD), speaker, vibrator, etc.; storage device 58, such as a magnetic tape, hard disk, etc.; and communication device 59. Communication device 59 may allow electronic device 500 to communicate with other electronic devices wirelessly or by wire to exchange data.

[0144] Although FIG. 10 shows the electronic device 500 having various devices, it should be understood that it is not required to implement or possess all of the devices shown, and the electronic device 500 may alternatively implement or possess more or fewer devices.

[0145] It should be noted that for the sake of clarity and brevity, the embodiments of the present disclosure do not illustrate all components of the electronic device 400 / 500. To implement the necessary functions of the electronic device, those skilled in the art may provide and configure other components not shown according to specific needs, and the embodiments of the present disclosure do not limit this.

[0146] For detailed description and technical effects of the electronic device 400 / 500, please refer to the above description of the data conversion method, which will not be repeated here.

[0147] FIG11 is a schematic diagram of a storage medium provided by at least one embodiment of the present disclosure.

[0148] 11 , a storage medium 600 stores non-transitory computer-readable instructions 610. For example, when the non-transitory computer-readable instructions 610 are executed by a computer, one or more steps in the data conversion method described above are performed.

[0149] For example, the storage medium 600 can be applied to the data conversion device 300 shown in Figure 8. For example, the storage medium 600 can be the memory 320 in the data conversion device 300. For example, the relevant description of the storage medium 600 can refer to the corresponding description of the memory 320 in the data conversion device 300 shown in Figure 8, and will not be repeated here.

[0150] Regarding this disclosure, the following points need to be explained:

[0151] (1) The drawings of the embodiments of the present disclosure only relate to the structures related to the embodiments of the present disclosure, and other structures may refer to conventional designs.

[0152] (2) Unless there is any conflict, the features of the same embodiment and different embodiments of the present disclosure may be combined with each other.

[0153] The above are only specific embodiments of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A data conversion method, for converting the dimension of a first data combination, wherein: The first data combination includes at least one batch, the dimensions of data corresponding to each batch in the at least one batch include a first dimension, a second dimension, and a third dimension, and the data conversion method includes: Reading n elements in the first data combination along a first dimension direction to obtain a first processing group, wherein the first element to the nth element in the first processing group are arranged along the first dimension direction, wherein n is a positive integer; Transposing the first dimension and the third dimension of the first processing group to obtain a second processing group, wherein the first element to the nth element in the second processing group are arranged along the third dimension; Write the 1st to nth elements in the second processing group into the first memory.

2. The data conversion method according to claim 1, wherein: The multiple elements in the first data combination are stored in the first memory in a first order, where the first order is a priority arrangement order of the third dimension direction, the second dimension direction, the first dimension direction, and the batch direction.

3. The data conversion method according to claim 1 or 2, further comprising: The first data combination is read multiple times to obtain multiple first processing groups.

4. The data conversion method according to any one of claims 1 to 3, further comprising: writing a plurality of the second processing groups into the first memory to obtain a second data combination, The number of elements included in the second data combination is the same as the number of elements included in the first data combination.

5. The data conversion method according to claim 4, wherein: The multiple elements in the second data combination are stored in the first memory in a second order, where the second order is a priority arrangement order of the first dimension direction, the third dimension direction, the second dimension direction, and the batch direction.

6. The data conversion method according to any one of claims 1 to 5, wherein: The first dimension is the number of channels, the second dimension is the height, and the third dimension is the width.

7. The data conversion method according to any one of claims 1 to 6, wherein: The first dimension of each element in the first processing group is 1, the second dimension of each element in the first processing group is l, and the third dimension of each element in the first processing group is m, The first dimension of each element in the second processing group is m, the second dimension of each element in the second processing group is 1, and the third dimension of each element in the second processing group is 1, Wherein, m is a positive integer.

8. The data conversion method according to claim 7, wherein: m=n.

9. The data conversion method according to any one of claims 1 to 8, wherein: The first processing group is written into a second memory, the second memory comprising n cache lines, Transposing the first dimension and the third dimension of the first processing group to obtain the second processing group includes: Shifting and writing the first element to the nth element in the first processing group into the first cache line to the nth cache line in sequence, to obtain a cache data group, wherein the cache data group includes a plurality of diagonal lines; The plurality of diagonal rows are read sequentially to obtain the second processing group.

10. The data conversion method according to claim 9, wherein: Each element in the first processing group includes n sub-elements, Shifting the first element to the nth element in the first processing group sequentially into the first cache line to the nth cache line in the n cache lines, comprising: Writing n sub-elements of the first element in the first processing group into the first cache line of the n cache lines, wherein the first to nth sub-elements of the first element are sequentially written into the first to nth columns of the first cache line; Writing n sub-elements of the kth element in the first processing group into the kth cache line of the n cache lines, wherein the kth to nth sub-elements of the kth element are written into the 1st column to the n-k+1th column of the kth cache line, and the 1st to k-1th sub-elements of the kth element are written into the n-k+2th column to the nth column of the kth cache line, wherein k=2, 3, ..., n-1; Writing n sub-elements of an nth element in the first processing group into an nth cache line among the n cache lines, wherein the nth sub-element of the nth element is written into the first column of the nth cache line, and the first to n-1th sub-elements of the nth element are written into the second to nth columns of the nth cache line.

11. The data conversion method according to claim 9 or 10, wherein: Each element in the second processing group includes n sub-elements, The cache data group includes n×n sub-elements of n rows and n columns, and the plurality of diagonal rows include the 1st diagonal row to the 2n-1th diagonal row, The plurality of diagonal rows are read sequentially to obtain the second processing group, comprising: Reading k sub-elements in a k-th diagonal row of the plurality of diagonal rows and nk sub-elements in an n+k-th diagonal row of the plurality of diagonal rows as the k-th element in the second processing group, where k=1, 2, ..., n-1; Read n sub-elements in an n-th diagonal row of the plurality of diagonal rows as an n-th element in the second processing group.

12. The data conversion method according to any one of claims 9 to 11, wherein: The first memory is a memory, and the second memory is a buffer.

13. A data conversion device, for converting the dimension of a first data combination, wherein: The first data combination includes at least one batch, and the dimensions of the data corresponding to each batch in the at least one batch include a first dimension, a second dimension, and a third dimension. The data conversion device comprises: a reading module configured to read n elements in the first data combination along a first dimension to obtain a first processing group, wherein the first element to the nth element in the first processing group are arranged along the first dimension, wherein n is a positive integer; a transposition module configured to perform transposition on the first dimension and the third dimension of the first processing group to obtain a second processing group, wherein the first element to the nth element in the second processing group are arranged along the third dimension; A writing module is configured to write the 1st element to the nth element in the second processing group into the first memory.

14. The data conversion device according to claim 13, wherein: The multiple elements in the first data combination are stored in the first memory in a first order, where the first order is a priority arrangement order of the third dimension direction, the second dimension direction, the first dimension direction, and the batch direction.

15. The data conversion device according to claim 13 or 14, wherein: The reading module is further configured to read the first data combination multiple times to obtain multiple first processing groups.

16. The data conversion device according to any one of claims 13 to 15, wherein: The writing module is further configured to write a plurality of the second processing groups into the first memory to obtain a second data combination, The number of elements included in the second data combination is the same as the number of elements included in the first data combination.

17. The data conversion device according to any one of claims 13 to 16, wherein: The multiple elements in the second data combination are stored in the first memory in a second order, where the second order is a priority arrangement order of the first dimension direction, the third dimension direction, the second dimension direction, and the batch direction.

18. A data conversion device, comprising: processor; a memory including one or more computer program modules; The one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules are used to implement the data conversion method according to any one of claims 1 to 12.

19. An electronic device comprising: The data conversion device according to any one of claims 13 to 18; as well as A data processing device is configured to process the first data combination.

20. A storage medium storing non-transitory computer-readable instructions, which implement the data conversion method according to any one of claims 1 to 12 when executed by a computer.

Citation Information

Patent Citations

  • Data batch standardization method, computing device and computer readable storage medium

    CN112926646A

  • Data storage method and device

    CN115291802A

  • Data conversion circuit and method, electronic equipment and computer readable storage medium

    CN115760538A

  • Data processing method and device, electronic equipment and storage medium

    CN115906932A

  • Method for permuting dimensions of a multi-dimensional tensor

    US20220129744A1