Data conversion method and device, electronic equipment and storage medium
By transposing the first and third dimensions of data combinations in the neural network accelerator, the problem of data arrangement format conversion between different neural network frameworks is solved, reducing data conversion delay and reducing device burden.
Patent Information
- Application Number
- CN202310514473.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-09
- Publication Date
- 2024-11-12
AI Technical Summary
Existing neural network accelerators are difficult to support the conversion between data layout formats between different neural network frameworks, resulting in a large delay in data conversion and increasing the burden on the device.
Provide a data conversion method and device, by reading elements in the data combination in the first dimension direction, performing transposition of the first dimension and the third dimension, and writing the result to the memory, supporting the different neural network frameworks 数据排布格式转换,减少加速器的负担。
It reduces the delay in data conversion, reduces the burden on accelerator and other devices, and realizes efficient conversion between data layout formats between different neural network frameworks.
Smart Images

Figure CN118939235A_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present disclosure relate to a data conversion method, a data conversion device, an electronic device, and a storage medium. Background Art
[0002] The field of deep learning in artificial intelligence involves a large number of operations such as convolutional neural networks (CNNs) and matrix multiplications. In current computing devices, data processing integrated circuits such as coprocessors and accelerators can execute programs to complete various functions of artificial intelligence operations. An accelerator for deep learning can be a configurable fixed-function hardware accelerator that can be used for training and reasoning operations in deep learning applications; it provides complete hardware acceleration for convolutional neural networks by exposing operations associated with each convolutional layer (e.g., convolution, pooling, full connection, activation, aggregation, local response normalization, etc.). Summary of the invention
[0003] At least one embodiment of the present disclosure provides a data conversion method for converting the dimension of a first data combination, wherein the first data combination includes at least one batch, and the dimension of data corresponding to each batch in the at least one batch includes a first dimension, a second dimension and a third dimension. The data conversion method includes: reading n elements in the first data combination along a first dimension direction to obtain a first processing group, wherein the first element to the nth element in the first processing group are arranged along the first dimension direction, wherein n is a positive integer; performing transposition on the first dimension and the third dimension of the first processing group to obtain a second processing group, wherein the first element to the nth element in the second processing group are arranged along the third dimension direction; and writing the first element to the nth element in the second processing group into a first memory.
[0004] At least one embodiment of the present disclosure further provides a data conversion device for converting the dimension of a first data combination, wherein the first data combination includes at least one batch, and the dimension of the data corresponding to each batch in the at least one batch includes a first dimension, a second dimension and a third dimension, and the data conversion device includes: a reading module, configured to read n elements in the first data combination in the direction of the first dimension to obtain a first processing group, wherein the 1st element to the nth element in the first processing group are arranged in the direction of the first dimension, wherein n is a positive integer; a transposition module, configured to perform transposition on the first dimension and the third dimension of the first processing group to obtain a second processing group, wherein the 1st element to the nth element in the second processing group are arranged in the direction of the third dimension; a writing module, configured to write the 1st element to the nth element in the second processing group into a first memory.
[0005] At least one embodiment of the present disclosure further provides a data conversion device. The data conversion device includes: a processor; a memory, including one or more computer program modules; wherein the one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules are used to implement the data conversion method provided in any embodiment of the present disclosure.
[0006] At least one embodiment of the present disclosure further provides an electronic device. The electronic device comprises: a data conversion device provided by any embodiment of the present disclosure; and a data processing device configured to process the first data combination.
[0007] At least one embodiment of the present disclosure further provides a storage medium storing non-transitory computer-readable instructions, which, when executed by a computer, implements the data conversion method provided by any embodiment of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, but are not intended to limit the present disclosure.
[0009] Figure 1 A schematic diagram of an example of dimensions of a data combination;
[0010] Figure 2 is a schematic diagram of a data conversion device;
[0011] Figure 3 An exemplary flow chart of a data conversion method provided for at least one embodiment of the present disclosure;
[0012] Figure 4A schematic diagram of an example of a data conversion method provided by at least one embodiment of the present disclosure;
[0013] Figure 5 for Figure 3 An exemplary flowchart of an example of step S120 in FIG.
[0014] Figure 6 A schematic diagram of another example of a data conversion method provided by at least one embodiment of the present disclosure;
[0015] Figure 7 A schematic block diagram of a data conversion device provided by at least one embodiment of the present disclosure;
[0016] Figure 8 A schematic block diagram of another data conversion device provided for at least one embodiment of the present disclosure;
[0017] Fig. 9 A schematic block diagram of an electronic device provided for at least one embodiment of the present disclosure;
[0018] Fig.10 A schematic block diagram of another electronic device provided for at least one embodiment of the present disclosure; and
[0019] Fig.11 A schematic diagram of a storage medium provided for at least one embodiment of the present disclosure. DETAILED DESCRIPTION
[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.
[0021] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, similar words such as "one", "one" or "the" do not indicate quantity restrictions, but indicate that there is at least one. Similar words such as "include" or "comprise" mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Similar words such as "connect" or "connected" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0022] The present disclosure is described below through several specific embodiments. In order to keep the following description of the embodiments of the present disclosure clear and concise, detailed descriptions of known functions and known components may be omitted. When any component of the embodiments of the present disclosure appears in more than one figure, the component is represented by the same or similar reference numeral in each figure.
[0023] For operations such as convolutional neural networks and matrix multiplication and addition, the implementation methods of various existing processors, such as coprocessors, accelerators and other hardware structures are different, resulting in different requirements for the format of data arrangement for each hardware structure. For example, for convolutional neural network operations, the data combination to be processed has multiple dimensions, the data combination includes at least one batch, and the data corresponding to each batch further includes multiple dimensions; for A×B matrix multiplication operations (both A matrix and B matrix are two-dimensional data combinations), the data arrangement format in memory is usually row-major order (height×width), while the ideal data arrangement format in hardware calculation is row-major order (height×width) for A matrix and column-major order (width×height) for B matrix.
[0024] Figure 1 A schematic diagram of an example of dimensions for a data combination.
[0025] For example, Figure 1As shown, the data combination has multiple dimensions [N, C, H, W], and the data combination includes at least one batch (Batch), N represents the number of batches; the data corresponding to each batch includes multiple channels, C represents the number of channels (Channel). For example, in each channel, multiple data elements are arranged in two dimensions to form an array, the height of the data in each channel in the vertical dimension is H, H represents the number of data elements in the array along the column direction, and the width of the data in each channel in the horizontal dimension is W, W represents the number of data elements in the array along the row direction.
[0026] For example, Figure 1 The data combination includes 2 batches, that is, the number of batches N=2; each batch includes 16 channels, that is, the number of channels in each batch C=16; the number of data elements in the data element array along the column direction in each channel is 5, that is, the height H=5; the number of data elements in the data element array along the row direction in each channel is 4, that is, the width W=4.
[0027] For example, data combinations in different neural network frameworks have different data arrangement formats, that is, when processing data combinations, the order of reading and writing memory is different according to the dimensions of the data combinations. For example, data arrangement formats include NHWC, NCHW, and private data arrangement formats in some accelerators (such as NCHWC'). For example, when using some neural network accelerators to process data combinations, it is necessary to use the data conversion module in the neural network accelerator to convert the data arrangement format.
[0028] Figure 2 A schematic diagram of a data conversion device.
[0029] For example, Figure 2 As shown, the data conversion device 110 includes a buffer 113 and a direct memory access (DMA) controller (including a read DMA 111 and a write DMA 112) for reading and writing respectively. For example, the data conversion device 110 can be a module for data arrangement format conversion in a neural network accelerator, and the data conversion device 110 is connected to other data processing devices in the accelerator through a bus interface 120. For example, the bus interface 120 can be a bus protocol interface (Advanced eXtensible Interface, AXI), or an AXI cache extension interface (AXICoherency Extensions, ACE), etc.
[0030] For example, the read DMA 111 reads part of the data to be processed from the memory into the buffer 113 through the bus interface 120. After the data is rearranged in the buffer 113, the write DMA 112 writes the rearranged data into the memory through the bus interface 120. For example, the buffer 113 may be composed of a plurality of static random access memories (SRAMs).
[0031] For example, the data conversion device in the neural network accelerator only supports the conversion between a specific data arrangement format (such as NCHW) in a specific neural network framework (such as the Pytorch framework) and a private data arrangement format (such as NCHWC'), and it is difficult to support the conversion between data arrangement formats (such as NHWC) in other neural network frameworks (such as the TensorFlow framework). Therefore, for the conversion between different data arrangement formats corresponding to different neural network frameworks (for example, the conversion between NCHW and NHWC, etc.), it is usually necessary to perform the corresponding data conversion tasks in processing devices such as accelerators, which leads to a large delay in data conversion and increases the burden on devices such as accelerators.
[0032] At least one embodiment of the present disclosure provides a data conversion method for converting the dimension of a first data combination. The first data combination includes at least one batch, and the dimensions of the data corresponding to each batch include a first dimension, a second dimension, and a third dimension. The data conversion method includes: reading n elements in the first data combination in the direction of the first dimension to obtain a first processing group, wherein the 1st element to the nth element in the first processing group are arranged in the direction of the first dimension, and n is a positive integer; performing a transposition on the first dimension and the third dimension of the first processing group to obtain a second processing group, wherein the 1st element to the nth element in the second processing group are arranged in the direction of the third dimension; and writing the 1st element to the nth element in the second processing group into a first memory.
[0033] At least one embodiment of the present disclosure also provides at least one data conversion device, electronic device and storage medium.
[0034] The method, apparatus, device and storage medium provided by at least one embodiment of the present disclosure support conversion between different data arrangement formats corresponding to different neural network frameworks, and can offload corresponding data conversion tasks from processing devices such as accelerators, thereby reducing data conversion delays and alleviating the burden on devices including accelerators.
[0035] Hereinafter, at least one embodiment of the present disclosure will be described in detail with reference to the accompanying drawings. It should be noted that the same reference numerals in different drawings will be used to refer to the same elements described.
[0036] Figure 3 An exemplary flow chart of a data conversion method provided for at least one embodiment of the present disclosure.
[0037] For example, Figure 3 As shown, at least one embodiment of the present disclosure provides a data conversion method for converting the dimension of a first data combination. For example, the first data combination includes at least one batch, and the dimensions of the data corresponding to each batch include a first dimension, a second dimension, and a third dimension. For example, the data conversion method includes the following steps S110 to S130.
[0038] Step S110: reading n elements in the first data combination along the first dimension direction to obtain a first processing group, where n is a positive integer;
[0039] Step S120: performing transposition on the first dimension and the third dimension of the first processing group to obtain a second processing group;
[0040] Step S130: writing the 1st to nth elements in the second processing group into the first memory.
[0041] For example, the first dimension, the second dimension, and the third dimension are different from each other; in some examples, the first dimension may be the number of channels, the second dimension may be the height, and the third dimension may be the width. For example, the first data combination includes N batches, and each batch corresponds to a plurality of channels of data, and the first dimension is the number of channels C; for example, in each channel, a plurality of elements are arranged two-dimensionally to form an array (i.e., an H×W plane), the second dimension is the height H of the data in each channel in the vertical dimension (i.e., the number of elements in the array along the column direction), and the third dimension is the width W of the data in each channel in the horizontal dimension (i.e., the number of elements in the array along the row direction).
[0042] It should be noted that the first dimension, the second dimension and the third dimension are not limited to the above description, and other selections may be made according to the specific format of the first data combination and actual needs, and the embodiments of the present disclosure are not limited to this.
[0043] In some examples, multiple elements in the first data combination are stored in the first memory in a first order, and the first order is a priority arrangement order in the third dimension direction, the second dimension direction, the first dimension direction, and the batch direction. For example, when the first dimension is the number of channels C, the second dimension is the height H, and the third dimension is the width W, the data arrangement format of the multiple elements in the first data combination in the first memory is NCHW.
[0044] For example, for the first data combination arranged as NCHW in the first order, in the first memory, starting from the starting element, the elements in the row where the starting element is located are arranged in sequence along the W direction, and then move to the next row along the H direction to arrange the elements of the row in sequence along the W direction... After the elements in the H×W plane where the starting element is located are arranged, if multiple elements in the first data combination have not been arranged, move to the next H×W plane along the C direction to arrange the multiple elements in the H×W plane in sequence in the aforementioned manner... After all elements in the batch N where the starting reading position is located are arranged, if multiple elements in the first data combination have not been arranged, move to the next batch along the N direction to arrange the multiple elements in the batch in sequence in the aforementioned manner.
[0045] It should be noted that the arrangement of the first data combination in the first memory is not limited to the above description, and other options may be made according to actual needs, which is not limited in the embodiments of the present disclosure.
[0046] For example, in step S110, when reading n elements in the first data combination in the first dimensional direction, the starting reading position (for example, the position of the starting element) is first determined, and then the n elements are sequentially read from the starting reading position along the first dimensional direction (for example, the C direction). For example, the n elements read in the first dimensional direction can be used as the first processing group, and the 1st element to the nth element in the first processing group are arranged in the first dimensional direction. For example, the first processing group can include only the first dimension, the second dimension, and the third dimension, the first dimension can be the number of channels C, the second dimension can be the height H, and the third dimension can be the width W.
[0047] In some examples, the first dimension of the first processing group may be the number of elements n contained in the first processing group, the second dimension may be 1, and the third dimension may be the number of sub-elements m contained in each element, where m is a positive integer; that is, the first dimension of each element in the first processing group may be 1, the second dimension of each element in the first processing group may be 1, and the third dimension of each element in the first processing group may be m. In some examples, m=n.
[0048] For example, in step S120, when the first dimension of the first processing group is n and the third dimension W is the number of sub-elements contained in each element, after transposing the first dimension and the third dimension of the first processing group, the first dimension of the second processing group obtained is the number of sub-elements m contained in each element, and the third dimension is the number of elements n contained in the second processing group, that is, the 1st to nth elements in the second processing group are arranged in the direction of the third dimension, and the first dimension and the third dimension of each element in the second processing group are also transposed compared with each element in the first processing group.
[0049] In some examples, the first dimension of the second processing group may be the number m of sub-elements contained in each element, the second dimension may be 1, and the third dimension may be the number n of elements contained in the second processing group; that is, the first dimension of each element in the second processing group is m, the second dimension of each element in the second processing group is 1, and the third dimension of each element in the second processing group is 1. In some examples, m=n.
[0050] For example, in step S130, n elements in the second processing group are written into corresponding storage locations in the first memory. In some examples, the storage locations of the n elements in the first memory are the same as the arrangement locations in the second processing group.
[0051] In some examples, the data conversion method may further include step S140:
[0052] Step S140: reading the first data combination multiple times to obtain multiple first processing groups.
[0053] For example, in step S140, the first data combination can be read multiple times in the manner of step S110, and n elements can be read each time, so as to obtain multiple first processing groups, wherein each first processing group includes n elements arranged in the direction of the first dimension; in some examples, after reading the first first processing group, the second first processing group can be read in the first order, and then the third first processing group can be read in the first order... until the first data combination is read. For example, further, the first dimension and the third dimension can be transposed on the multiple first processing groups respectively in the manner of step S120 to obtain multiple corresponding second processing groups, wherein each second processing group includes n elements arranged in the direction of the third dimension.
[0054] In some examples, the data conversion method may further include step S150:
[0055] Step S150: writing a plurality of second processing groups into the first memory to obtain a second data combination.
[0056] For example, in step S150, multiple second processing groups may be written into the first memory multiple times in the manner of step S130 to obtain a second data combination. For example, when the first data combination is read multiple times in the manner of step S110, if all elements in the first data combination are read, that is, the multiple first processing groups include all elements in the first data combination, then the multiple second processing groups obtained by transposing in the manner of step S120 include transposed elements of all elements in the first data combination; therefore, after the multiple second processing groups are written into the first memory, the number of elements included in the obtained second data combination is the same as the number of elements included in the first data combination.
[0057] In some examples, the multiple elements in the second data combination are stored in the first memory in a second order, and the second order may be a priority arrangement order in the first dimension direction, the third dimension direction, the second dimension direction, and the batch direction. For example, when the first dimension is the number of channels C, the second dimension is the height H, and the third dimension is the width W, the data arrangement format of the multiple elements in the second data combination in the first memory is NHWC.
[0058] For example, for the second data combination arranged as NHWC in the second order, in the first memory, starting from the starting element, the elements in the row where the starting element is located are arranged sequentially along the C direction, and then move to the next row along the W direction to arrange the elements of the row sequentially along the C direction... After the C×W plane where the starting write position is located is filled, if multiple elements in the second data combination have not been arranged, move to the next C×W plane along the H direction to continue to arrange the multiple elements in the C×W plane sequentially in the aforementioned manner... After all elements in the batch N where the starting write position is located are arranged, if multiple elements in the second data combination have not been arranged, move to the next batch along the N direction to arrange the multiple elements in the batch sequentially in the aforementioned manner.
[0059] It should be noted that the arrangement of the second data combination in the first memory is not limited to the above description, and other options may be made according to actual needs, which is not limited in the embodiments of the present disclosure.
[0060] In some examples, the data arrangement format of the first data combination is NCHW, and after conversion in steps S110 to S150, the data arrangement format of the second data combination obtained is NHWC; in other examples, the data arrangement format of the first data combination and the data arrangement format of the second data combination obtained after conversion may also be other types, which can be selected according to actual needs, and the embodiments of the present disclosure are not limited to this.
[0061] For example, one can Figure 2 The data conversion device 110 shown is implemented as follows Figure 3 For example, the first data combination to be read may be stored in a first memory (e.g., internal memory). In step S110, the read DMA 111 reads n elements of the first data combination from the first memory into the buffer 113 through the bus interface 120 to obtain a first processing group. In step S120, the second memory (e.g., internal memory) Figure 2 The buffer 113 in the first processing group performs a transposition on the first dimension and the third dimension of the first processing group to obtain a second processing group; in step S130, the write DMA 112 writes the 1st element to the nth element in the second processing group into the first memory through the bus interface 120.
[0062] For example, further, for example Figure 2 The data conversion device 110 shown can also implement steps S140 and S150; in step S140, the read DMA 111 can read the first data combination stored in the first memory multiple times, and read n elements in the first data combination from the first memory into the buffer 113 multiple times to obtain multiple first processing groups; in step S150, the write DMA 112 writes multiple second processing groups into the first memory, thereby obtaining converted second data combinations.
[0063] It should be noted that the first memory is a memory, and the second memory is a buffer (for example, a ping-pang buffer, including two cache units, which are used for alternating reading and writing). The first memory and the second memory can also select other types according to actual conditions, and the embodiments of the present disclosure are not limited to this.
[0064] It should be noted that in addition to using Figure 2 The data conversion device 110 shown may also use other types of devices or equipment to execute the data conversion method provided by at least one embodiment of the present disclosure. The specific selection may be based on actual needs, and the embodiments of the present disclosure are not limited to this.
[0065] Figure 4 A schematic diagram of an example of a data conversion method provided by at least one embodiment of the present disclosure. For example, Figure 4 for Figure 3 A specific example of the data conversion method in .
[0066] For example, the first data combination is as follows Figure 4As shown in the lower left, the first data combination to be read is stored in a first storage device (such as a memory); the first data combination includes N batches, and the data corresponding to each batch includes multiple channels, and the first dimension is the number of channels C; in each channel, multiple elements are arranged in two dimensions to form an array (i.e., an H×W plane), the second dimension is the height H of the data in each channel in the vertical dimension (i.e., the number of elements in the array along the column direction), and the third dimension is the width W of the data in each channel in the horizontal dimension (i.e., the number of elements in the array along the row direction). For example, in Figure 4 In the example, N=1, C=64, H=4, and W=2.
[0067] For example, Figure 4 As shown in the lower left, the first data combination includes 512 elements, and the number on each element indicates the storage position of the element in the first memory, that is, the data arrangement format of the 512 elements in the first data combination in the first memory is NCHW. For example, specifically, according to the priority arrangement order of the third dimension W direction, the second dimension H direction, the first dimension C direction and the batch N direction (i.e., the first order), starting from the starting element (i.e., storage position 1), the elements of the row where the starting element is located are arranged in sequence along the W direction (i.e., storage position 1, storage position 2), and then move to the next row along the H direction to arrange the elements of the row in sequence along the W direction (i.e., storage position 3, storage position 4)... After the elements in the H×W plane where the starting element is located (i.e., storage position 1 to storage position 8) are arranged, move to the next H×W plane along the C direction to arrange multiple elements in the array in the aforementioned manner (i.e., storage position 9 to storage position 16)... until 512 elements are arranged.
[0068] For example, Figure 4 As shown, the number of sub-elements contained in each element is 32 (i.e., m=32), and multiple sub-elements are arranged along the direction of the third dimension W'; that is, the first dimension C' and the second dimension H' of each element are both 1, and the third dimension W' is 32 (i.e., C'×H'×W'=1×1×32).
[0069] For example, according to Figure 3 In step S110, 32 elements (ie, n=32) in the first data combination may be read in the direction of the first dimension C, that is, Figure 4 As shown, the storage locations 1, 9, 17 ... 249 in the first memory are read in sequence. For example, the 32 elements (i.e., elements 1 to 32) read in the first order can be used as the first processing group; the first processing group is as follows Figure 4 As shown in the upper left (i.e., the shaded portion), the 1st to 32nd elements in the first processing group are arranged in the direction of the first dimension C. For example, Figure 4The first processing group shown may only include the first dimension C1, the second dimension H1 and the third dimension W1, and the first dimension C1 of the first processing group may be the number of elements contained in the first processing group 32 (i.e., n=32), the second dimension H1 may be 1, and the third dimension W1 may be the number of sub-elements contained in each element 32 (i.e., m=32), i.e., C1×H1×W1=32×1×32. For example, Figure 4 As shown, the number of elements included in the first processing group is n=32, and the number of sub-elements contained in each element is m=32, that is, m=n.
[0070] For example, according to Figure 3 In step S120, the first dimension C1 and the third dimension W1 of the first processing group are transposed to obtain a second processing group. For example, the second processing group is as follows: Figure 4 As shown in the upper right corner (i.e., the shaded part), the 1st to 32nd elements in the second processing group are arranged in the direction of the third dimension W, and the first dimension C2 of the second processing group is the number of sub-elements contained in each element 32 (i.e., m=32), the second dimension H2 can be 1, and the third dimension W2 is the number of elements contained in the second processing group 32 (i.e., n=32), that is, C2×H2×W2=32×1×32.
[0071] For example, Figure 4 As shown, the first dimension C" of each element in the second processing group is 32, and the second dimension H" and the third dimension W" are both 1 (i.e., C"×H"×W"=32×1×1), that is, the first dimension C" and the third dimension W" of each element in the second processing group are also transposed compared to the first dimension C' and the third dimension W' of each element in the first processing group.
[0072] For example, according to Figure 3 In step S130, the first to the 32nd elements in the second processing group are written into corresponding storage locations in the first memory; a second data group obtained after writing multiple second processing groups into the first memory is as follows Figure 4 As shown in the lower right, the number on each element of the second data combination represents the storage location of the element in the first memory. Figure 4 As shown, according to Figure 3 In step S130, the 32 elements are sequentially written into storage locations 1, 3, 5, 7, ..., 63 in the first memory, that is, the storage locations of the 32 elements in the first memory are the same as the arrangement locations in the second processing group.
[0073] For example, Figure 4 As shown in the upper left, according to step S140, elements 1 to 32 in the first data combination are Figure 3After completing the conversion in steps S110 to S130, the next group of elements (i.e., elements 33 to 64) can be processed according to Figure 3 The conversion is performed in the manner of steps S110 to S130 until all elements of the first data combination are converted, thereby obtaining Figure 4 The converted second data combination is shown at the lower right.
[0074] For example, specifically, in step S140, the first data combination can be read multiple times in the manner of step S110. After reading the first first processing group, the second first processing group can be read in the first order, and then the third first processing group can be read in the first order... until the first data combination is read; 32 elements are read each time to obtain 16 first processing groups, wherein each first processing group includes 32 elements arranged in the direction of the first dimension C.
[0075] For example, further, the first dimension and the third dimension can be transposed on the 32 first processing groups respectively in the manner of step S120 to obtain 32 corresponding second processing groups, wherein each second processing group includes 32 elements arranged in the direction of the third dimension W.
[0076] For example, further, in step S150, multiple second processing groups may be written into the first memory multiple times in the manner of step S130, thereby obtaining a second data combination. Figure 4 As shown in the lower right, the number of elements included in the obtained second data combination is the same as the number of elements included in the first data combination.
[0077] For example, Figure 4 As shown, the data arrangement format of the first data combination in the first memory is NCHW, and after the conversion in steps S110 to S150, the data arrangement format of the second data combination in the first memory is NHWC.
[0078] For example, Figure 4As shown in the lower right corner, in the first memory, for the second data combination in the NHWC format, starting from the starting element (i.e., storage position 1), the elements in the row where the starting element is located are arranged in sequence along the C direction (i.e., storage position 1, storage position 2), and then move to the next row along the W direction to arrange the elements of the row in sequence along the C direction (i.e., storage position 3, storage position 4)... After the C×W plane where the starting write position is located (i.e., storage position 1 to storage position 128) is filled, move to the next C×W plane along the H direction to continue to arrange multiple elements in the C×W plane in sequence in the aforementioned manner (i.e., storage position 129 to storage position 256)... until 512 elements are arranged.
[0079] For example, for Figure 4 For example, each time a request is made to read 32 elements in the first data combination, the requested read address RD_addr can be expressed by the following formula (1):
[0080] RD_addr=datain_addr+C×IN_PLANAR_STRIDE+H×IN_LINE_STRIDE+W×2 (1)
[0081] Where datain_addr indicates the starting address of the data requested to be read (for example Figure 4 The address of the lower left element 1), IN_PLANAR_STRIDE represents the spacing of each H×W plane in the C direction in the first memory, and IN_LINE_STRIDE represents the spacing of each line of data in the W direction in the first memory. For example, as shown in formula (1), the read address RD_addr is equal to the sum of the starting address datain_addr, the product of C and the H×W plane spacing (planar-stride), the product of H and the W direction line spacing (line-stride), and W and the space occupied by each element (for example, 2 bytes for int16 data precision).
[0082] For example, for Figure 4 For example, each time a request is made to write 32 elements in the second processing group into the first memory, the requested write address WR_addr can be expressed by the following formula (2):
[0083] WR_addr=dataout_addr+W×OUT_PLANAR_STRIDE+H×OUT_LINE_STRIDE+C×2(2)
[0084] Where dataout_addr indicates the starting address of the data requested to be written (for example Figure 4 The address of the lower right element 1), OUT_PLANAR_STRIDE represents the spacing of each H×C plane in the first memory in the W direction, and OUT_LINE_STRIDE represents the spacing of each line of data in the first memory in the C direction. For example, as shown in formula (2), the write address WR_addr is equal to the sum of the product of the starting address dataout_addr, W and the H×C plane spacing (planar-stride), H and the C direction line spacing (line-stride), and C and the space occupied by each element (for example, the data precision of int16 is 2 bytes).
[0085] It should be noted that Figure 4 The example shown takes the data precision of int16 as an example. The data conversion method provided in the embodiment of the present disclosure can also be used for other data precisions (for example, int8, fp64, fp32, fp16, etc.) according to actual needs, and the embodiment of the present disclosure is not limited to this.
[0086] The data conversion method provided by at least one embodiment of the present disclosure supports conversion between different data arrangement formats corresponding to different neural network frameworks (i.e., conversion between dimensions of the first data combination), and can offload the corresponding data conversion tasks from processing devices such as accelerators, thereby reducing the delay of data conversion and alleviating the burden on devices such as accelerators.
[0087] Figure 5 for Figure 3 An exemplary flowchart of an example of step S120 in FIG.
[0088] For example, in step S120, the first processing group is written into the second memory, and the second memory includes n cache lines; Figure 5 As shown, in order to perform a transposition on the first and third dimensions of the first processing group, Figure 3 Step S120 may include the following steps S121 to S122.
[0089] Step S121: shifting the first element to the nth element in the first processing group sequentially to write the first cache line to the nth cache line in n cache lines to obtain a cache data group, wherein the cache data group includes a plurality of diagonal lines;
[0090] Step S122: Read multiple diagonal rows in sequence to obtain a second processing group.
[0091] For example, in step S121, taking the number of elements included in the first processing group as n, and the number of sub-elements contained in each element as m, the first dimension of the first processing group is n, the second dimension is 1, and the third dimension is m. At this time, for example, the n×m sub-elements of the first processing group are written into n cache lines by row, each cache line includes m columns, and the n×m sub-elements written into the second memory are arranged in two dimensions to form a sub-element array of n rows and m columns as a cache data group. For example, when m=n, the cache data group includes 2n-1 diagonal rows.
[0092] For example, in step S121, taking the case where each element in the first processing group includes n sub-elements (i.e., m=n) as an example, first, the n sub-elements in the first element in the first processing group are written into the first cache line of the n cache lines, wherein the first sub-element to the nth sub-element in the first element are sequentially written into the first column to the nth column of the first cache line; then, the n sub-elements in the kth element in the first processing group are written into the kth cache line of the n cache lines, wherein the kth sub-element to the nth sub-element in the kth element are sequentially written into the kth cache line of the n cache lines. Write the 1st column to the n-k+1th column of the kth cache line, and the 1st sub-element to the k-1th sub-element of the kth element are written to the n-k+2th column to the nth column of the kth cache line, where k = 2, 3, ..., n-1; finally, write the n sub-elements of the nth element in the first processing group to the nth cache line of the nth cache line, wherein the nth sub-element of the nth element is written to the 1st column of the nth cache line, and the 1st sub-element to the n-1th sub-element of the nth element are written to the 2nd column to the nth column of the nth cache line. For example, through the above process, a cache data group can be obtained, which includes n×n sub-elements of n rows and n columns, and the multiple diagonal rows of the cache data group include the 1st diagonal row to the 2n-1th diagonal row.
[0093] For example, in step S122, taking the example that each element in the first processing group includes n sub-elements (i.e., m=n), first, read the k sub-elements in the k-th diagonal row among the multiple diagonal rows and the nk sub-elements in the n+k-th diagonal row among the multiple diagonal rows as the k-th element in the second processing group, where k=1, 2, ..., n-1; then, read the n sub-elements in the n-th diagonal row among the multiple diagonal rows as the n-th element in the second processing group. For example, through the above process, a second processing group is obtained, and the first dimension of the second processing group is m, the second dimension is 1, and the third dimension is n.
[0094] For example, since the 1st to nth elements in the first processing group are arranged in the direction of the first dimension, correspondingly, the 1st to mth sub-elements in each element are arranged in the direction of the third dimension, the process of shifting and writing the n elements in the first processing group into n cache lines in sequence is the process of shifting and writing n cache lines in sequence in the direction of the first dimension, and the process of sequentially reading multiple diagonal rows is the process of reading and obtaining the n elements of the second processing group in the direction of the third dimension. Therefore, the 1st to nth elements in the second processing group obtained by reading are arranged in the direction of the third dimension, and the 1st to mth sub-elements in each element are arranged in the direction of the first dimension, that is, the first dimension and the third dimension of the second processing group are transposed compared with the first processing group.
[0095] Figure 6 A schematic diagram of another example of a data conversion method provided by at least one embodiment of the present disclosure. For example, Figure 6 for Figure 3 or Figure 5 A specific example of the data conversion method shown.
[0096] For example, Figure 6 As shown, the first data combination is stored in the first memory. Figure 3 In step S110, the first element to the nth element in the first data combination are read out in a first order as a first processing group, and the n elements in the first processing group are written into the second memory respectively; Figure 3 In step S120, the second memory performs transposition on the first dimension and the third dimension of the first processing group to obtain a second processing group. For example, the first dimension of the first processing group is, for example, the number of channels C1, and the third dimension is, for example, the width W1. After the processing of step S120, the first dimension C2 and the third dimension W2 of the second processing group are transposed compared with the first dimension C1 and the third dimension W1 of the first processing group (i.e., C1W1=>W2C2); Figure 3 In step S130, the 1st element to the nth element in the second processing group are written into the first memory according to the second sequence.
[0097] For example, in Figure 6 In the example of , the first memory may be a memory, and the second memory may be a buffer (eg, a ping-pong buffer). Figure 6 As shown, the second memory can be composed of multiple SRAMs, and each column of the second memory is an SRAM, which can independently control the read and write addresses of each position. Since the space occupied by SRAM is small, compared with registers with large area overhead, SRAM is more suitable for format conversion of large-scale data, thereby saving space and improving data conversion efficiency.
[0098] For example, since for each SRAM, only one sub-element at a position can be read at a time, the sub-elements in each column of the second memory (i.e., the sub-elements at all positions of an SRAM) cannot be read at the same time; therefore, the transposition between rows and columns can be completed by circular shift writing and oblique reading. Figure 5 Steps S121 to S122 in FIG. 1 are an example of circular shift writing and oblique reading; when writing by row, the addresses of the sub-elements in each cache row are the same, and when reading by oblique direction, the addresses of the sub-elements in each diagonal row are different.
[0099] For example, Figure 6 As shown, multiple elements in the first processing group complete the transposition of the first dimension and the third dimension by circular shift writing and oblique reading. Figure 6 In the example, the number of elements included in the first processing group is 16 (i.e., n=16), and the number of sub-elements contained in each element is also 16 (m=16), that is, the first processing group contains 16×16 sub-elements, and the first dimension C1=16, the second dimension H1=1, and the third dimension W1=16 of the first processing group.
[0100] For example, Figure 6 As shown, according to Figure 5 In step S121, the 1st to 16th elements in the first processing group are shifted and written into the 1st to 16th cache lines in sequence to obtain a cache data group; that is, the 16×16 sub-elements of the first processing group are written into the 16 cache lines of the second memory in rows, and each row is shifted by an additional length of 1 sub-element than the previous row. For example, each cache line includes 16 columns, and the 16×16 sub-elements written into the second memory are arranged in two dimensions to form a sub-element array of 16 rows and 16 columns as a cache data group, and the cache data group includes 31 diagonal rows. For example, further, according to Figure 5 In step S122, 31 diagonal rows are read in sequence to obtain a second processing group.
[0101] For example, Figure 6 As shown, according to Figure 5 In step S121, the specific process of cyclic shift writing is as follows:
[0102] (1) For the first element in the first treatment group (e.g. Figure 6 The example element read from the first memory, cache line 1 is as follows Figure 6 As shown in the first row of the second memory on the left), its 16 sub-elements are written into the first column to the 16th column of the first cache row in sequence, that is, no shifting is required when writing the first element;
[0103] (2) For the kth element in the first processing group (the kth cache line is as follows Figure 6, where k=2,3,…,15, write its kth to 16th sub-elements into the 1st to 17-k columns in sequence, and write its 1st to k-1th sub-elements into the 18-k to 16th columns of the kth cache row (taking the 3rd element in the first processing group as an example, write its 3rd to 16th sub-elements into the 1st to 14th columns in sequence, and write its 1st to 2nd sub-elements into the 15th to 16th columns of the 3rd cache row);
[0104] (3) For the 16th element in the first processing group (the 16th cache line Figure 6 The 16th sub-element is written into the 1st column of the 16th cache row, and the 1st to 15th sub-elements are written into the 2nd to 16th columns of the 16th cache row.
[0105] For example, Figure 6 As shown in FIG. 1 , the above process is equivalent to shifting the sub-elements of the dotted line portion on the left side of each row of the second memory to the position indicated by the arrow on the right side. For example, the cache data group obtained by the above process is as follows: Figure 6 As shown in the solid line portion of the second memory in FIG. 1 , the cache data group includes 16 rows and 16 columns of 16×16 sub-elements, and the multiple diagonal rows of the cache data group include the 1st diagonal row to the 31st diagonal row.
[0106] For example, Figure 6 As shown, according to Figure 5 In step S122, the specific process of the exemplary oblique reading is as follows:
[0107] (1) Read the k-th sub-element in the k-th diagonal row and the 16-k-th sub-element in the 16+k-th diagonal row as the k-th element in the second processing group, where k = 1, 2, ..., 15 (in Figure 6 Taking the sub-elements marked by the shaded part in the second memory on the right as an example, 14 sub-elements in the 14th diagonal row and 2 sub-elements in the 30th diagonal row are read as the 14th element in the second processing group);
[0108] (2) Read the 16 sub-elements in the 16th diagonal row as the 16th element in the second processing group.
[0109] For example, through the above process, a transposed second processing group is obtained. For example, after the 16 elements in the first processing group are shifted and written into 16 cache lines in sequence, the addresses of the sub-elements in each cache line are the same, while the addresses of the sub-elements in each diagonal row are different when read obliquely. For example, since the 1st to 16th elements in the first processing group are arranged in the direction of the first dimension, correspondingly, the 1st to 16th sub-elements in each element are arranged in the direction of the third dimension, according to the address correspondence between the sub-elements written and read, the 1st to 16th elements in the second processing group obtained by reading are arranged in the direction of the third dimension, while the 1st to 16th sub-elements in each element are arranged in the direction of the first dimension. Therefore, the first dimension and the third dimension of the second processing group are transposed compared to the first processing group.
[0110] In the data conversion method provided in the embodiment of the present disclosure, if Figure 5 or Figure 6 The embodiment shown can accomplish Figure 3 Step S120 can also be used in other application scenarios that require transposition of two-dimensionally arranged data. Therefore, the data conversion method provided by at least one embodiment of the present disclosure, in addition to supporting the conversion between different data arrangement formats corresponding to different neural network frameworks (for example, NCHW=>NHWC), can also be used to achieve the transposition of two-dimensionally arranged data, and thus further used for the transposition of row-major sequence and column-major sequence in matrix multiplication operations (for example, height×width=>width×height) or other application scenarios, and the embodiments of the present disclosure are not limited to this.
[0111] It should be noted that Figure 3 to Figure 6 The data arrangement format (type of dimension or arrangement method), the number of elements included, and the corresponding conversion method of the first data combination, the first processing group, the second processing group, and the second data combination described are only exemplary. Other arrangement forms or corresponding conversion methods may be selected according to actual needs. The embodiments of the present disclosure are not limited to this.
[0112] Figure 7 A schematic block diagram of a data conversion device provided in at least one embodiment of the present disclosure.
[0113] For example, Figure 7 As shown, the data conversion device 200 is used to convert the dimension of the first data combination. For example, the data conversion device 200 includes a reading module 210, a transposition module 220 and a writing module 230. For example, the first data combination includes at least one batch, and the dimensions of the data corresponding to each batch include a first dimension, a second dimension and a third dimension. In some examples, the first dimension is the number of channels, the second dimension is the height, and the third dimension is the width.
[0114] For example, the reading module 210 is configured to read n elements in the first data combination in a first order to obtain a first processing group. For example, the first element to the nth element in the first processing group are arranged in a first dimension direction, where n is a positive integer. That is, the acquisition module 210 can be configured to perform, for example Figure 3 Step S110 is shown.
[0115] For example, the transposition module 220 is configured to perform a transposition on the first dimension and the third dimension of the first processing group to obtain a second processing group. For example, the first element to the nth element in the second processing group are arranged along the third dimension. That is, the processing module 220 can be configured to perform, for example Figure 3 Step S120 is shown.
[0116] For example, the writing module 230 is configured to write the first element to the nth element in the second processing group into the first memory. That is, the calculation module 230 can be configured to perform, for example Figure 3 Step S130 is shown.
[0117] For example, the reading module 210 is further configured to read the first data combination multiple times to obtain multiple first processing groups. That is, the reading module 210 can be configured to perform step S140.
[0118] For example, the writing module 230 is further configured to write the plurality of second processing groups into the first memory to obtain a second data combination. For example, the number of elements included in the second data combination is the same as the number of elements included in the first data combination. That is, the writing module 230 can be configured to perform step S150.
[0119] In some examples, multiple elements in the first data combination are stored in the first memory in a first order, and the first order is a priority arrangement order of the third dimension direction, the second dimension direction, the first dimension direction, and the batch direction. In some examples, multiple elements in the second data combination are stored in the first memory in a second order, and the second order is a priority arrangement order of the first dimension direction, the third dimension direction, the second dimension direction, and the batch direction.
[0120] In some examples, the first dimension of each element in the first processing group is 1, the second dimension of each element in the first processing group is 1, and the third dimension of each element in the first processing group is m; the first dimension of each element in the second processing group is m, the second dimension of each element in the second processing group is 1, and the third dimension of each element in the second processing group is 1, where m is a positive integer. In some examples, m=n.
[0121] In some examples, the first processing group is written into the second memory, and the second memory includes n cache lines. For example, the transposition module 220 can also be configured to sequentially shift the first element to the nth element in the first processing group and write them into the first cache line to the nth cache line in the n cache lines to obtain a cache data group, for example, the cache data group includes a plurality of diagonal lines; and sequentially read the plurality of diagonal lines to obtain a second processing group. In some examples, the first memory is a memory, and the second memory is a buffer.
[0122] In some examples, each element in the first processing group includes n sub-elements; the transposition module 220 can also be configured to: write the n sub-elements of the first element in the first processing group to the first cache line of the n cache lines, wherein the first sub-element to the nth sub-element of the first element are written to the first column to the nth column of the first cache line in sequence; write the n sub-elements of the kth element in the first processing group to the kth cache line of the n cache lines, wherein the kth sub-element to the nth sub-element of the kth element are written to the kth cache line of the n cache lines. The 1st column to the n-k+1th column of the kth cache row, and the 1st sub-element to the k-1th sub-element of the kth element are written into the n-k+2th column to the nth column of the kth cache row, where k=2,3,…,n-1; the n sub-elements of the nth element in the first processing group are written into the nth cache row of the n cache rows, where the nth sub-element of the nth element is written into the 1st column of the nth cache row, and the 1st sub-element to the n-1th sub-element of the nth element are written into the 2nd column to the nth column of the nth cache row.
[0123] In some examples, each element in the second processing group includes n sub-elements, the cache data group includes n×n sub-elements of n rows and n columns, and the multiple diagonal rows include the 1st diagonal row to the 2n-1th diagonal row; the transposition module 220 can also be configured to: read k sub-elements in the kth diagonal row among the multiple diagonal rows and nk sub-elements in the n+kth diagonal row among the multiple diagonal rows as the kth element in the second processing group, where k=1,2,…,n-1; read n sub-elements in the nth diagonal row among the multiple diagonal rows as the nth element in the second processing group.
[0124] Since in the above description, for example Figure 3 In the process of the data conversion method shown in FIG. 1 , the details of the operation of the data conversion device 200 have been introduced, so for the sake of brevity, they will not be repeated here. For the relevant details, please refer to the above Figure 1 to Figure 6 Description.
[0125] It should be noted that Figure 7The above modules in the data conversion device 200 shown can be configured as software, hardware, firmware or any combination of the above items to perform specific functions. For example, these modules can correspond to dedicated integrated circuits, pure software codes, or modules that combine software and hardware. Figure 7 The described device may be a PC computer, a tablet device, a personal digital assistant, a smart phone, a web application, or other device capable of executing program instructions, but is not limited thereto.
[0126] In addition, although the data conversion device 200 is divided into modules for respectively performing corresponding processing in the above description, it is clear to those skilled in the art that the processing performed by each module can also be performed without any specific module division in the device or without clear demarcation between the modules. Figure 7 The described data conversion device 200 is not limited to including the modules described above, but some other modules (for example, a storage module, a control module, etc.) may be added as needed, or the above modules may also be combined.
[0127] At least one embodiment of the present disclosure also provides another data conversion device, which includes a processor and a memory; the memory includes one or more computer program modules; the one or more computer program modules are stored in the memory and are configured to be executed by the processor, and the one or more computer program modules include a method for implementing the data conversion method provided by the embodiment of the present disclosure described above.
[0128] Figure 8 A schematic block diagram of another data conversion device provided for at least one embodiment of the present disclosure.
[0129] For example, Figure 8 As shown, the data conversion device 300 includes a processor 310 and a memory 320. For example, the memory 320 is used to store non-transitory computer-readable instructions (e.g., one or more computer program modules). The processor 310 is used to run non-transitory computer-readable instructions, and when the non-transitory computer-readable instructions are run by the processor 310, one or more steps of the data conversion method described above can be executed. The memory 320 and the processor 310 can be interconnected through a bus system and / or other forms of connection mechanisms (not shown).
[0130] For example, the processor 310 may be a digital signal processor (DSP) or other forms of processing units with data conversion capability and / or program execution capability, such as X86, ARM architecture or field programmable gate array (FPGA), etc. The processor 310 may be a general-purpose processor or a dedicated processor, and may control other components in the data conversion device 300 to perform desired functions.
[0131] For example, the memory 320 may include any combination of one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disk read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer program modules may be stored on the computer-readable storage medium, and the processor 310 may run one or more computer program modules to implement various functions of the data conversion device 300. Various applications and various data, as well as various data used and / or generated by the application, etc. may also be stored in the computer-readable storage medium.
[0132] It should be noted that, in the embodiments of the present disclosure, the specific functions and technical effects of the data conversion device 300 can refer to the above description of the data conversion method provided by at least one embodiment of the present disclosure, and will not be repeated here.
[0133] At least one embodiment of the present disclosure further provides an electronic device, which includes a data processing device and a data conversion device provided by any embodiment of the present disclosure.
[0134] Fig. 9 A schematic block diagram of an electronic device provided for at least one embodiment of the present disclosure.
[0135] For example, Fig. 9 As shown, the electronic device 400 includes a data conversion device 410 and a data processing device 420. For example, the electronic device 400 may be a neural network accelerator or other AI accelerator, or may be any device such as a mobile phone, a tablet computer, a laptop computer, an e-book, a game console, a television, a digital photo frame, a navigator, or any combination of electronic devices and hardware, which is not limited in the embodiments of the present disclosure.
[0136] For example, the data conversion device 410 may be a data conversion device provided by any embodiment of the present disclosure, and the data processing device 420 may be a module in a neural network accelerator for data processing (e.g., data transmission, convolution calculation, etc.), including but not limited to a processor, a controller, a memory, a bus or other electronic devices for data processing.
[0137] Fig.10 A schematic block diagram of another electronic device provided for at least one embodiment of the present disclosure.
[0138] For example, Fig.10 As shown, the electronic device 500 is suitable for implementing the data conversion method provided by the embodiment of the present disclosure. It should be noted that, Fig.10 The electronic device 500 shown is only an example and does not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0139] For example, Fig.10 As shown, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 51, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 52 or a program loaded from a storage device 58 into a random access memory (RAM) 53. In the RAM 53, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 51, the ROM 52, and the RAM 53 are connected to each other through a bus 54. An input / output (I / O) interface 55 is also connected to the bus 54. Generally, the following devices can be connected to the I / O interface 55: an input device 56 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 57 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 58 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 59. The communication device 59 can allow the electronic device 500 to communicate with other electronic devices wirelessly or by wire to exchange data.
[0140] Although Fig.10 The electronic device 500 is shown to have various devices, but it should be understood that it is not required to implement or possess all the devices shown, and the electronic device 500 may alternatively implement or possess more or fewer devices.
[0141] It should be noted that, for the sake of clarity and simplicity, the embodiments of the present disclosure do not provide all the components of the electronic device 400 / 500. To achieve the necessary functions of the electronic device, those skilled in the art may provide and set other components not shown according to specific needs, and the embodiments of the present disclosure are not limited to this.
[0142] For the detailed description and technical effects of the electronic device 400 / 500, please refer to the above description of the data conversion method, which will not be repeated here.
[0143] Fig.11 A schematic diagram of a storage medium provided for at least one embodiment of the present disclosure.
[0144] For example, Fig.11As shown, the storage medium 600 stores non-transitory computer-readable instructions 610. For example, when the non-transitory computer-readable instructions 610 are executed by a computer, one or more steps in the data conversion method described above are performed.
[0145] For example, the storage medium 600 can be applied to Figure 8 For example, the storage medium 600 may be the memory 320 in the data conversion device 300. For example, the description of the storage medium 600 may refer to Figure 8 The corresponding description of the memory 320 in the data conversion device 300 is not repeated here.
[0146] There are a few points to note about this disclosure:
[0147] (1) In the drawings of the embodiments of the present disclosure, only the structures related to the embodiments of the present disclosure are involved, and other structures can refer to the general design.
[0148] (2) In the absence of conflict, features in the same embodiment or in different embodiments of the present disclosure may be combined with each other.
[0149] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.
Claims
1. A data conversion method, for converting the dimension of a first data combination, wherein: The first data combination includes at least one batch, the dimensions of data corresponding to each batch in the at least one batch include a first dimension, a second dimension, and a third dimension, and the data conversion method includes: Reading n elements in the first data combination along a first dimension direction to obtain a first processing group, wherein the first element to the nth element in the first processing group are arranged along the first dimension direction, wherein n is a positive integer; Transposing the first dimension and the third dimension of the first processing group to obtain a second processing group, wherein the first element to the nth element in the second processing group are arranged along the third dimension; Write the 1st to nth elements in the second processing group into the first memory.
2. The data conversion method according to claim 1, wherein: The multiple elements in the first data combination are stored in the first memory in a first order, where the first order is a priority arrangement order of the third dimension direction, the second dimension direction, the first dimension direction, and the batch direction.
3. The data conversion method according to claim 1, further comprising: The first data combination is read multiple times to obtain multiple first processing groups.
4. The data conversion method according to claim 1, further comprising: writing a plurality of the second processing groups into the first memory to obtain a second data combination, The number of elements included in the second data combination is the same as the number of elements included in the first data combination.
5. The data conversion method according to claim 4, wherein: The multiple elements in the second data combination are stored in the first memory in a second order, where the second order is a priority arrangement order of the first dimension direction, the third dimension direction, the second dimension direction, and the batch direction.
6. The data conversion method according to claim 1, wherein: The first dimension is the number of channels, the second dimension is the height, and the third dimension is the width.
7. The data conversion method according to claim 1, wherein: The first dimension of each element in the first processing group is 1, the second dimension of each element in the first processing group is 1, and the third dimension of each element in the first processing group is m, The first dimension of each element in the second processing group is m, the second dimension of each element in the second processing group is 1, and the third dimension of each element in the second processing group is 1, Wherein, m is a positive integer.
8. The data conversion method according to claim 7, wherein: m=n.
9. The data conversion method according to claim 1, wherein: The first processing group is written into a second memory, the second memory comprising n cache lines, Transposing the first dimension and the third dimension of the first processing group to obtain the second processing group includes: Shifting and writing the first element to the nth element in the first processing group into the first cache line to the nth cache line in sequence, to obtain a cache data group, wherein the cache data group includes a plurality of diagonal lines; The plurality of diagonal rows are read sequentially to obtain the second processing group.
10. The data conversion method according to claim 9, wherein: Each element in the first processing group includes n sub-elements, Shifting the first element to the nth element in the first processing group sequentially into the first cache line to the nth cache line in the n cache lines, comprising: Writing n sub-elements of the first element in the first processing group into the first cache line of the n cache lines, wherein the first to nth sub-elements of the first element are sequentially written into the first to nth columns of the first cache line; Writing n sub-elements of the kth element in the first processing group into the kth cache line of the n cache lines, wherein the kth to nth sub-elements of the kth element are written into the 1st column to the n-k+1th column of the kth cache line, and the 1st to k-1th sub-elements of the kth element are written into the n-k+2th column to the nth column of the kth cache line, wherein k=2, 3, ..., n-1; Writing n sub-elements of an nth element in the first processing group into an nth cache line among the n cache lines, wherein the nth sub-element of the nth element is written into the first column of the nth cache line, and the first to n-1th sub-elements of the nth element are written into the second to nth columns of the nth cache line.
11. The data conversion method according to claim 9, wherein: Each element in the second processing group includes n sub-elements, The cache data group includes n×n sub-elements of n rows and n columns, the plurality of diagonal rows include the 1st diagonal row to the 2n-1th diagonal row, The plurality of diagonal rows are read sequentially to obtain the second processing group, comprising: Read the k sub-elements in the k-th diagonal row of the plurality of diagonal rows and the nk sub-elements in the n+k-th diagonal row of the plurality of diagonal rows as the first sub-elements in the second processing group. k elements, where k = 1, 2, ..., n-1; Read n sub-elements in an n-th diagonal row of the plurality of diagonal rows as an n-th element in the second processing group.
12. The data conversion method according to claim 9, wherein: The first memory is a memory, and the second memory is a buffer.
13. A data conversion device, for converting the dimension of a first data combination, wherein: The first data combination includes at least one batch, and the dimensions of the data corresponding to each batch in the at least one batch include a first dimension, a second dimension, and a third dimension. The data conversion device comprises: a reading module configured to read n elements in the first data combination along a first dimension to obtain a first processing group, wherein the first element to the nth element in the first processing group are arranged along the first dimension, wherein n is a positive integer; a transposition module configured to perform transposition on the first dimension and the third dimension of the first processing group to obtain a second processing group, wherein the first element to the nth element in the second processing group are arranged along the third dimension; A writing module is configured to write the 1st element to the nth element in the second processing group into the first memory.
14. The data conversion device according to claim 13, wherein: The multiple elements in the first data combination are stored in the first memory in a first order, where the first order is a priority arrangement order of the third dimension direction, the second dimension direction, the first dimension direction, and the batch direction.
15. The data conversion device according to claim 13, wherein: The reading module is further configured to read the first data combination multiple times to obtain multiple first processing groups.
16. The data conversion device according to claim 13, wherein: The writing module is further configured to write a plurality of the second processing groups into the first memory to obtain a second data combination, The number of elements included in the second data combination is the same as the number of elements included in the first data combination.
17. The data conversion device according to claim 13, wherein: The multiple elements in the second data combination are stored in the first memory in a second order, where the second order is a priority arrangement order of the first dimension direction, the third dimension direction, the second dimension direction, and the batch direction.
18. A data conversion device, comprising: processor; a memory including one or more computer program modules; The one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules are used to implement the data conversion method according to any one of claims 1 to 12.
19. An electronic device comprising: The data conversion device according to any one of claims 13 to 18; as well as A data processing device is configured to process the first data combination.
20. A storage medium storing non-transitory computer-readable instructions, which implement the data conversion method according to any one of claims 1 to 12 when executed by a computer.
Citation Information
Cited By
Data conversion method and apparatus, electronic device, and storage medium
EP4462263A1
Data conversion method and apparatus, electronic device, and storage medium
EP4462263B1
