Data storage method, data processing method, data storage device, data processor

Abstract

A data storage method, a data processing method, a data storage device, a data processor and a storage medium. The data storage method comprises: for each batch of matrix data, dividing the matrix data into at least one data block, wherein each data block comprises m rows and n columns of elements, wherein m and n are positive integers; storing at least one data block in a data storage device in units of data blocks, wherein elements in the same data block are stored continuously in the data storage device, and elements belonging to the same data block correspond to the same storage address in the data storage device. The data storage method provides a new storage method, which implements the storage of matrix data in units of data blocks, and each data block corresponds to a storage address, so that the data of the entire data block can be extracted using one storage address, which greatly improves the bandwidth and efficiency when loading data, reduces the complexity of address calculation, and thus improves the utilization rate of the computing unit.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to a data storage method, a data processing method, a data storage device, a data processor, and a non-transitory computer-readable storage medium. Background Art

[0002] A tensor is a multilinear map defined on the Cartesian product of some vector space and some dual space. For example, a scalar can be viewed as a 0-dimensional tensor, a vector can be viewed as a 1-dimensional tensor, and a matrix can be viewed as a 2-dimensional tensor. Tensor operations are widely used in processors such as parallel processors.

[0003] With the development of artificial intelligence and machine learning, new requirements have been put forward for many parallel processor devices represented by parallel processors (for example, multi-core processors, digital signal processors, etc.). In general computing, the computing unit of a parallel processor requires a large amount of data, and these data are generally stored in the storage component of the parallel processor, such as a memory. Through data loading instructions, these data can be extracted from the storage component to the computing unit for calculation. Summary of the invention

[0004] At least one embodiment of the present disclosure provides a data processing method, including: a data storage method for storing tensor data, wherein the tensor data includes P batches of matrix data, each batch of matrix data includes M rows and N columns of elements, P, M and N are positive integers, and the data storage method includes: for each batch of matrix data, dividing the matrix data into at least one data block, wherein each data block includes m rows and n columns of elements, wherein m and n are positive integers; storing the at least one data block in a data memory in units of data blocks, wherein elements in the same data block are stored continuously in the data memory, and elements belonging to the same data block correspond to the same storage address in the data memory.

[0005] For example, in the data processing method provided in at least one embodiment of the present disclosure, the matrix data is divided into at least one data block, including: dividing the matrix data into at least one data block according to the data bit width of the elements, wherein the sizes of the data blocks obtained by dividing the matrix data with different data bit widths of the elements are different and the total amount of data is the same.

[0006] For example, in the data processing method provided in at least one embodiment of the present disclosure, the matrix data is divided into at least one data block according to the data bit width of the element, including: determining a data block template corresponding to the data bit width according to the data bit width, wherein the data block template indicates the size of the data block obtained by dividing the matrix data of the data bit width; dividing the matrix data according to the data block template to obtain the at least one data block, wherein the union of the at least one data block is the matrix data, and the at least one data block does not overlap with each other when divided.

[0007] For example, in the data processing method provided by at least one embodiment of the present disclosure, in response to the data bit width being 8 bits, the data block template indicates that the number of rows m of the data block is 2×a, and the number of columns n is 2×b; in response to the data bit width being 16 bits, the data block template indicates that the number of rows m of the data block is 2×a, and the number of columns n is b; in response to the data bit width being 32 bits, the data block template indicates that the number of rows m of the data block is a, and the number of columns n is b, wherein a and b are positive integers.

[0008] For example, in the data processing method provided in at least one embodiment of the present disclosure, the at least one data block is stored in a data storage device in units of data blocks, including: storing the at least one data block in the data storage device in sequence according to row priority or column priority in units of data blocks; and for each data block: determining a storage address corresponding to the data block; and storing the elements in the data block in sequence according to a preset storage order in a storage space corresponding to the storage address.

[0009] For example, in the data processing method provided by at least one embodiment of the present disclosure, the preset storage order is indicated by a data block template determined according to the data bit width of the element.

[0010] For example, in the data processing method provided by at least one embodiment of the present disclosure, the data block template indicates that the data block includes multiple data sub-blocks, the multiple data sub-blocks are obtained by evenly dividing the data block, each data sub-block includes multiple data units, each data unit includes at least one element, the preset storage order indicates that the multiple data sub-blocks are stored in the data storage device in sequence along the column direction, the multiple data units in each data sub-block are stored in the data storage device in sequence along the row direction, and at least one element in each data unit is stored in the data storage device in sequence according to the preset element order indicated by the data block template.

[0011] For example, in the data processing method provided by at least one embodiment of the present disclosure, in response to the data bit width being 16 bits, the data block template indicates that the data block includes a data sub-blocks, each data sub-block is composed of 2 adjacent rows and n columns of elements, each data sub-block includes n data units, each data unit includes 2 elements in the data sub-block located in the same column, and at least one element in each data unit is stored in the data storage device in sequence according to the preset element order indicated by the data block template, including: storing the 2 elements in each data unit in the data storage device in sequence in the column direction, where a is a positive integer and m=2×a.

[0012] For example, in the data processing method provided by at least one embodiment of the present disclosure, in response to the data bit width being 8 bits, the data block template indicates that the data block includes a data sub-blocks, each data sub-block is composed of 2 adjacent rows and n columns of elements, each data sub-block includes b data units, each data unit includes two adjacent rows and two columns of elements in the data sub-block, and at least one element in each data unit is stored in the data storage device in sequence according to the preset element order indicated by the data block template, including: storing the 4 elements in each data unit in the data storage device in sequence according to row priority or column priority, where a and b are positive integers, m=2×a, n=2×b.

[0013] For example, in the data processing method provided by at least one embodiment of the present disclosure, in response to the data bit width being 32 bits, the data block template indicates that the data block includes m data sub-blocks, each data sub-block consists of 1 row and n column elements, each data unit includes 1 element, and multiple data units in each data sub-block are stored in the data storage device in sequence along the row direction, and at least one element in each data unit is stored in the data storage device in sequence according to the preset element order indicated by the data block template, including: storing n elements in each data sub-block in the data storage device in sequence along the row direction.

[0014] For example, in the data processing method provided in at least one embodiment of the present disclosure, in response to storing the at least one data block in the data memory in sequence according to row priority order, the storage addresses of two data blocks adjacent in the row direction in the matrix data in the data memory are continuous.

[0015] For example, in the data processing method provided by at least one embodiment of the present disclosure, storing the at least one data block in the data storage device in sequence according to row priority includes storing the at least one data block in the data storage device first in a row direction and then in a column direction, wherein the at least one data block is arranged in the form of G rows and Q columns, and the storage addresses of the last data block stored in the data storage device in the i-th row of the G rows are continuous with the storage addresses of the first data block stored in the data storage device in the i+1-th row, wherein G, Q and i are all positive integers, and i is less than or equal to G.

[0016] For example, in the data processing method provided in at least one embodiment of the present disclosure, in response to storing the at least one data block in the data memory in sequence according to column priority order, the storage addresses of two data blocks adjacent in the column direction in the matrix data in the data memory are continuous.

[0017] For example, in the data processing method provided by at least one embodiment of the present disclosure, storing the at least one data block in the data storage device in sequence according to column priority includes storing the at least one data block in the data storage device first in the column direction and then in the row direction, wherein the at least one data block is arranged in the form of G rows and Q columns, and the storage addresses of the last data block stored in the data storage device in the jth column of the Q column and the first data block stored in the data storage device in the j+1th column are continuous, wherein P, Q and j are all positive integers.

[0018] For example, in the data processing method provided by at least one embodiment of the present disclosure, before dividing the matrix data into at least one data block, the data storage method further includes: in response to M and m not being an integer multiple relationship: filling a plurality of elements with a first value outside the boundary of the matrix data to increase the number of rows of the matrix data to M', wherein, In response to the fact that N and n are not in an integer multiple relationship: filling a plurality of elements having a first value outside the boundary of the matrix data to increase the number of columns of the matrix data to N', wherein, in, Represents the ceiling function.

[0019] At least one embodiment of the present disclosure provides a data processing method, comprising: receiving a data loading instruction, wherein the data loading instruction comprises a storage address of tensor data to be loaded, the tensor data comprises P batches of matrix data, each batch of matrix data comprises M rows and N columns of elements, P, M and N are positive integers, and executing the data loading instruction after parsing the data loading instruction, wherein executing the data loading instruction comprises: determining the storage addresses of each data block corresponding to the tensor data, wherein each data block comprises m rows and n columns of elements, the each data block is obtained by dividing the tensor data, elements belonging to the same data block correspond to the same storage address in a data memory, and m and n are positive integers; extracting data from the data memory using the storage address of each data block in units of data blocks to obtain the tensor data, wherein elements of the same data block are stored continuously in the data memory.

[0020] At least one embodiment of the present disclosure provides a data storage device for storing tensor data, wherein the tensor data includes P batches of matrix data, each batch of matrix data includes M rows and N columns of elements, P, M and N are positive integers, and the data storage device is configured to store the tensor data in units of data blocks, wherein each data block includes m rows and n columns of elements, elements in the same data block are stored consecutively in the data storage device, and elements belonging to the same data block correspond to the same storage address in the storage device.

[0021] For example, in a data processor provided in at least one embodiment of the present disclosure, the total amount of data in each data block is the same and is determined by the storage capacity of a storage unit corresponding to a single storage address in the data storage.

[0022] At least one embodiment of the present disclosure provides a data processor, comprising a data storage as described in any embodiment of the present disclosure.

[0023] At least one embodiment of the present disclosure provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the data storage method according to any embodiment of the present disclosure or the data processing method according to any embodiment of the present disclosure is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, but are not intended to limit the present disclosure.

[0025] Figure 1 A schematic diagram of the structure of a general-purpose graphics processing unit (GPGPU);

[0026] Figure 2 A schematic diagram of data for matrix calculation;

[0027] Figure 3 A schematic flow chart of a data storage method provided for at least one embodiment of the present disclosure;

[0028] Figure 4 A schematic diagram of data block templates corresponding to different data bit widths provided by at least one embodiment of the present disclosure;

[0029] Figure 5A A schematic diagram of the relationship between matrix data and data blocks provided in at least one embodiment of the present disclosure;

[0030] Figure 5B A schematic diagram of a row priority order provided for at least one embodiment of the present disclosure;

[0031] Figure 5C A schematic diagram of column priority provided for at least one embodiment of the present disclosure;

[0032] Figure 6 A schematic diagram of a data block storage order provided for at least one embodiment of the present disclosure;

[0033] Figure 7 A schematic flow chart of a data processing method provided for at least one embodiment of the present disclosure;

[0034] Figure 8 A schematic flow chart of a data processing method provided for at least one embodiment of the present disclosure;

[0035] Fig. 9 A schematic block diagram of a data processor provided for at least one embodiment of the present disclosure;

[0036] Fig.10 A schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0038] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits the detailed description of some known functions and known components.

[0039] Figure 1 A structural diagram of a general-purpose graphics processing unit (GPGPU).

[0040] In parallel computing, computing tasks are generally executed by multiple threads. Figure 1 As shown, before these threads are executed in the general-purpose graphics processor (or parallel computing processor), they are divided into multiple thread blocks in the thread block scheduling module, and then distributed to each computing unit (Compute Unit, referred to as CU) through the thread block distribution module. All threads in a thread block must be assigned to the same computing unit for execution. At the same time, the thread block will be split into minimum execution thread bundles (or thread warps for short), each of which contains a fixed number (or less than this fixed number) of threads, for example, 32 threads. Multiple thread blocks can be executed in the same computing unit, or in different computing units.

[0041] In each computing unit, the thread warp scheduling / distribution module schedules and distributes the thread warps so that multiple computing cores of the computing unit can run the thread warps. Each computing core includes an arithmetic logic unit (ALU), a floating-point computing unit, etc. Depending on the number of computing cores in the computing unit, multiple thread warps in a thread block can be executed simultaneously or in time-sharing. Multiple threads in each thread warp will execute the same instructions. The reading, decoding, and emission of instructions are all completed in the thread warp scheduling / distribution module. Memory execution instructions will be emitted to the shared cache in the computing unit (such as the shared L1 cache) or further emitted to the unified cache for read and write operations, etc.

[0042] General computing operations, such as matrix computing operations, usually require a large amount of data, which is usually stored in Figure 1 The memory shown, for example, can be stored in HBM (High Bandwidth Memory), and data needs to be loaded from the memory (Load operation) when performing general computing operations. The storage method of data in the memory affects the memory access bandwidth, and further affects the hardware utilization of the computing unit.

[0043] The data that needs to be calculated by matrix can be expressed as [N, H, W], where N represents the batch size, that is, the number of data samples captured in one training, H represents the height of the matrix, and W represents the width of the matrix. For example, if the data for matrix calculation is a picture, N = 1, H represents the height of the picture, and W represents the width of the picture.

[0044] Matrix calculations may include general matrix multiplication (GEMM). For example, in neural network processing (e.g., convolutional neural network) often used in fields such as artificial intelligence, it is often necessary to perform matrix multiplication and accumulation (MACC) calculations, which include multiplying elements at corresponding positions in two matrices and then accumulating the multiplication results to obtain a calculation result.

[0045] Figure 2 is a schematic diagram of data for matrix calculation. Figure 2 As shown, there are N batches of data that need to be calculated by matrix, such as Figure 2 In the figure, there are multiple large rectangular boxes batch0, batch1, ... batch(N-1). A batch includes H×W elements. Figure 2 A small grid in a batch represents an element. For example, an element can have a data width of 8 bits, 16 bits, etc.

[0046] For example, taking HBM as an example, a storage unit in the memory can store 512B (Byte, byte) of data, for example, a storage unit indicated by a storage address can store 512B of data. A data load instruction (used for load operation) needs to load 512B of data from the memory, for example, the 512B data is Figure 2 The bold rectangle in the figure is composed of 4 rows and 32 columns of elements, and the bit width of each element is 32 bits.

[0047] Currently, data for matrix calculations are usually stored in memory in NHW or NWH formats. For NHW, the elements are arranged in memory in row-major order, and for NWH, the elements are arranged in memory in column-major order.

[0048] like Figure 2 The data shown is stored in the memory in NHW mode, and the elements in the same row have the same address in the memory, for example, the first row elements correspond to address 0 in the memory, the second row elements correspond to address 1 in the memory, the third row elements correspond to address 2 in the memory, and the fourth row elements correspond to address 3 in the memory. Figure 2 When there are 4 rows and 32 columns of elements, the Load instruction for loading 512B of data needs to be decomposed into 4 requests to extract all 4 rows of data. Specifically, when extracting data, address 0, address 1, address 2, and address 3 are calculated, and the 4 addresses are sent to the memory with 4 requests. Each request is used to extract a row of elements, and the returned data is also divided into 4 records, each of which includes a row of elements, totaling 128B.

[0049] Similarly, if Figure 2 The data shown is stored in the memory in NWH mode. Elements in the same column correspond to one address in the memory. Figure 2 As shown in the figure, for the 4 rows and 32 columns of elements, the instruction to load 512B of data needs to be decomposed into 32 requests. When extracting data, 32 addresses are calculated (one address for each column of elements). The 32 addresses are sent to the memory with 32 requests. Each request is used to extract a column of elements. The returned data will also be divided into 32 records, each of which includes a column of elements.

[0050] Due to the data characteristics of matrix calculation itself, it is always composed of multiple rows and columns of elements. Therefore, when performing matrix calculations, the current data storage method always requires calculation and sending multiple requests to load data, and the amount of data returned by each request is also small, which greatly reduces the data bandwidth when loading data and reduces the hardware computing efficiency.

[0051] At least one embodiment of the present disclosure provides a data storage method, a data processing method, a data storage device, a data processor, and a non-transitory computer-readable storage medium.

[0052] The data storage method comprises: for each batch of matrix data, dividing the matrix data into at least one data block, wherein each data block comprises m rows and n columns of elements, wherein m and n are positive integers; taking data blocks as units, storing at least one data block into a data storage device, wherein elements in the same data block are stored continuously in the data storage device, and elements belonging to the same data block correspond to the same storage address in the data storage device.

[0053] Different from the traditional NHW or NWH storage method, this data storage method provides a new storage method, which implements the storage of matrix data in data blocks. Each data block corresponds to a storage address, so that the data of the entire data block can be extracted using one storage address, which greatly improves the bandwidth and efficiency during data loading, reduces the complexity of address calculation, and thus improves the utilization rate of the computing unit.

[0054] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings, but the present disclosure is not limited to these specific embodiments.

[0055] Figure 3 A schematic flowchart of a data storage method provided for at least one embodiment of the present disclosure.

[0056] like Figure 3 As shown, the data storage method provided by at least one embodiment of the present disclosure includes at least steps S10-S20.

[0057] For example, a data storage method provided by at least one embodiment of the present disclosure is used to store tensor data.

[0058] For example, the tensor data includes P batches of matrix data, each batch of matrix data includes M rows and N columns of elements, where P, M and N are positive integers.

[0059] For example, the dimension of the tensor data is 3. For example, the tensor data can be expressed as [P, M, N]. As before, the tensor data can include P batches, each batch including M rows and N columns of elements.

[0060] For example, the tensor data may be a weight, such as the tensor data includes a weight value of a convolutional layer, or the tensor data may be input data, such as including input data of an input convolutional layer.

[0061] In at least one embodiment of the present disclosure, the shape and size of the tensor data can be set as needed, and the present disclosure does not impose any specific limitation on this.

[0062] like Figure 3 As shown, in step S10, for each batch of matrix data, the matrix data is divided into at least one data block.

[0063] For example, each data block includes m rows and n columns of elements, where m and n are positive integers.

[0064] In step S20, at least one data block is stored in a data storage device in units of data blocks.

[0065] For example, elements in the same data block are stored consecutively in the data memory, and elements belonging to the same data block correspond to the same storage address in the data memory.

[0066] For example, the data storage may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disk read-only memory (CD-ROM), USB memory, flash memory, etc.

[0067] For example, the data storage device may be a high bandwidth memory. The present disclosure does not impose any specific restrictions on the form, structure, application environment, etc. of the data storage device, and those skilled in the art may select the data storage device according to their needs.

[0068] For matrix data, since the matrix data is divided into data blocks whose elements are arranged in the form of an array of m rows and n columns, the data blocks are stored in the data memory, and the elements belonging to the same data block are stored continuously in the data memory, and the elements belonging to the same data block correspond to the same storage address in the data memory. Different from the current storage of matrix data in row priority or column priority order, the data storage method provided by at least one embodiment of the present disclosure has a larger granularity when storing matrix data, and does not require each row to correspond to a storage address. The data of the entire data block can be extracted using one storage address. This storage method is more suitable for the storage of tensor data, greatly improving the bandwidth and efficiency when loading data, reducing the complexity of address calculation, and thus improving the utilization rate of the computing unit.

[0069] The processes of step S10 and step S20 in the data storage method provided by at least one embodiment of the present disclosure are described in detail below with reference to the accompanying drawings.

[0070] For example, step S10 may include: dividing the matrix data into at least one data block according to the data bit width of the element.

[0071] For example, the total amount of data in data blocks obtained by dividing matrix data with different data bit widths of elements is the same, but the sizes of the data blocks are different.

[0072] For example, the total amount of data in a data block is determined by the storage capacity of a storage unit corresponding to a single storage address in the data storage device. Taking HBM as an example, the storage capacity of a storage unit corresponding to a single storage address in HBM is 512B. Therefore, for matrix data with different data bit widths, they can be divided into multiple data blocks in units of 512B, and the total amount of data in each data block is 512B.

[0073] Since the data bit widths of the elements are different, when the total amount of data in the data blocks is the same, the sizes of the data blocks obtained by dividing the matrix data with different data bit widths must be different.

[0074] For example, the data formats of the elements are different, but the data bit widths are the same, for example, INT16 and FP16, the data formats are different but the bit widths are both 16 bits, then the sizes of the data blocks obtained by dividing the data formats are the same.

[0075] For example, in at least one embodiment, dividing the matrix data into at least one data block according to the data bit width of the element may include: determining a data block template corresponding to the data bit width according to the data bit width, wherein the data block template indicates the size of the data block obtained by dividing the matrix data of the data bit width; dividing the matrix data according to the data block template to obtain at least one data block, wherein the union of at least one data block is the matrix data, and at least one data block does not overlap with each other when divided.

[0076] For example, data block templates for different data bit widths can be pre-set. The data block templates can include provisions for the size of data blocks obtained by dividing matrix data of the data bit width. For example, the data block module can specify the number of rows m and columns n of data blocks obtained by dividing matrix data of a certain data bit width.

[0077] For example, the size of the data block can be determined according to the data bit width and the storage capacity of the storage unit corresponding to a single storage address in the data storage device. For example, the product of m and n can be determined according to the quotient of the storage capacity of a single storage unit and the data bit width, and the specific values ​​of m and n can be set as needed, as long as their product is the calculated quotient.

[0078] For example, when the matrix data is divided according to the data block template, the union of all the data blocks obtained by the division is the matrix data, and these data blocks do not overlap each other when divided. Here, the data blocks do not overlap each other means that each element of the matrix data is only divided into one data block, and the union of all the data blocks is the matrix data, which means that each element in the matrix data has been divided into a certain data block, and there is no undivided element.

[0079] For example, in one example, in response to the data bit width of the element being 8 bits, the data block template indicates that the number of rows m of the data block is 2×a, and the number of columns n is 2×b; in response to the data bit width of the element being 16 bits, the data block template indicates that the number of rows m of the data block is 2×a, and the number of columns n is b; in response to the data bit width of the element being 32 bits, the data block template indicates that the number of rows m of the data block is a, and the number of columns n is b, where a and b are positive integers.

[0080] For example, the specific values ​​of a and b may be different depending on the storage capacity of a single storage unit in the data storage. The following takes the storage capacity of a single storage unit in the data storage as 512B as an example to specifically describe the shape and size of a data block in an example.

[0081] Figure 4 A schematic diagram of data block templates corresponding to different data bit widths provided in at least one embodiment of the present disclosure.

[0082] exist Figure 4 In the figure, data block template 1 is a data block template corresponding to a data bit width of 32 bits, data block template 2 is a data block template corresponding to a data bit width of 16 bits, and data block template 3 is a data block template corresponding to a data bit width of 8 bits.

[0083] Figure 4 Each small grid in the data block template represents an element. For example, a small grid in data block template 1 represents an element with a data bit width of 32 bits, a small grid in data block template 2 represents an element with a data bit width of 16 bits, and a small grid in data block template 3 represents an element with a data bit width of 8 bits.

[0084] For example, although the data bit widths corresponding to the three data block templates are different, the total amount of data is the same. Figure 4 In the example, the total amount of data in each data block is 512B. Figure 4 It can be seen that the sizes of data blocks in different data block templates are different.

[0085] like Figure 4 As shown, for data block template 1, which corresponds to an element with a data bit width of 32 bits, data block template 1 indicates that the number of rows m and the number of columns n of a single data block obtained by dividing the matrix data with an element data bit width of 32 bits is 4, and the number of columns n is 32.

[0086] like Figure 4 As shown, for data block template 2, which corresponds to an element with a data bit width of 16 bits, data block template 2 indicates that the number of rows m and the number of columns n of a single data block obtained by dividing the matrix data with an element data bit width of 16 bits is 8, and the number of columns n is 32.

[0087] like Figure 4As shown, for data block template 3, which corresponds to an element with a data bit width of 8 bits, data block template 3 indicates that the number of rows m and the number of columns n of a single data block obtained by dividing the matrix data with an element data bit width of 8 bits is 8, and the number of columns n is 64.

[0088] In this embodiment, data block templates corresponding to different data bit widths are as similar as possible in a certain dimension, and the relationship between the number of rows and the number of columns is more balanced, and the division is more reasonable.

[0089] Of course, it should be noted that Figure 4 The figure shows a definition of a data block template. Those skilled in the art may also set the data block to other shapes and sizes, and the present disclosure does not impose any specific limitation on this.

[0090] For example, after obtaining the data block template, the matrix data may be divided according to the data block template. For example, the elements of m rows and n columns indicated by the data block template are regarded as one data block, and the matrix data is divided into a plurality of data blocks.

[0091] For example, in one example, reference Figure 4 As shown in the data block template, if the matrix data includes 16 rows and 128 columns of elements and the data bit width of the elements is 32 bits, it can be divided into 4 data blocks, the 1st to 8th rows and the 1st to the 64th columns in the matrix data are used as data block 1, the 9th to the 16th rows and the 1st to the 64th columns in the matrix data are used as data block 2, the 1st to the 8th rows and the 65th to the 128th columns in the matrix data are used as data block 3, and the 9th to the 16th rows and the 65th to the 128th columns in the matrix data are used as data block 4.

[0092] For example, in some embodiments, the size of the matrix data may not be an exact integer multiple of the data block template. In this case, the matrix data needs to be border-filled to expand the size of the matrix data to an integer multiple of the size of the data block template, thereby ensuring that the union of the divided data blocks is the matrix data and the data blocks do not overlap when divided.

[0093] For example, in some embodiments, before dividing the matrix data into at least one data block, the data storage method further includes: in response to M and m not being an integer multiple relationship: filling a plurality of elements having a first value around the boundary of the matrix data to increase the number of rows of the matrix data to M', wherein, Represents the ceiling function.

[0094] For example, in some embodiments, before dividing the matrix data into at least one data block, the data storage method further includes: in response to N and n not being an integer multiple relationship: filling a plurality of elements having a first value around the boundary of the matrix data to increase the number of columns of the matrix data to N', wherein, Represents the ceiling function.

[0095] Afterwards, the matrix data after boundary filling is divided to obtain multiple data blocks.

[0096] For example, the border peripheral filling may be filling on the left side, right side, top side or bottom side of the matrix data, and the filling direction is not specifically limited.

[0097] For example, in one example, the size of the matrix data is M=120, N=60, the size of the data block is m=32, n=16, M and m are not integer multiples, N and n are not integer multiples, and the matrix data needs to be border-filled.

[0098] For example, multiple elements with a first value may be filled around the upper or lower periphery of each column of the matrix data. For example, the first value may be 0 or another predetermined value. For example, 8 zeros may be filled around the upper boundary of each column of the matrix data to increase the number of rows of the matrix data to 1.

[0099]

[0100] For example, multiple elements with a first value may be filled on the left or right side of each row of the matrix data. For example, the first value may be 0 or another predetermined value. For example, four zeros may be filled on the right side of each row of the matrix data to increase the number of columns of the matrix data to 1.

[0101]

[0102] Thus, the matrix data is expanded from 120 rows and 60 columns to 128 rows and 64 columns. The size of the expanded matrix data is an integer multiple of the data block size, and complete division can be achieved.

[0103] For example, in at least one embodiment of the present disclosure, step S20 may include: taking data blocks as units, storing at least one data block in a data storage device in sequence according to row priority or column priority; and for each data block: determining a storage address corresponding to the data block; storing the elements in the data block in sequence according to a preset storage order in a storage space corresponding to the storage address.

[0104] When storing matrix data, it is stored in data blocks.

[0105] When storing multiple data blocks, the storage order of the multiple data blocks can be row-major order or column-major order.

[0106] For example, in at least one embodiment of the present disclosure, the row direction refers to the horizontal direction, and the column direction refers to the vertical direction. In addition, in the present disclosure, the upper left corner is used as the origin to describe the relevant position information in the matrix data or data block. For example, the first row and the first column position can refer to Figure 6 Examples are shown in .

[0107] For example, in at least one embodiment of the present disclosure, when at least one data block is sequentially stored in a data memory in a row priority order, two data blocks adjacent in the row direction in the matrix data have consecutive storage addresses in the data memory.

[0108] For example, in at least one embodiment of the present disclosure, the row priority order indicates that at least one data block is first stored in the data memory in the row direction and then in the column direction. For example, at least one data block is arranged in the form of G rows and Q columns, and the storage address of the last data block stored in the data memory in the i-th row of the G rows is continuous with the storage address of the first data block stored in the data memory in the i+1-th row. Here, G, Q and i are all positive integers, and i is less than or equal to G.

[0109] Figure 5A A schematic diagram of the relationship between matrix data and data blocks provided in at least one embodiment of the present disclosure.

[0110] like Figure 5A As shown, the matrix data can be divided into multiple data blocks through step S10, which are Figure 5A Data block B00, data block B01, ..., data block B0x, data block B10, data block B11, ... data block B1x, data block B20, data block B21, ... data block B2x, ..., data block By0, data block By1, ... data block Byx.

[0111] The specific division process can refer to the relevant description of the aforementioned embodiment, which will not be repeated here.

[0112] For example, Figure 5A As shown, multiple data blocks can be arranged in the form of G rows and Q columns, where G=x+1, Q=y+1.

[0113] For example, two data blocks adjacent to each other in the row direction in the matrix data, such as data block B00 and data block B01, have consecutive storage addresses in the data memory. Continuous data blocks in a row are adjacent to each other in the data memory.

[0114] Figure 5B A schematic diagram of row priority order provided for at least one embodiment of the present disclosure.

[0115] For example, in row-major order, Figure 5A The multiple data blocks shown are arranged into Figure 5B For example, Figure 5A The multiple data blocks in are arranged first in the row direction and then in the column direction to obtain Figure 5B Then, multiple data blocks are arranged in the order shown. Figure 5B The data blocks are sequentially stored in the data memory in the order shown, and the storage addresses of these data blocks are continuous. For example, in addition to the continuous storage addresses of the continuous data blocks in the same row in the data memory, the storage address of the last data block stored in the data memory in each row of the G row (for example, data block B0x) is also continuous with the storage address of the first data block stored in the data memory in the next row (for example, data block B10).

[0116] For example, in at least one embodiment of the present disclosure, when at least one data block is sequentially stored in a data memory in column priority order, two data blocks adjacent to each other in the column direction in the matrix data have consecutive storage addresses in the data memory.

[0117] For example, in at least one embodiment of the present disclosure, the column priority order indicates that at least one data block is first stored in the data memory in the column direction and then in the row direction. For example, at least one data block is arranged in the form of G rows and Q columns, and the storage addresses of the last data block stored in the data memory in the jth column of the Q column and the first data block stored in the data memory in the j+1th column are continuous, where P, Q and j are all positive integers.

[0118] 5C is a schematic diagram of column priority provided by at least one embodiment of the present disclosure.

[0119] For example, two data blocks adjacent to each other in the column direction in the matrix data, such as data block B00 and data block B10, have consecutive storage addresses in the data memory. Continuous data blocks in one column are adjacent to each other in the data memory.

[0120] For example, in column-major order, Figure 5A The multiple data blocks in are arranged into Figure 5C For example, in Figure 5C middle, Figure 5A The multiple data blocks in the first column direction, then the row direction. Then, the multiple data blocks are arranged in Figure 5C The data blocks are sequentially stored in the data memory in the order shown, and the storage addresses of these data blocks are continuous. For example, in addition to the continuous storage addresses of the continuous data blocks in the same column in the data memory being continuous, the storage address of the last data block stored in the data memory in each column in the Q column (for example, data block By0) is also continuous with the storage address of the first data block stored in the data memory in the next column (for example, data block B01).

[0121] In at least one embodiment of the present disclosure, the continuous storage addresses indicate that the difference between the two storage addresses is the storage capacity of the storage unit corresponding to the single storage address in the data storage device, and the two data blocks are adjacent to each other in the data storage device.

[0122] In the above embodiment, the storage address is set to be continuous, then the data loading instruction only needs to provide a starting address, such as the storage address of data block B00, and the request to extract the data of each data block can be directly calculated based on the storage address of data block B00, without transmitting the storage addresses of all data blocks, reducing the amount of data to be transmitted, and improving the data loading efficiency. Moreover, when the data blocks are stored in the data storage device according to the row priority or column priority, and the storage addresses of the data blocks are continuous according to the row priority or column priority, the returned data has a certain regularity, and the returned data can be directly spliced ​​into matrix data in the order specified by the row priority or column priority, thereby improving the data loading efficiency. Further, usually, when data is loaded, data of some storage addresses near the storage address to be loaded can be pre-fetched. If the storage addresses are continuous, the data hit rate can be improved when the data is loaded, and the data reading performance can be improved.

[0123] Of course, the storage order of data blocks can also be set to other forms, the storage addresses of data blocks can also be set to be discontinuous, or the storage addresses of adjacent data blocks in the row direction can be set to be continuous, and the storage addresses of the last data block stored in the data storage device in a row and the first data block stored in the data storage device in the next row can be set to be discontinuous, and the present disclosure does not make specific restrictions on this.

[0124] For a single data block, at least one embodiment of the present disclosure chooses to store the elements in the data block into a data memory in a preset storage order, and adjacent elements in the data block are also adjacent in the data memory, and the data in the single data block is continuous in the data memory.

[0125] For example, the preset storage order can be set as needed. For example, in at least one embodiment of the present disclosure, the preset storage order can be indicated by a data block template determined according to the data bit width of the element. Of course, the present disclosure is not limited to this, and other preset storage orders can be set as needed.

[0126] For example, the data block template may also specify the internal hierarchical structure of the data block and the preset storage order of the elements within the data block.

[0127] For example, the data block template indicates that the data block includes a plurality of data sub-blocks, the plurality of data sub-blocks are obtained by evenly dividing the data block, each data sub-block includes a plurality of data units, and each data unit includes at least one element.

[0128] For example, the preset storage order indicates that multiple data sub-blocks are stored in the data storage device in sequence in the column direction, multiple data units in each data sub-block are stored in the data storage device in the row direction, and at least one element in each data unit is stored in the data storage device in sequence according to the preset element order indicated by the data block template.

[0129] For example, in response to the data bit width being 16 bits, the data block template indicates that the data block includes a data sub-blocks, each data sub-block consists of 2 adjacent rows and n columns of elements, each data sub-block includes n data units, and each data unit includes 2 elements located in the same column in the data sub-block.

[0130] For example, at this time, the preset storage order indicates that a data sub-blocks are sequentially stored in the data memory in the column direction, n data units in each data sub-block are sequentially stored in the data memory in the row direction, and 2 elements in each data unit are sequentially stored in the data memory in the column direction. a is a positive integer, and m=2×a.

[0131] For example, in response to the data bit width being 8 bits, the data block template indicates that the data block includes a data sub-blocks, each data sub-block consists of 2 adjacent rows and n columns of elements, each data sub-block includes b data units, and each data unit includes two adjacent rows and two columns of elements in the data sub-block.

[0132] For example, at this time, the preset storage order indicates that a data sub-blocks are sequentially stored in the data memory in the column direction, b data units in each data sub-block are sequentially stored in the data memory in the row direction, and the 4 elements in each data unit are sequentially stored in the data memory in the row priority order or the column priority order. a and b are positive integers, m=2×a, n=2×b.

[0133] For example, storing four elements in a data memory in a row-first order includes storing the four elements in a row direction and then in a column direction. For example, storing four elements in a data memory in a column-first order includes storing the four elements in a column direction and then in a row direction.

[0134] For example, suppose that the coordinates of the four elements in the row direction and column direction are: element 1 (row = 0, col = 0), element 2 (row = 0, col = 1), element 3 (row = 1, col = 0), element 4 (row = 1, col = 1). At this time, when the four elements are sequentially stored in the data memory in row priority order, they are sequentially stored in the data memory in the order of element 1 -> element 2 -> element 3 -> element 4. When the four elements are sequentially stored in the data memory in column priority order, they are sequentially stored in the data memory in the order of element 1 -> element 3 -> element 2 -> element 4.

[0135] In response to the data bit width being 32 bits, the data block template indicates that the data block includes m data sub-blocks, each data sub-block consists of 1 row and n columns of elements, each data sub-block includes n data units, and each data unit includes 1 element.

[0136] For example, at this time, the preset storage order indicates that the m data sub-blocks are sequentially stored in the data storage device in the column direction, and the n data units (ie, n elements) in each data sub-block are sequentially stored in the data storage device in the row direction.

[0137] Figure 6 A schematic diagram of the data block storage order provided by at least one embodiment of the present disclosure. Figure 6 The preset storage order of elements in a data block provided by at least one embodiment of the present disclosure is specifically described.

[0138] exist Figure 6 In the figure, data block template 1 is a data block template corresponding to a data bit width of 32 bits, data block template 2 is a data block template corresponding to a data bit width of 16 bits, and data block template 3 is a data block template corresponding to a data bit width of 8 bits.

[0139] Figure 6 Each small grid in the data block template represents an element. For example, a small grid in data block template 1 represents an element with a data bit width of 32 bits, a small grid in data block template 2 represents an element with a data bit width of 16 bits, and a small grid in data block template 3 represents an element with a data bit width of 8 bits.

[0140] For example, Figure 6 As shown, for data block template 1, it includes 4 data sub-blocks, for example Figure 6 As shown in the data sub-block 1, data sub-block 2, data sub-block 3 and data sub-block 4. For example, a data block can be evenly divided into 4 parts to obtain 4 data sub-blocks, each data sub-block consists of a row of elements. For example, for data block template 1, a row of 32 elements is used as a data sub-block, and one element is used as a data unit.

[0141] For example, Figure 6 As shown, for data block template 2, it includes 4 data sub-blocks, for example Figure 6As shown in the data sub-block 5, data sub-block 6, data sub-block 7 and data sub-block 8. For example, a data block can be divided into 4 parts to obtain 4 data sub-blocks, each of which is composed of two adjacent rows of elements. For example, for data block template 2, two adjacent rows of elements are used as a data sub-block, for example, data sub-block 5 includes all elements in the first row and the second row of the data block, and data sub-block 6 includes all elements in the third row and the fourth row of the data block. For example, each data sub-block includes n data units, and each data unit includes 2 elements in the same column of the data sub-block. For example, for data sub-block 5, two elements in the first column are used as a data unit, and two elements in the second column are used as a data unit, and so on. There are 32 data units in data sub-block 5.

[0142] For example, Figure 6 As shown, for data block template 3, it also includes 4 data sub-blocks, such as Figure 6 As shown in the data sub-block 9, data sub-block 10, data sub-block 11 and data sub-block 12. For example, a data block can be divided into 4 parts to obtain 4 data sub-blocks, each of which is composed of two adjacent rows of elements. For example, for data block template 3, two adjacent rows of elements are used as a data sub-block, for example, data sub-block 9 includes the first row of elements and the second row of elements in the data block, and data sub-block 10 includes the third row of elements and the fourth row of elements in the data block. For example, each data sub-block includes n / 2 data units, and each data unit includes two adjacent rows and two columns of elements in the data sub-block. For example, for data sub-block 9, the four elements of the first row and the first column, the first row and the second column, the second row and the first column, and the second row and the second column are used as a data unit, and so on.

[0143] For example, Figure 6 As shown, data block template 1 indicates that the data block includes 4 data sub-blocks, each data sub-block consists of 1 row and 32 columns of elements, and each data unit includes 1 element, that is, 1 element is regarded as a data unit. For example, the 4 data sub-blocks can be stored in the data memory in the order of data sub-block 1, data sub-block 2, data sub-block 3, and data sub-block 4; for one data sub-block, 32 elements are stored in the data memory in the row direction.

[0144] Therefore, for a data block with a data bit width of 32 bits, the preset storage order (such as Figure 6The data block template in 1 (as shown by the arrow) can be: row(0)col(0)->row(0)col(1)->...->row(0)col(31)->row(1)col(0)->...->row(1)col(31)->row(2)col(0)->...->row(2)col(31)->row(3)col(0)->...->row(3)col(31), where row(0) represents the first row in the data block, row(1) represents the second row in the data block, row(2) represents the third row in the data block, row(3) represents the fourth row in the data block, col(0) represents the first column in the data block, col(1) represents the second column in the data block, ..., col(31) represents the 32nd column in the data block.

[0145] For example, Figure 6 As shown, for Figure 6 In the data block template 2, a=4, b=32, the data block template 2 indicates that the data block includes 4 data sub-blocks, each data sub-block consists of 2 rows and 32 columns of elements, and each data unit includes 2 elements located in the same column in the data sub-block. For example, the 4 data sub-blocks can be stored in the data memory in the order of data sub-block 5, data sub-block 6, data sub-block 7, and data sub-block 8; for a data sub-block, 32 data units are stored in the data memory in the row direction, and for a single data unit, two elements are stored in the data memory in the column direction.

[0146] Therefore, for a data block with a data bit width of 16 bits, the preset storage order (such as Figure 6 The data block template in 2 (as shown by the arrow) can be: row(0)col(0)->row(1)col(0)->row(0)col(1)->row(1)col(1)->...->row(0)col(31)->row(1)col(31)->row(2)col(0)->row(3)col(0)->row(2)col(1)->row(3)col(1)->...->row(2)col(31)->row(3)col(31)->row(4)col(0)->row(5)col(0)->row(4)col(1)->row(5)col(1)->...->row(4)col(31)->row(5)col(31)->row(6)col(0)->row(7)col(0)->row(6)col(1)->row(7)col(1)->...->row(6)col(31)->row(7)col(31).

[0147] For example, Figure 6 As shown, for Figure 6 In the data block template 3, a=4, b=32, the data block template 3 indicates that the data block includes 4 data sub-blocks, each data sub-block consists of 2 rows and 64 columns of elements, and each data unit includes 4 elements of two adjacent rows and two columns in the data sub-block. For example, the 4 data sub-blocks can be stored in the data memory in the order of data sub-block 9, data sub-block 10, data sub-block 11, and data sub-block 12; for one data sub-block, 32 data units are stored in the data memory in the row direction, and for a single data unit, 4 elements are stored in the data memory in the column priority order.

[0148] Therefore, for a data block with a data bit width of 8 bits, the preset storage order (such as Figure 6 The data block template in 3 (as shown by the arrow) can be: row(0)col(0)->row(1)col(0)->row(0)col(1)->row(1)col(1)->...->row(0)col(63)->row(1)col(63)->row(2)col(0)->row(3)col(0)->row(2)col(1)->row(3)col(1)->...->row(2)col(63)->row(3)col(63)->row(4)col(0)->row(5)col(0)->row(4)col(1)->row(5)col(1)->...->row(4)col(63)->row(5)col(63)->row(6)col(0)->row(7)col(0)->row(6)col(1)->row(7)col(1)->...->row(6)col(63)->row(7)col(63).

[0149] exist Figure 6 The embodiment shown provides a specific preset storage order, but the present disclosure is not limited thereto. The division method of the data sub-blocks, the preset storage order within the data sub-blocks, and the setting of the preset element order can be changed and adjusted as needed, and will not be repeated here.

[0150] For example, if P is greater than 1, then for each batch of matrix data, the above steps S10-S20 can be used to store a batch of matrix data in the data storage. For example, the above steps S10 and S20 are sequentially performed on P batches of matrix data until all P batches of matrix data are stored in the data storage.

[0151] For example, different batches of matrix data may be stored continuously or discontinuously in the data memory. For example, the storage address of the last storage block of the pth batch of matrix data stored in the data memory is, for example, addr0, and the storage address of the first storage block of the p+1th batch of matrix data stored in the data memory is, for example, addr1, where p is a positive integer and less than P, addr0 may be continuous with addr1, for example, addr1=addr0+data block size, or addr0 may be discontinuous with addr1, and the present disclosure does not impose any specific limitation on this.

[0152] At least one embodiment of the present disclosure provides a data storage method, in which tensor data can be stored in a data storage device in units of data blocks, and in the data storage device, elements in a single data block are arranged continuously, and the data of the entire data block can be read out at one time when the data is loaded. Compared with the traditional NWH or NHW storage method, the data loading efficiency is improved by several times, the data loading bandwidth is improved, the reading performance of the data storage device is improved, and the utilization rate of the computing unit is improved; and the storage addresses of the data blocks are continuous, which reduces the complexity of address calculation and improves data reading performance and loading efficiency.

[0153] At least one embodiment of the present disclosure also provides a data processing method. Figure 7 A schematic flowchart of a data processing method provided for at least one embodiment of the present disclosure.

[0154] For example, Figure 7 As shown, the data processing method provided by the embodiment of the present disclosure includes steps S30 to S40.

[0155] In step S30, a data loading instruction is received.

[0156] For example, the data loading instruction includes the storage address of the tensor data to be loaded, the tensor data includes P batches of matrix data, each batch of matrix data includes M rows and N columns of elements, and P, M and N are positive integers.

[0157] For the relevant description of tensor data, please refer to the relevant content in the aforementioned data storage method, and the repeated parts will not be repeated here.

[0158] In step S40, the data loading instruction is parsed and then executed.

[0159] For example, the data processing method provided by at least one embodiment of the present disclosure can be applied to Fig. 9 A data processor 200 is shown.

[0160] For example, in the data processing method provided by at least one embodiment of the present disclosure, a data loading instruction is provided, and the data loading instruction includes the storage address of the tensor data to be loaded. For example, after receiving the data loading instruction, the data processor parses the data loading instruction, such as decoding the data loading instruction, generates a microinstruction and sends the microinstruction to the instruction dispatch unit; the instruction dispatch unit sends the microinstruction to the corresponding scheduling queue according to the microinstruction category; in response to the microinstruction, after the storage address (all or a required part) of the tensor data to be loaded is prepared, the data loading instruction operation is executed.

[0161] Figure 8 A schematic flowchart of a data processing method provided for at least one embodiment of the present disclosure.

[0162] For example, Figure 8 As shown, the execution of the data loading instruction in step S40 in the data processing method provided by the embodiment of the present disclosure may include steps S401 to S402.

[0163] In step S401, the storage address of each data block corresponding to the tensor data is determined.

[0164] For example, each data block includes m rows and n columns of elements, where m and n are positive integers.

[0165] For example, each data block corresponding to the tensor data includes data blocks obtained by dividing each batch of matrix data in the tensor data. For example, each batch of matrix data is divided into at least one data block when stored in a data storage device. The specific division process can refer to the relevant content of step S10 in the aforementioned data storage method, which will not be repeated here.

[0166] For example, the storage addresses of the data blocks corresponding to the tensor data may be determined based on the storage addresses of the tensor data to be loaded.

[0167] For example, in some embodiments, the storage address of the tensor data to be loaded includes the storage addresses of each data block.

[0168] For example, in some other embodiments, there are some predetermined relationships between the storage addresses of each storage block. For example, the storage addresses of data blocks in a matrix data are continuous in row priority order or column priority order. Thus, the storage address of the tensor data to be loaded provides the storage address of the first data block in each matrix data, and the storage addresses of other data blocks can be calculated based on the storage address.

[0169] For example, in other embodiments, the storage addresses of data blocks in a matrix data are continuous in row priority order or column priority order, and there are also some predetermined relationships between the storage addresses of different batches of matrix data. For example, the storage address of the last data block stored in the data storage device in the current matrix data and the storage address of the first data block stored in the data storage device in the next matrix data stored in the data storage device are continuous. Thus, the storage address of the tensor data to be loaded provides the storage address of the first data block in the first matrix data stored in the data storage device, and the storage addresses of all other data blocks in the tensor data can be calculated based on the storage address.

[0170] For example, in some embodiments, the data loading instruction may also include the number of data blocks to be loaded, or the size of the tensor data to be loaded. The number of data blocks can be determined based on the size of the tensor data to be loaded and the data bit width of the elements, thereby determining how many data blocks need to be loaded and how many data blocks' storage addresses need to be calculated.

[0171] For example, elements belonging to the same data block correspond to the same storage address in the data memory. When storing tensor data, data blocks are used as units for storage, so elements in a data block correspond to the same storage address in the data memory.

[0172] In step S402, data is extracted from the data memory using the storage address of each data block in units of data blocks to obtain tensor data.

[0173] For example, elements of the same data block are stored consecutively in the data memory.

[0174] For example, data blocks are stored in a data storage device in row priority or column priority order. When extracting data, the extracted data blocks are also combined into a matrix data in row priority or column priority order, and multiple batches of matrix data are then combined into tensor data.

[0175] For the relevant contents about the storage of data blocks according to row priority and column priority, please refer to the relevant description of step S20 in the aforementioned data storage method, which will not be repeated here.

[0176] For example, for a single data block, the extracted data is rearranged in combination with a preset storage order of elements in the data block to restore it to the state of the single data block in the original matrix data.

[0177] For example, the preset storage order is indicated by a data block template determined according to the data bit width of the element.

[0178] For the preset storage order, please refer to the relevant description of step S20 in the aforementioned data storage method, which will not be repeated here.

[0179] In the data processing method provided by at least one embodiment of the present disclosure, the data of the entire data block can be read out at one time in units of data blocks when loading data. Compared with the traditional NWH or NHW storage method, the data loading efficiency is improved several times, the data loading bandwidth is improved, the reading performance of the data storage is improved, and the utilization rate of the computing unit is improved; and the storage addresses of the data blocks are continuous, which reduces the complexity of address calculation and improves data reading performance and loading efficiency.

[0180] Corresponding to the above data storage method, at least one embodiment of the present disclosure further provides a data storage device.

[0181] The data memory is used to store tensor data. For example, the tensor data includes P batches of matrix data, each batch of matrix data includes M rows and N columns of elements, and P, M and N are positive integers.

[0182] For the relevant description of tensor data, please refer to the relevant content in the aforementioned data storage method, and the repeated parts will not be repeated here.

[0183] The data memory is configured to store tensor data in units of data blocks, wherein each data block includes m rows and n columns of elements, elements in the same data block are stored consecutively in the data memory, and elements belonging to the same data block correspond to the same storage address in the data memory.

[0184] For example, the data blocks are obtained by dividing each batch of matrix data.

[0185] For example, the matrix data is divided into at least one data block according to the data bit width of the element. The sizes of the data blocks obtained by dividing the matrix data with different data bit widths are different, but the total amount of data in the data blocks is the same.

[0186] For example, when dividing, according to the data bit width, a data block template corresponding to the data bit width is determined, and the data block template indicates the size of the data block obtained by dividing the matrix data of the data bit width; then, the matrix data is divided according to the data block template to obtain at least one data block. For example, the union of at least one data block obtained by dividing a matrix data is the matrix data, and the at least one data block does not overlap each other when divided.

[0187] For a detailed description of data block division, please refer to the related description in the aforementioned data storage method, and the repeated parts will not be repeated here.

[0188] For example, when the data storage device executes the storage of matrix data in units of data blocks, the following operations are performed: in units of data blocks, at least one data block is sequentially stored in the data storage device in row priority order or column priority order; and for each data block: a storage address corresponding to the data block is determined; and elements in the data block are sequentially stored in a storage space corresponding to the storage address in a preset storage order.

[0189] For a detailed description of row priority and column priority, please refer to the related description in the aforementioned data storage method, and the repeated parts will not be repeated here.

[0190] For a single data block, the elements in the single data block are stored consecutively in the data memory, and the preset storage order followed during storage is indicated by the data block template determined according to the data bit width of the elements.

[0191] For the related description about the preset storage order, please refer to the related description in the aforementioned data storage method, and the repeated parts will not be repeated here.

[0192] The data storage device may include any storage device with a structure capable of storing data, such as a memory, a cache, etc. The present disclosure does not impose any specific restrictions on the hardware structure, performance, and principle of the data storage device, as long as the data storage function can be realized.

[0193] For example, in the data storage device provided by at least one embodiment of the present disclosure, tensor data can be stored in the data storage device in units of data blocks, and in the data storage device, the elements in a single data block are arranged continuously, so that the data of the entire data block can be read out at one time when the data is loaded. Compared with the traditional NWH or NHW storage method, the data loading efficiency is improved several times, the data loading bandwidth is improved, the reading performance of the data storage device is improved, and the utilization rate of the computing unit is improved; and the storage addresses of the data blocks are continuous, which reduces the complexity of address calculation and improves data reading performance and loading efficiency.

[0194] Fig. 9 A schematic block diagram of a data processor provided for at least one embodiment of the present disclosure.

[0195] like Fig. 9 As shown, the data processor 200 includes the data storage 100 as described in any embodiment of the present disclosure. The structure, function, and technical effect of the data storage 100 refer to the relevant description in the embodiment of the data processor described above, and will not be repeated here. It should be noted that Fig. 9 The components of the data processor 200 shown are merely exemplary and non-limiting. The data processor 200 may also have other components according to actual application requirements.

[0196] For example, the data processor may be configured to execute the data processing method provided by at least one embodiment of the present disclosure. For the relevant content and technical effects of the data processing method, reference may be made to the relevant description in the embodiment of the data processing method as described above, and no further description will be given here.

[0197] For example, the data processor 200 and the data storage 100 may communicate with each other directly or indirectly.

[0198] For example, the data processor 200 and the data storage 100 can communicate via a network. The network can include a wireless network, a wired network, and / or any combination of a wireless network and a wired network. The data processor 200 and the data storage 100 can also communicate with each other via a system bus, which is not limited by the present disclosure.

[0199] For example, the data processor 200 can be implemented as a GPU, a CPU (central processing unit), an NPU (neural network processor), a DSP (digital signal processor), etc. according to actual needs, and the present disclosure does not limit this.

[0200] For example, in one embodiment, the data processor 200 may be a GPU, and the data storage 100 may be an HBM.

[0201] It should be noted that, in the embodiments of the present disclosure, the data processor 200 may include more or fewer circuits or units, for example, may also include other circuits or units that support the operation of the data processor 200, and the present disclosure does not impose specific limitations on this.

[0202] The connection relationship between each circuit or unit is not limited and can be determined according to actual needs. The specific configuration of each circuit or unit is not limited and can be composed of analog devices according to circuit principles, or can be composed of digital chips, or in other applicable ways.

[0203] Fig.10 A schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. Fig.10 As shown, the storage medium 300 may be a non-transitory computer-readable storage medium, and one or more computer-readable instructions 301 may be non-transitory stored on the storage medium 300. For example, when the computer-readable instructions 301 are executed by a processor, one or more steps in the data storage method or one or more steps in the data processing method described above may be executed.

[0204] For example, the storage medium 300 may be applied to the above-mentioned data processor, for example, the storage medium 300 may include a data memory in the data processor.

[0205] For example, the storage device may include any combination of one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disk read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer-readable instructions may be stored on the computer-readable storage medium, and the data processor may run the computer-readable instructions to implement various functions of the data processor. Various applications and various data may also be stored in the storage medium.

[0206] For example, the storage medium may include a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), flash memory, or any combination of the above storage media, or other applicable storage media.

[0207] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0208] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0209] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0210] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0211] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0212] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

[0213] There are a few points to note about this disclosure:

[0214] (1) The drawings of the embodiments of the present disclosure only relate to the structures related to the embodiments of the present disclosure, and other structures may refer to the general design.

[0215] (2) In the absence of conflict, the embodiments of the present disclosure and the features therein may be combined with each other to obtain new embodiments.

[0216] The above description is only a specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto, and the protection scope of the present disclosure shall be based on the protection scope of the claims.

Claims

1. A data storage method for storing tensor data, wherein: The tensor data includes P batches of matrix data, each batch of matrix data includes M rows and N columns of elements, P, M and N are positive integers, The data storage method comprises: For each batch of matrix data, the matrix data is divided into at least one data block, wherein each data block includes m rows and n columns of elements, wherein m and n are positive integers; The at least one data block is stored in a data memory in units of data blocks, wherein elements in the same data block are stored continuously in the data memory, and elements belonging to the same data block correspond to the same storage address in the data memory; Wherein, dividing the matrix data into at least one data block comprises: Dividing the matrix data into at least one data block according to the data bit width of the elements, wherein the sizes of the data blocks obtained by dividing the matrix data with different data bit widths are different but the total amount of data is the same; Wherein, the matrix data is divided into at least one data block according to the data bit width of the element, including: Determine, according to the data bit width, a data block template corresponding to the data bit width, wherein the data block template indicates the size of a data block obtained by dividing the matrix data of the data bit width; The matrix data is divided according to the data block template to obtain the at least one data block, wherein the union of the at least one data block is the matrix data, and the at least one data block does not overlap each other when divided.

2. The data storage method according to claim 1, wherein: In response to the data bit width being 8 bits, the data block template indicates that the number of rows m of the data block is 2×a, and the number of columns n is 2×b, In response to the data bit width being 16 bits, the data block template indicates that the number of rows m of the data block is 2×a, and the number of columns n is b. In response to the data bit width being 32 bits, the data block template indicates that the number of rows m of the data block is a, the number of columns n is b, Wherein, a and b are positive integers.

3. The data storage method according to claim 1, wherein: The step of storing the at least one data block in a data storage device in units of data blocks includes: Taking data blocks as units, storing the at least one data block in the data storage device in sequence according to row priority or column priority; and For each data block: Determine a storage address corresponding to the data block; The elements in the data block are sequentially stored in the storage space corresponding to the storage address according to a preset storage order.

4. The data storage method according to claim 3, wherein: The preset storage order is indicated by a data block template determined according to the data bit width of the element.

5. The data storage method according to claim 4, wherein: The data block template indicates that each data block includes a plurality of data sub-blocks, the plurality of data sub-blocks are obtained by evenly dividing the data block, each data sub-block includes a plurality of data units, and each data unit includes at least one element, The preset storage order indicates that the multiple data sub-blocks are stored in the data storage device in sequence along the column direction, the multiple data units in each data sub-block are stored in the data storage device in sequence along the row direction, and at least one element in each data unit is stored in the data storage device in sequence according to the preset element order indicated by the data block template.

6. The data storage method according to claim 5, wherein: In response to the data bit width being 16 bits, the data block template indicates that each data block includes a data sub-blocks, each data sub-block is composed of adjacent 2 rows and n columns of elements, each data sub-block includes n data units, and each data unit includes 2 elements located in the same column of the data sub-block, At least one element in each data unit is sequentially stored in the data storage device according to the preset element sequence indicated by the data block template, including: storing two elements in each data unit into the data memory in sequence according to the column direction, Wherein, a is a positive integer, and m=2×a.

7. The data storage method according to claim 5, wherein: In response to the data bit width being 8 bits, the data block template indicates that each data block includes a data sub-blocks, each data sub-block is composed of adjacent 2 rows and n columns of elements, each data sub-block includes b data units, each data unit includes adjacent two rows and two columns of elements in the data sub-block, At least one element in each data unit is sequentially stored in the data storage device according to the preset element sequence indicated by the data block template, including: The four elements in each data unit are sequentially stored in the data memory in a row priority order or a column priority order, Wherein, a and b are positive integers, m=2×a, n=2×b.

8. The data storage method according to claim 5, wherein: In response to the data bit width being 32 bits, the data block template indicates that each data block includes m data sub-blocks, each data sub-block consists of 1 row and n columns of elements, each data sub-block includes n data units, each data unit includes 1 element, The multiple data units in each data sub-block are sequentially stored in the data memory in a row direction, and at least one element in each data unit is sequentially stored in the data memory in a preset element order indicated by the data block template, including: The n elements in each data sub-block are sequentially stored in the data memory in the row direction.

9. The data storage method according to claim 3, wherein: In response to storing the at least one data block in the data memory in sequence according to the row priority order, the storage addresses of two data blocks adjacent in the row direction in the matrix data in the data memory are continuous.

10. The data storage method according to claim 9, wherein: Storing the at least one data block in the data memory in sequence according to the row priority order includes storing the at least one data block in the data memory in sequence first in the row direction and then in the column direction, Among them, the at least one data block is arranged in the form of G rows and Q columns, and the storage addresses of the last data block stored in the data storage device in the i-th row of the G rows and the first data block stored in the data storage device in the i+1-th row are continuous, wherein G, Q and i are all positive integers, and i is less than or equal to G.

11. The data storage method according to claim 3, wherein: In response to the at least one data block being sequentially stored in the data memory in a column priority order, the storage addresses of two data blocks adjacent in the column direction in the matrix data in the data memory are continuous.

12. The data storage method according to claim 11, wherein: Storing the at least one data block in the data memory in sequence according to the column priority order includes storing the at least one data block in the data memory in sequence first in the column direction and then in the row direction, Among them, the at least one data block is arranged in the form of G rows and Q columns, and the storage addresses of the last data block stored in the data storage device in the jth column of the Q columns and the first data block stored in the data storage device in the j+1th column are continuous, wherein G, Q and j are all positive integers.

13. The data storage method according to any one of claims 1 to 12, before dividing the matrix data into at least one data block, the data storage method further comprises: In response to the fact that M and m are not integer multiples: A plurality of elements having a first value are filled around the boundary of the matrix data to increase the number of rows of the matrix data to M', wherein: In response to the fact that N and n are not integer multiples: A plurality of elements having the first value are filled around the boundary of the matrix data to increase the number of columns of the matrix data to N', wherein: in, Represents the ceiling function, where M' and N' are positive integers.

14. A data processing method, comprising: receiving a data loading instruction, wherein the data loading instruction includes a storage address of tensor data to be loaded, the tensor data includes P batches of matrix data, each batch of matrix data includes M rows and N columns of elements, and P, M and N are positive integers, After parsing the data loading instruction, executing the data loading instruction, Wherein, executing the data loading instruction includes: Determine the storage address of each data block corresponding to the tensor data, wherein each data block includes m rows and n columns of elements, and each data block is obtained by dividing the P batches of matrix data, and elements belonging to the same data block correspond to the same storage address in the data memory, and m and n are positive integers; Extracting data from the data memory using the storage address of each data block in units of data blocks to obtain the tensor data, wherein elements of the same data block are stored continuously in the data memory; Wherein, each data block is obtained by the following operations: For each batch of matrix data, the matrix data is divided into at least one data block according to the data bit width of the elements, wherein the sizes of the data blocks obtained by dividing the matrix data with different data bit widths are different but the total amount of data is the same; Wherein, the matrix data is divided into at least one data block according to the data bit width of the element, including: Determine, according to the data bit width, a data block template corresponding to the data bit width, wherein the data block template indicates the size of a data block obtained by dividing the matrix data of the data bit width; The matrix data is divided according to the data block template to obtain the at least one data block, wherein the union of the at least one data block is the matrix data, and the at least one data block does not overlap each other when divided.

15. A data storage device for storing tensor data, wherein: The tensor data includes P batches of matrix data, each batch of matrix data includes M rows and N columns of elements, P, M and N are positive integers, The data memory is configured to store the tensor data in units of data blocks, wherein each data block includes m rows and n columns of elements, m and n are positive integers, each data block is obtained by dividing the P batches of matrix data, elements in the same data block are stored continuously in the data memory, and elements belonging to the same data block correspond to the same storage address in the memory; Wherein, each data block is obtained by the following operations: For each batch of matrix data, the matrix data is divided into at least one data block according to the data bit width of the elements, wherein the sizes of the data blocks obtained by dividing the matrix data with different data bit widths are different but the total amount of data is the same; Wherein, the matrix data is divided into at least one data block according to the data bit width of the element, including: Determine, according to the data bit width, a data block template corresponding to the data bit width, wherein the data block template indicates the size of a data block obtained by dividing the matrix data of the data bit width; The matrix data is divided according to the data block template to obtain the at least one data block, wherein the union of the at least one data block is the matrix data, and the at least one data block does not overlap each other when divided.

16. The data storage device according to claim 15, wherein: The total amount of data in each data block is the same and is determined by the storage capacity of the storage unit corresponding to a single storage address in the data storage device.

17. A data processor comprising a data storage device as claimed in claim 15 or 16.

18. A non-transitory computer-readable storage medium, wherein: The non-transitory computer-readable storage medium stores computer-executable instructions, When the computer executable instructions are executed by a processor, the data storage method according to any one of claims 1 to 13 or the data processing method according to claim 14 is implemented.

Images