A matrix reading and writing method and device

By establishing the correspondence between thread groups and cache blocks in matrix processing, optimizing the reading and writing operations of matrix blocks, the problem of low computational efficiency between threads is solved, and the reading efficiency and storage access efficiency of matrix blocks are improved .

CN118626762BActive Publication Date: 2024-12-03INNOSILICON MICROELECTRONICS (WUHAN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411116529.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-15
Publication Date
2024-12-03
Estimated Expiration
2044-08-15

AI Technical Summary

Technical Problem

In the prior art, each thread needs to calculate the data address separately, resulting in inefficient matrix reading and writing, and especially inability to effectively deal with matrix data of various storage methods.

Method used

By pre-establishing the correspondence between the cache blocks in the preset cache and the thread group, the thread group performs a single-time matrix block reading or writing operation as a whole, layout calculates the external storage address of each element by using the matrix storage method, and then It is written or read into a cache block, optimizing data access between threads.

Benefits of technology

This greatly reduces the time of position calculation in matrix reading between threads, improves matrix block reading efficiency, and improves the bandwidth utilization of storage access.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118626762B_ABST
    Figure CN118626762B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of matrix processing technology, and provides a matrix reading and writing method and device. The method includes: when it is necessary to read a matrix block in a matrix into each thread, the thread group as a whole performs a single matrix block reading operation, and the single matrix block reading operation includes: obtaining the external storage address of the first element of the matrix block, the storage mode of the matrix, and the address to be written; according to the storage mode of the matrix and the external storage address of the first element, obtaining the external storage address of each element in the matrix block, reading each element from the external storage address of each element, and writing it into the corresponding cache block; according to the corresponding relationship, writing the elements in each cache block to the address position to be written of each thread register. The present invention converts the matrix writing of multiple threads in the thread group into a single matrix block writing operation, thereby greatly reducing the time required for position calculation in the matrix reading of each thread, and improving the efficiency of matrix block reading.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of matrix processing, and in particular to a matrix reading and writing method and device. Background Art

[0002] Multi-core general-purpose processors based on Single Instruction Multiple Threads (SIMT) are widely used in matrix calculation algorithms. Generally speaking, SIMT processors use multiple threads as the smallest processing unit, and these multiple threads form a thread group, and multiple threads in the thread group jointly execute processing-related tasks.

[0003] like Figure 1 As shown in the figure, for large matrix multiplication calculations, the large matrix calculations are usually divided into several small matrix block calculations. A thread group is responsible for completing the calculation of a matrix block. Multiple threads in the thread group need to work together to complete the calculation of this matrix block. Figure 1 In the example, four thread groups complete the multiplication operation between matrix A and matrix B. Specifically, when it comes to a single matrix block, Figure 2 As shown in the figure, the corresponding elements in the matrix block are loaded into the corresponding threads in the thread group (actually, the elements are written into the thread registers), so that each thread can operate on the loaded elements. Each thread in the thread group needs a part of the elements of the matrix block to complete the matrix calculation. How to efficiently read the matrix block from the external storage and write it into the register file corresponding to each thread in the thread group has become a key issue.

[0004] In the prior art, there are two methods for reading data from external storage to the corresponding thread, namely using the Sample (abbreviated as: SMP) instruction and using the load / store (abbreviated as: LD / ST) instruction. The Sample instruction needs to store the matrix data in the specified image object, and then read the data corresponding to each thread through the built-in Sample function. This requires modifying the user's code. For most applications, in order to facilitate the central processing unit (Central Processing Unit, abbreviated as: CPU) and the graphics processing unit (Graphics Processing Unit, abbreviated as: GPU) to share the matrix cache, users prefer to store the matrix in a linear storage structure, such as an array object, which makes the Sample instruction inconvenient to use.

[0005] Although the LD / ST instruction supports storing matrix data in array objects, under the existing implementation scheme, each thread needs to calculate the corresponding data address to be obtained, and the addresses of multiple threads must be effectively merged. If they cannot be effectively merged, the bandwidth utilization of external storage access will be reduced. In addition, the Sample instruction and LD / ST instruction cannot handle matrices stored in multiple storage methods.

[0006] In view of this, overcoming the defects of the prior art is an urgent problem to be solved in the field of this technology. Summary of the invention

[0007] The technical problem to be solved by the present invention is that in the prior art, each thread needs to calculate the corresponding data address to be obtained, resulting in low matrix reading and writing performance.

[0008] The present invention adopts the following technical solution:

[0009] In a first aspect, the present invention provides a matrix reading and writing method, in which a correspondence between each cache block in a preset cache and each thread in a thread group is pre-established, wherein the i-th cache block corresponds to the i-th thread; when the m-th matrix block in the matrix needs to be read into each thread, the thread group as a whole performs a single matrix block reading operation, and the single matrix block reading operation includes:

[0010] Get the external storage address of the first element of the mth matrix block , the storage layout of the matrix and the address to be written R0;

[0011] According to the storage layout of the matrix and the external storage address of the first element , obtain the external storage address of each element in the mth matrix block, read each element from the external storage address of each element, and write it into the corresponding cache block;

[0012] According to the corresponding relationship, the elements in each cache block are written to the to-be-written address R0 position of each thread register.

[0013] Preferably, when the storage layout of the matrix is ​​row-major storage, the external storage address of the first element according to the storage layout of the matrix , obtain the external storage address of each element in the mth matrix block, read each element from the external storage address of each element, and write it into the corresponding cache block, specifically including:

[0014] Calculate the external starting storage address row_addr and row size row_size of each row in the m-th matrix block;

[0015] Read data of row size row_size from the external starting storage address row_addr of row row, and write the read data as a whole to the preset cache block_buffer[row] position;

[0016] in, ; The block_buffer[row] position is Cache blocks to The position between cache blocks; is the size of an element in the matrix, is the line currently being read, Greater than or equal to 0 and less than is the total number of rows in the mth matrix block, is the total number of columns in the matrix, is the total number of columns of the mth matrix block.

[0017] Preferably, the external storage address of the first element of the m-th matrix block is It is based on the external storage address of the matrix The calculated results include:

[0018] No. External storage address of the first element of a matrix block ; in, is the size of an element in the matrix, is the total number of columns in the matrix, is the total number of columns of the mth matrix block, is the total number of rows in the mth matrix block.

[0019] Preferably, when the storage layout of the matrix is ​​column-major storage, the storage layout of the matrix and the external storage address of the first element , obtain the external storage address of each element in the mth matrix block, read each element from the external storage address of each element, and write it into the corresponding cache block, specifically including:

[0020] Calculate the starting storage address elem_addr[i] of each element in the m-th matrix block in turn, and read the size of elem_addr[i] from the starting storage address of the element. The data obtained by reading is written into the i-th cache block position of the preset cache;

[0021] in, ; is the size of an element in the matrix, i represents the i-th element in the currently read matrix block, is the total number of columns of the mth matrix block, is the total number of rows in the matrix.

[0022] Preferably, the external storage address of the first element of the m-th matrix block is It is based on the external storage address of the matrix The calculated results include:

[0023] No. External storage address of the first element of a matrix block ; in, is the size of an element in the matrix, is the total number of columns in the matrix, is the total number of rows in the matrix, is the total number of columns of the mth matrix block, is the total number of rows in the mth matrix block.

[0024] Preferably, when it is necessary to write the m-th matrix block composed of elements in each thread to the external cache, the thread group as a whole performs a single matrix block write operation, and the single matrix block write operation includes:

[0025] Get the external storage address where the first element of the mth matrix block needs to be written in the external storage , the storage layout required by the matrix in external storage and the address to be read R0;

[0026] Read each element from the to-be-read address R0 of each thread register, according to the storage layout required by the matrix in the external storage and the external storage address of the first element , write each element to the corresponding location in external storage.

[0027] Preferably, when the storage layout required for the matrix in the external storage is row-major storage, each element is read from the to-be-read address R0 of each thread register, according to the storage layout required for the matrix in the external storage and the external storage address of the first element. , write each element to the corresponding location of the external storage, including:

[0028] The size of the read address R0 from the i-th thread register is The data read is written to the out_addr[i] position of the external storage;

[0029] in, ; is the size of an element in the matrix, i represents the element in the register of the i-th thread currently being read, is the total number of columns of the mth matrix block, is the total number of columns in the matrix.

[0030] Preferably, when the storage layout required for the matrix in the external storage is column-major storage, each element is read from the to-be-read address R0 of each thread register, according to the storage layout required for the matrix in the external storage and the external storage address of the first element. , write each element to the corresponding location of the external storage, including:

[0031] The size of the read address R0 from the i-th thread register is The data read is written to the out_addr[i] position of the external storage;

[0032] in, ; is the size of an element in the matrix, i represents the element in the register of the i-th thread currently being read, is the total number of columns of the mth matrix block, is the total number of rows in the matrix.

[0033] Preferably, the multiple threads in the thread group bind all threads in the thread group together in a lock-step manner, so that the multiple threads read the elements synchronously.

[0034] In a third aspect, the present invention further provides a matrix reading and writing device, for implementing the matrix reading and writing method described in the first aspect, the device comprising:

[0035] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor to execute the matrix reading and writing method described in the first aspect.

[0036] In a fourth aspect, the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, which are executed by one or more processors to complete the matrix reading and writing method described in the first aspect.

[0037] The present invention converts the matrix writing of multiple threads in a thread group into a single matrix block writing operation, thereby greatly reducing the time required for position calculation in matrix reading of each thread and improving the matrix block reading efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 is a schematic diagram of matrix processing in the prior art provided by an embodiment of the present invention;

[0040] Figure 2 is a schematic diagram of another matrix processing in the prior art provided by an embodiment of the present invention;

[0041] Figure 3 It is a flowchart of a matrix reading and writing method provided by an embodiment of the present invention;

[0042] Figure 4 is a schematic diagram of storing a matrix in a thread register in row-major order provided by an embodiment of the present invention;

[0043] Figure 5 is a schematic diagram of storing a matrix in a thread register in column-major order provided by an embodiment of the present invention;

[0044] Figure 6 is a schematic diagram of storing a matrix in row-major order in an external storage provided by an embodiment of the present invention;

[0045] Figure 7 is a schematic diagram of storing a matrix in column-major order in an external storage provided by an embodiment of the present invention;

[0046] Figure 8 It is a flowchart diagram of another matrix reading and writing method provided by an embodiment of the present invention;

[0047] Fig. 9 It is a flowchart diagram of another matrix reading and writing method provided by an embodiment of the present invention;

[0048] Fig.10 It is a schematic diagram of the meaning of each parameter in a matrix reading and writing method provided by an embodiment of the present invention;

[0049] Fig.11 It is a schematic diagram of a pipeline architecture in the prior art provided by an embodiment of the present invention;

[0050] Fig.12 It is a schematic diagram of a pipeline architecture provided by an embodiment of the present invention;

[0051] Fig.13is a schematic diagram of performing matrix calculation using a matrix reading and writing method provided by an embodiment of the present invention;

[0052] Fig.14 is a schematic diagram of the architecture of a matrix processing module provided by an embodiment of the present invention;

[0053] Fig.15 It is a schematic diagram of the architecture of a matrix reading and writing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0055] The terms "first", "second", etc. in the present invention are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of this application, unless otherwise specified, "plurality" means two or more.

[0056] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0057] The embodiment of the present invention provides a matrix reading and writing method, which pre-establishes a corresponding relationship between each cache block in a preset cache and each thread in a thread group, wherein the i-th cache block corresponds to the i-th thread; when it is necessary to read the m-th matrix block in the matrix into each thread, the thread group as a whole performs a single matrix block reading operation, such as Figure 3 As shown, a single matrix block read operation includes:

[0058] In step 201, the external storage address of the first element of the mth matrix block is obtained , the storage layout of the matrix and the address to be written R0; the storage layout includes one or more of row-major storage, column-major storage and block-linear storage. Row-major storage is also called row major in actual use, column-major storage is also called column major in actual use, and block-linear storage is also called block-linear in actual use. Among them, the thread group as a whole executes a single matrix read operation, which can be performed by one thread in the thread group, or by adding an additional management thread to the thread group, and the management thread executes the matrix read operation. The external storage address of the first element , the storage layout of the matrix, and the address to be written R0 are all carried in the corresponding instruction for acquisition, such as carried in the matrix block read instruction, wherein the matrix block read instruction may also carry the total number of columns of the mth matrix block , the total number of rows in the mth matrix block , the total number of rows in the matrix , the total number of columns in the matrix . Wherein, m is an integer, m is greater than or equal to 0 and less than the total number of matrix blocks in the matrix. It should be noted here that the mth matrix block here is only used to highlight a certain matrix block from the multiple matrix blocks that the user needs to read into the thread, and does not represent m objects in the corresponding arrangement order. In actual use, the total number of columns and the total number of rows of the matrix block are obtained by technical personnel in this field according to the needs and the number of threads in the thread group. For a matrix, the total number of columns and the total number of rows of the matrix block can have multiple combinations, which are not limited in this embodiment.

[0059] In step 202, according to the storage layout of the matrix and the external storage address of the first element , obtain the external storage address of each element in the mth matrix block, read each element from the external storage address of each element, and write it into the corresponding cache block; because the arrangement of each element in the storage of the corresponding matrix block may be different in different storage methods, the elements in the matrix block are read into the cache block first.

[0060] In step 203, the elements in each cache block are written to the to-be-written address R0 position of each thread register according to the corresponding relationship, that is, each element in a matrix block is written to the same address of the thread registers of different threads in the same thread group.

[0061] Each thread corresponds to a thread register, and the thread registers of each thread in the thread group have the same address arrangement, that is, when the address of the first thread register is at position R1, the addresses of the thread registers of other threads in the thread group are also R1.

[0062] This embodiment converts the matrix write of multiple threads in the thread group into a single matrix block write operation, thereby greatly reducing the thread register resources and processor computing units required for position calculation in matrix reading of each thread, thereby improving the efficiency of matrix block reading.

[0063] In actual usage scenarios, when the storage layout of the matrix is ​​row-major storage, the external storage address of the first element according to the storage layout of the matrix , obtain the external storage address of each element in the mth matrix block, read each element from the external storage address of each element, and write it into the corresponding cache block, specifically including:

[0064] The external starting storage address row_addr and the row size row_size of each row in the m-th matrix block are calculated.

[0065] Read data of row size row_size from the external starting storage address row_addr of row row, and write the read data as a whole to the preset cache block_buffer[row] position;

[0066] in, ; The block_buffer[row] position is Cache blocks to The position between cache blocks (including cache blocks and cache blocks); is the size of an element in the matrix, is the line currently being read, Greater than or equal to 0 and less than is the total number of rows in the mth matrix block, is the total number of columns in the matrix, is the total number of columns of the mth matrix block.

[0067] Among them, the external storage address of the first element of the mth matrix block is It can be obtained by a person skilled in the art according to demand analysis, or it can be specified in advance by a person skilled in the art in a block-by-block manner, or it can be calculated by the arrangement of the matrix blocks, that is, the external storage address of the first element of the mth matrix block It is based on the external storage address of the matrix The calculated results include:

[0068] No. External storage address of the first element of a matrix block ; in, is the size of an element in the matrix, is the total number of columns in the matrix, is the total number of columns of the mth matrix block, is the total number of rows of the m-th matrix block. In this embodiment, similar The expression represents the value obtained by dividing a by b, similar to The expression represents the modulus operation of a on b.

[0069] When the storage layout of the matrix is ​​column-major storage, the external storage address of the first element according to the storage layout of the matrix , obtain the external storage address of each element in the mth matrix block, read each element from the external storage address of each element, and write it into the corresponding cache block, specifically including:

[0070] Calculate the starting storage address elem_addr[i] of each element in the m-th matrix block in turn, and read the size of elem_addr[i] from the starting storage address of the element. The read data is written into the i-th cache block position of the preset cache.

[0071] in, ; is the size of an element in the matrix, i represents the i-th element in the currently read matrix block, is the total number of columns of the mth matrix block, is the total number of rows in the matrix.

[0072] Among them, the external storage address of the first element of the mth matrix block is It can be specified by a person skilled in the art when dividing the matrix into matrix blocks in advance, or it can be calculated by the arrangement of the matrix blocks, that is, the external storage address of the first element of the mth matrix block It is based on the external storage address of the matrix The calculated results include:

[0073] No. External storage address of the first element of a matrix block ; in, is the size of an element in the matrix, is the total number of columns in the matrix, is the total number of rows in the matrix, is the total number of columns of the mth matrix block, is the total number of rows in the mth matrix block.

[0074] In actual use, the storage method src_layout of the matrix in the external storage address and the arrangement method dst_layout of the values ​​in the matrix in each thread register can also be specified separately. Under this implementation mode, there may be four combinations of corresponding matrix reading methods, namely, both the external storage address and the thread register are in row-major order; both the external storage address and the thread register are in column-major order; the external storage address is in row-major order and the thread register is in column-major order; the external storage address is in column-major order and the thread register is in row-major order.

[0075] Among them, in the thread register, the row-major order is stored as follows Figure 4 As shown, T0 to T15 represent the corresponding thread registers respectively, and the elements in the matrix are as follows Figure 4 The direction indicated by the arrows in the middle is stored in each thread register, that is, the elements of the first row of the 4×4 matrix are stored in thread registers T0 to T3, the elements of the second row are stored in thread registers T4 to T7, and so on; in the thread register, the column-major order is stored as follows Figure 5 As shown, T0 to T15 represent the corresponding thread registers respectively, and the elements in the matrix are as follows Figure 5 The directions indicated by the arrows are stored in the thread registers respectively, that is, they are stored in sequence according to the rows. The elements of the first column in the 4×4 matrix are stored in thread registers T0 to T3 in sequence, and the elements of the second column are stored in thread registers T4 to T7 in sequence, and so on.

[0076] In external storage addresses, row-major order is stored as follows Figure 6 As shown, with position 0 as the storage start position of the matrix, the n-1 elements of the first row of the matrix are stored in the address positions 0 to n-1 of the external storage address, the n-1 elements of the second row of the matrix are stored in the address positions n to 2n-1 of the external storage address, and so on; in the external storage address, the column-major order is stored as follows Figure 7 As shown, with position 0 as the storage starting position of the matrix, the n-1 elements of the first column of the matrix are stored in address positions 0 to n-1 of the external storage address, the n-1 elements of the second column of the matrix are stored in address positions n to 2n-1 of the external storage address, and so on.

[0077] In this embodiment, only two situations are specifically explained, namely, the situation that both the external storage address and the preset cache are in row-major order, and the situation that the external storage address is in column-major order and the preset cache is in row-major order. In actual use, the situation that both the external storage address and the preset cache are in column-major order can be directly stored in address order. The situation that the external storage address is in row-major order and the preset cache is in column-major order can first read the elements in the external storage address in row-major order, and then write the elements to the preset cache in column-major order. The method of storing elements in column-major order is based on the same concept as the implementation method of matrix block writing described below.

[0078] The number of matrix blocks to be read m can also be calculated based on the number of matrix blocks to be read in the width direction burst_x and the number of matrix blocks to be read in the height direction burst_y. At this time, the corresponding matrix block reading instruction can be load_matrix.src_layout.dst_layout.height.width.src_type.dst_typedst, addr, stride, burst_x, burst_y; where src_type is the storage type of the element in the external storage address, and dst_type is the storage type of the element in the preset cache (that is, the storage type of the element in the thread register). dst is the address to be written, and addr is the external storage address of the first element of the mth matrix block, that is When src_type is row-major, stride is as above When src_type is column-major, stride is as above .

[0079] In actual use, the method further includes: when it is necessary to write the elements in each thread into the mth matrix block to the external cache, the thread group as a whole performs a single matrix block write operation, such as Figure 8 As shown, a single matrix block write operation includes:

[0080] In step 301, the external storage address to which the first element of the mth matrix block needs to be written in the external storage is obtained. , the storage layout required for the matrix in external storage and the address R0 to be read.

[0081] In step 302, each element is read from the to-be-read address R0 of each thread register, according to the storage layout required by the matrix in the external storage and the external storage address of the first element , write each element to the corresponding location in external storage.

[0082] It should be noted here that the matrix reading described in this embodiment refers to reading the matrix from the external storage and writing the matrix into the thread's register, while the matrix writing refers to reading the matrix value from the thread's register and writing the matrix value into the external storage.

[0083] When the storage layout required by the matrix in the external storage is row-major storage, each element is read from the to-be-read address R0 of each thread register, according to the storage layout required by the matrix in the external storage and the external storage address of the first element. , write each element to the corresponding location of the external storage, including:

[0084] The size of the read address R0 from the i-th thread register is The read data is written to the out_addr[i] position of the external storage.

[0085] in, ; is the size of an element in the matrix, i represents the element in the register of the i-th thread currently being read, is the total number of columns of the mth matrix block, is the total number of columns in the matrix.

[0086] When the storage layout required by the matrix in the external storage is column-major storage, each element is read from the to-be-read address R0 of each thread register, according to the storage layout required by the matrix in the external storage and the external storage address of the first element , write each element to the corresponding location of the external storage, including:

[0087] The size of the read address R0 from the i-th thread register is The read data is written to the out_addr[i] position of the external storage.

[0088] in, ; is the size of an element in the matrix, i represents the element in the register of the i-th thread currently being read, is the total number of columns of the mth matrix block, is the total number of rows in the matrix.

[0089] The matrix block write instruction used in actual use can be store_matrix.src_layout.dst_layout.height.width.src_type.dst_typedst, addr, stride, burst_x, burst_y; among them, dst is the register address where the matrix is ​​stored, addr is the external storage address where the matrix needs to be written, and the meanings of other parameters are consistent with the matrix block read instruction.

[0090] In order to ensure that multiple threads only need to perform a single matrix block read operation when reading a matrix, the multiple threads in the thread group bundle all threads in the thread group together in a lock-step manner so that the multiple threads read elements synchronously. Specifically:

[0091] All threads in a thread group are executed together in lock step, that is, the thread group sends a single instruction. In actual hardware, the command receives a matrix element read instruction, and then the hardware executes the instruction for each thread to obtain the element. The thread group still sends a single instruction, but this instruction is the thread group's instruction.

[0092] There is also an optional implementation mode: the matrix block reading operation is implemented by receiving a matrix block reading instruction control from each thread, such as Fig. 9 As shown, specifically including:

[0093] In step 401, when a first matrix block read instruction is first received from a corresponding thread, a first matrix block read operation is executed, and parameters in the first matrix block read instruction are stored in a read instruction table entry. When a second matrix block read instruction is subsequently received, the parameters of the second matrix block read instruction are compared with the parameters in the read instruction table entry to determine whether the second matrix block read instruction is the same as the first matrix block read instruction. If they are the same, the second matrix block read instruction is discarded; if they are not the same, they are considered to be different instructions, the second matrix block read instruction is executed, and the operation of writing the read instruction table entry is performed accordingly.

[0094] In step 402, after each data is written into each thread register, the corresponding matrix block read instruction in the read instruction table entry is deleted.

[0095] Based on the same concept, the matrix block write operation is also implemented by receiving the matrix block write instruction control from each thread, specifically including:

[0096] When the first matrix block write instruction is received from the corresponding thread for the first time, the first matrix block write operation is executed, and the parameters in the first matrix block write instruction are stored in the write instruction table entry. When the second matrix block write instruction is subsequently received, the parameters of the second matrix block write instruction are compared with the parameters in the instruction table entry to determine whether the second matrix block write instruction is the same as the first matrix block write instruction. If they are the same, the second matrix block write instruction is discarded.

[0097] After each data is written into each thread register, the corresponding matrix block write instruction in the instruction table entry is deleted.

[0098] The present invention is based on the method described in the above embodiment, combined with specific application scenarios, and uses technical descriptions in related scenarios to illustrate the implementation process of the present invention in characteristic scenarios.

[0099] This embodiment first provides the following two instructions:

[0100] The first type: matrix block read instruction, which is expressed in the following form: load_matrix.src_layout.dst_layout.height.width.src_type.dst_typedst, addr, stride, burst_x, burst_y.

[0101] The second type: matrix block write instructions, the specific form is:

[0102] store_matrix.src_layout.dst_layout.height.width.src_type.dst_typedst,addr, stride, burst_x, burst_y.

[0103] In the above matrix block read instructions and matrix block write instructions, Layout (including src_layout and dst_layout) is used to indicate the storage method of the matrix, including row major and column major.

[0104] Height indicates the height of the matrix block to be read, that is, the total number of rows in the matrix block.

[0105] Width indicates the width of the matrix block to be read, that is, the total number of columns of the matrix block.

[0106] src_type is used to indicate the storage type of the elements in the matrix, which can be int8, int16, fp16, fp32, etc.

[0107] dst_type is used to indicate the element storage type of the matrix block that needs to be stored in the register, which can also be int8, int16, fp16, fp32, etc.

[0108] dst indicates the register address to be used for storage.

[0109] addr stores the address of the first element of the matrix block to be read.

[0110] burst_x represents the number of matrix blocks that need to be read in the width direction, such as Fig.10 As shown, burst_x=2.

[0111] burst_y represents the number of matrix blocks that need to be read in the height direction, such as Fig.10 As shown, burst_x=2.

[0112] like Fig.10The figure shows a schematic diagram of a matrix stored in row-major order. For a row-major matrix, stride is the number of elements in a row of the source matrix; while for a column-major matrix, stride is the number of elements in a column of the source matrix.

[0113] The following is an example of reading a row-major matrix into each thread register. The execution process can be expressed as the following pseudo code:

[0114] block_base = base_addr + offset; / / block_base is the corresponding block_base[m] in the above embodiment;

[0115] Row_size = width * elem_size;

[0116] Block_buffer[4][4];

[0117] For ( row = 0; row <height; row ++){

[0118] row_addr = block_base + row * stride * elem_size;

[0119] Load( block_buffer[row], row_addr, row_size); / / Write the data starting from row_addr and with a length of row_size into block_buffer[row].

[0120] }

[0121] After the data of all rows in the matrix block are returned, the entire matrix block is written to the register of the thread group, which can be expressed as the following pseudo code:

[0122] thread0.ri = block_buffer[0][0]; / / Write the elements in each cache block to the ri (i.e., dst) position of the thread register.

[0123] Thread1.ri = block_buffer[0][1];

[0124] …

[0125] Thread3.ri = block_buffer[0][3];

[0126] Thread4.ri = block_buffer[1][0]; ..

[0128] Thread7.ri = block_buffer[1][3];

[0129] …

[0130] Thread15.ri = block_buffer[3][3];

[0131] in,

[0132] Offset is the address offset of the matrix block currently requested to be read relative to the first address of the matrix.

[0133] base_addr is the starting address where the entire matrix is ​​stored.

[0134] stride is the row width of the entire matrix.

[0135] elem_size is the number of bytes per element in the matrix.

[0136] In order to achieve the above functions without occupying the resources of existing threads, this embodiment also makes improvements in architecture relative to the existing technology, such as Fig.11 It is a pipeline architecture within a computing core commonly used in the prior art.

[0137] This embodiment adds a matrix processing module (also called Matrix Access Pipeline) based on the prior art. Fig.12 As shown in the figure, the control registers required for executing the load_matrix (also called matrix_load) instruction and the store_matrix (also called matrix_store) instruction are all from the group register file, and the thread register file does not need to be used, which can greatly reduce the access to the thread register file. The process of using the load_matrix instruction and the store_matrix instruction to complete the matrix calculation is as follows: Fig.13As shown, by using the matrix_load instruction, the elements in the matrix a and the matrix b are read into each thread in the thread group, and each thread performs operations between the elements to obtain each element in the matrix C. Then, by using the matrix_store instruction, each element in the matrix C that has been calculated in the thread is written into the external storage, and each element written into the external storage is stored in the form of a matrix, thereby obtaining a complete matrix C and realizing a complete matrix operation process. The matrix processing module is used in the matrix reading and writing device to realize the matrix reading and writing method described in the above embodiment (i.e., the load_matrix instruction and the store_matrix instruction).

[0138] In addition, the Matrix Access Pipeline will calculate the address of each row or column in the matrix block according to the information provided in the instruction, and send it to the memory subsystem to read the corresponding data. After all the data of the matrix block is returned, it can implement format conversion and transpose operations according to the requirements defined in the instruction. Write it to the thread register file according to the dst_layout required by the instruction.

[0139] The matrix processing module (in Fig.14 In practice, it can be represented as a matrix access pipeline. Fig.14 As shown, it includes an instruction operand reading module, a loop control module (also called a burst loop control module), a block request loop control module (also called a block request loop control module), a data return packaging module, and a register write control module, and uses the thread group register as a preset cache.

[0140] Among them, the instruction operand reading module receives instructions from the instruction issuing module, parses the instructions, and reads corresponding operands from the thread group register or thread register according to the source of the operands in the instructions. The main operands are described in the instruction definition.

[0141] After the instruction operands are read, the burst_x and burst_y fields in the instruction control whether the instruction reads one or more matrix blocks. If multiple matrix blocks need to be read, the burst loop control module is responsible for splitting the instruction into multiple instructions for reading / writing a matrix block, so that the subsequent modules only need to process the load / store operation of a single matrix block.

[0142] The block request loop control module receives the matrix load store instruction of a single block after upstream disassembly, and needs to further determine whether it needs to be processed row by row or column by column based on the layout of the matrix in the memory buffer. The address calculation formula for row by row processing is described above.

[0143] For matrix store instructions, the block request loop control module sends each row or column of the matrix block to be written out to the external storage system.

[0144] For the matrix load instruction, you only need to send the address of each row or column in the matrix block. After all the requests for the matrix block are sent out, you need to wait for the corresponding data of the matrix block to be returned and then package it and write it into the register specified in the instruction.

[0145] The register write control is responsible for writing the packed matrix data to the address of the specified thread. Each thread's register file will have a corresponding write port. According to the previously defined correspondence between the matrix block and the thread, the data can be written to the interface of the corresponding thread.

[0146] like Fig.15 , is a schematic diagram of the architecture of a matrix reading and writing device according to an embodiment of the present invention. The matrix reading and writing device according to this embodiment includes one or more processors 21 and a memory 22. Fig.15 A processor 21 is taken as an example.

[0147] The processor 21 and the memory 22 may be connected via a bus or other means. Fig.15 The example of connecting through bus is taken in the following.

[0148] The memory 22 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs and non-volatile computer executable programs, such as the matrix reading and writing method in the above embodiment. The processor 21 executes the matrix reading and writing method by running the non-volatile software programs and instructions stored in the memory 22.

[0149] The memory 22 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 22 may optionally include a memory remotely arranged relative to the processor 21, and these remote memories may be connected to the processor 21 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0150] The program instructions / modules are stored in the memory 22 , and when executed by the one or more processors 21 , the matrix reading and writing method in the above embodiment is executed.

[0151] It is worth noting that the information interaction, execution process, etc. between the modules and units within the above-mentioned devices and systems are based on the same concept as the processing method embodiment of the present invention. The specific contents can be found in the description of the method embodiment of the present invention and will not be repeated here.

[0152] A person skilled in the art may understand that all or part of the steps in the various methods of the embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0153] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A matrix reading and writing method, characterized in that: A correspondence between each cache block in a preset cache and each thread in a thread group is pre-established, wherein the i-th cache block corresponds to the i-th thread; when the m-th matrix block in the matrix needs to be read into each thread, the thread group as a whole performs a single matrix block reading operation, and the single matrix block reading operation includes: Get the external storage address of the first element of the mth matrix block , the storage layout of the matrix and the address to be written R0; According to the storage layout of the matrix and the external storage address of the first element , obtain the external storage address of each element in the mth matrix block, read each element from the external storage address of each element, and write it into the corresponding cache block; According to the corresponding relationship, the elements in each cache block are written to the to-be-written address R0 position of each thread register; When the storage layout of the matrix is ​​row-major storage, the external storage address of the first element according to the storage layout of the matrix , obtain the external storage address of each element in the mth matrix block, read each element from the external storage address of each element, and write it into the corresponding cache block including: Calculate the external starting storage address row_addr and row size row_size of each row in the m-th matrix block; Read data of row size row_size from the external starting storage address row_addr of row row, and write the read data as a whole to the preset cache block_buffer[row] position; in, ; The block_buffer[row] position is Cache blocks to The position between cache blocks; is the size of an element in the matrix, is the line currently being read, Greater than or equal to 0 and less than , is the total number of rows in the mth matrix block, is the total number of columns in the matrix, is the total number of columns of the mth matrix block.

2. The matrix reading and writing method according to claim 1, characterized in that: When the storage layout of the matrix is ​​column-major storage, the external storage address of the first element according to the storage layout of the matrix , obtain the external storage address of each element in the mth matrix block, read each element from the external storage address of each element, and write it into the corresponding cache block, including: Calculate the starting storage address elem_addr[i] of each element in the m-th matrix block in turn, and read the size of elem_addr[i] from the starting storage address of the element The data obtained by reading is written into the i-th cache block position of the preset cache; in, ; is the size of an element in the matrix, i represents the i-th element in the currently read matrix block, is the total number of columns of the mth matrix block, is the total number of rows in the matrix, Represents the value obtained by dividing a by b.

3. The matrix reading and writing method according to claim 1, characterized in that: When it is necessary to write the elements in each thread into the mth matrix block to the external cache, the thread group as a whole performs a single matrix block write operation, and the single matrix block write operation includes: Get the external storage address where the first element of the mth matrix block needs to be written in the external storage , the storage layout required by the matrix in the external storage and the address to be read R1; Read each element from the to-be-read address R1 of each thread register, according to the storage layout required by the matrix in the external storage and the external storage address of the first element , write each element to the corresponding location in external storage.

4. The matrix reading and writing method according to claim 3, characterized in that: When the storage layout required by the matrix in the external storage is row-major storage, each element is read from the to-be-read address R1 of each thread register, according to the storage layout required by the matrix in the external storage and the external storage address of the first element , write each element to the corresponding location of the external storage, including: The size of the read address R1 from the i-th thread register is The data read is written to the out_addr[i] position of the external storage; in, ; is the size of an element in the matrix, i represents the element in the register of the i-th thread currently being read, is the total number of columns of the mth matrix block, is the total number of columns in the matrix, Represents the value obtained by dividing a by b.

5. The matrix reading and writing method according to claim 3, characterized in that: When the storage layout required by the matrix in the external storage is column-major storage, each element is read from the to-be-read address R1 of each thread register, according to the storage layout required by the matrix in the external storage and the external storage address of the first element , write each element to the corresponding location of the external storage, including: The size of the read address R1 from the i-th thread register is The data read is written to the out_addr[i] position of the external storage; in, ; is the size of an element in the matrix, i represents the element in the register of the i-th thread currently being read, is the total number of columns of the mth matrix block, is the total number of rows in the matrix, Represents the value obtained by dividing a by b.

6. The matrix reading and writing method according to any one of claims 1 to 5, characterized in that: The multiple threads in the thread group bind all the threads in the thread group together in a lock-step manner, so that the multiple threads read the elements synchronously.

7. A matrix reading and writing device, characterized in that: include: at least one processor; And, a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor to execute the matrix reading and writing method according to any one of claims 1 to 6.