Computing matrix storage processing method based on multiple memory blocks and memory controller

Abstract

The invention discloses a calculation matrix storage processing method based on multiple memory blocks and a memory controller. The method comprises the steps that elements in a calculation matrix are stored in a plurality of preset memory blocks according to a preset storage mode, so that elements in the same row in the calculation matrix are stored in different memory blocks respectively, and elements in the same column in the calculation matrix are stored in different memory blocks respectively.

Description

Technical Field

[0001] The present disclosure relates to the field of AI computing technology, and more specifically, to a computing matrix storage processing method based on multiple memory blocks and a memory controller. Background Art

[0002] With the development of artificial intelligence, the demand for computing power of AI chips is getting higher and higher. AI large model computing tasks are usually memory-intensive, and memory access performance directly affects the actual computing power of the chip. Matrix calculation is one of the most commonly used algorithms in AI operations. During the calculation process, large matrices are usually stored in multi-bank memories on the chip. The traditional way of storing matrices by row or column has the following problems. When the matrix is ​​stored in multiple banks by row, bank conflicts will occur when reading and writing the matrix by column, resulting in reduced memory read and write performance. Similarly, when the matrix is ​​stored in multiple banks by column, bank conflicts will occur when reading and writing the matrix by row. Summary of the invention

[0003] In view of the deficiencies in the prior art, the present invention provides a computing matrix storage processing method based on multiple memory blocks and a memory controller.

[0004] According to one aspect of the present disclosure, a computing matrix storage processing method based on multiple memory blocks is provided, comprising: Each element in the calculation matrix is ​​stored in a preset plurality of memory blocks according to a preset storage method, so that elements in the same row of the calculation matrix are stored in different memory blocks and elements in the same column of the calculation matrix are stored in different memory blocks.

[0005] Optionally, the preset storage methods include: The elements on the same diagonal line in the calculation matrix are stored in the same memory block, and the elements on different diagonal lines are stored in different memory blocks.

[0006] Optionally, each element in the calculation matrix is ​​stored in a preset plurality of memory blocks according to a preset storage method, including: According to a preset storage method, the elements in the calculation matrix are stored row by row in a plurality of preset memory blocks, and adjacent elements in the same row are stored in adjacent memory blocks.

[0007] Optionally, the first element of each row of the calculation matrix is ​​stored at the first address of the corresponding memory block.

[0008] Optionally, each element in the calculation matrix is ​​stored in a preset plurality of memory blocks according to a preset storage method, including: According to a preset storage method, the elements in the calculation matrix are stored column by column in a plurality of preset memory blocks, and adjacent elements in the same column are stored in adjacent memory blocks.

[0009] Optionally, the first element of each column of the calculation matrix is ​​stored at the first address of the corresponding memory block.

[0010] Optionally, before storing each element in the calculation matrix into a preset plurality of memory blocks according to a preset storage method, the method further includes: When the number of rows and columns of the calculation matrix is ​​inconsistent, the row elements or column elements are filled with zeros so that the number of rows and columns of the calculation matrix are consistent.

[0011] According to another aspect of the present disclosure, a memory controller is provided, wherein the memory controller is used to store the calculation matrix into multiple memory blocks of the memory according to a calculation matrix storage instruction based on the method described in any one of the above aspects of the present disclosure.

[0012] According to another aspect of the present disclosure, an electronic device is provided, comprising: a memory, a memory controller and a processor, wherein the memory comprises a plurality of memory blocks; the processor is used to send a calculation matrix storage instruction to the memory controller; the memory controller is used to store the calculation matrix into the plurality of memory blocks of the memory according to the calculation matrix storage instruction and based on the method described in any one of the above aspects of the present disclosure.

[0013] Thus, the present disclosure proposes a computing matrix storage processing method based on multiple memory blocks. When reading and writing a matrix by row, the elements of the same row are respectively located on different memory blocks. Therefore, when reading and writing a matrix by row, there will be no memory block read and write conflicts, and full bandwidth reading and writing can be achieved. At the same time, when reading and writing a matrix by column, the elements of the same column are also respectively located on different memory blocks. Therefore, when reading and writing a matrix by column, there will be no memory block conflicts, and full bandwidth reading and writing can be achieved. When the method disclosed in the present disclosure is used to store a matrix, the matrix can be read and written with full bandwidth, whether by row or by column, which significantly improves the matrix read and write performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] A more complete understanding of exemplary embodiments of the present disclosure may be obtained by referring to the following drawings: Figure 1 It is a flowchart of a calculation matrix storage processing method based on multiple memory blocks provided by an exemplary embodiment of the present disclosure; Figure 2 is a schematic diagram of a traditional computing matrix provided by an exemplary embodiment of the present disclosure; Figure 3 is a schematic diagram of a traditional computing matrix stored in rows according to an exemplary embodiment of the present disclosure; Figure 4 is a schematic diagram of a row-by-row diagonal storage method of a calculation matrix provided by an exemplary embodiment of the present disclosure; Figure 5 is a schematic diagram of a diagonal line of a calculation matrix provided by an exemplary embodiment of the present disclosure; Figure 6 is a schematic diagram of a diagonal line of a calculation matrix provided by an exemplary embodiment of the present disclosure; Figure 7 is a schematic diagram of storing the results of a calculation matrix according to row diagonals provided by an exemplary embodiment of the present disclosure; Figure 8 is a schematic diagram of a column-by-column diagonal storage method of a calculation matrix provided by an exemplary embodiment of the present disclosure; Fig. 9 It is a schematic diagram of storing the results of a calculation matrix by column diagonal provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0015] Below, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.

[0016] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0017] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0018] It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

[0019] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0020] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.

[0021] It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0022] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0023] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0024] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0025] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0026] Figure 1 1 is a flow chart of a method for storing and processing a computing matrix based on multiple memory blocks provided in the first aspect of the embodiment of the present disclosure. This embodiment can be applied to electronic devices, such as Figure 1 As shown, the calculation matrix storage processing method 100 based on multiple memory blocks includes the following steps: Step 101 , storing each element in a calculation matrix into a preset plurality of memory banks according to a preset storage method, so that elements in the same row of the calculation matrix are stored in different memory banks and elements in the same column of the calculation matrix are stored in different memory banks.

[0027] Specifically, as described in the background technology, reference Figure 2 and Figure 3 As shown, Figure 2 The original calculation matrix is ​​stored row by row as Figure 3 In the case of , if reading by column, for example, reading 1, 5, 9, 13, it is necessary to read from address 1, address 2, address 3 and address 4 in bank0 respectively, which will cause conflicts and reduce the matrix operation speed.

[0028] In view of this, the present disclosure proposes a computing matrix storage processing method based on multiple memory blocks, which can support a matrix storage method with full bandwidth reading and writing, and can effectively improve the speed of matrix operations, especially can significantly accelerate the processing speed of matrix transposition.

[0029] There are multiple storage methods preset in the present disclosure. As long as the elements in the same row of the calculation matrix can be stored in different memory blocks and the elements in the same column can be stored in different memory blocks, it can be used as the storage method preset in the present disclosure.

[0030] In one embodiment of the present disclosure, the preset storage method may be: elements on the same diagonal line in the calculation matrix are stored in the same memory block, and elements on different diagonal lines are stored in different memory blocks. In the embodiment of the present disclosure, the diagonal lines of the calculation matrix refer to the diagonal lines of the calculation matrix and the diagonal lines that are parallel to the diagonal lines and cover the same number of elements. If the calculation matrix is ​​a non-square matrix, the diagonal lines of the calculation matrix refer to the diagonal lines after the calculation matrix is ​​expanded into a square matrix. The present disclosure takes a 4×4 calculation matrix stored in a 4-bank memory as an example. The calculation matrix is ​​a 4×4 matrix, refer to Figure 4 In one implementation, the diagonal lines of the matrix are calculated as Figure 5 As shown, matrix elements 1, 8, 11, and 14 are elements on the same diagonal line of the calculation matrix, matrix elements 2, 5, 12, and 15 are elements on the same diagonal line of the calculation matrix, matrix elements 3, 6, 9, and 16 are elements on the same diagonal line of the calculation matrix, and matrix elements 4, 7, 10, and 13 are elements on the same diagonal line of the calculation matrix. In another implementation, the diagonal lines of the calculation matrix are as follows: Figure 6 As shown, matrix elements 1, 6, 11, and 16 are elements on the same diagonal line of the calculation matrix, matrix elements 2, 7, 12, and 13 are elements on the same diagonal line of the calculation matrix, matrix elements 3, 8, 9, and 14 are elements on the same diagonal line of the calculation matrix, and matrix elements 4, 5, 10, and 15 are elements on the same diagonal line of the calculation matrix.

[0031] The present disclosure does not limit the specific read and write order for performing matrix element storage. In one implementation, the elements in the calculation matrix are stored row by row in a preset multiple memory blocks according to a preset storage method, and adjacent elements in the same row are stored in adjacent memory blocks. Furthermore, the first element of each row of the calculation matrix is ​​stored at the first address of the corresponding memory block. In another implementation, the elements in the calculation matrix are stored column by column in a preset multiple memory blocks according to a preset storage method, and adjacent elements in the same column are stored in adjacent memory blocks. Furthermore, the first element of each column of the calculation matrix is ​​stored at the first address of the corresponding memory block.

[0032] Still Figure 4 The original matrix shown is stored in a 4-bank memory as an example. Figure 5 The calculation matrix diagonal line definition shown in the figure is stored row by row in Figure 7 On 4 banks. Figure 7 The four banks shown by the dotted lines are only used to intuitively show the element storage method. For example, the element "14" in bank0 shown by the dotted line means that it is stored at "address 2" of bank0.

[0033] That is, based on Figure 5The diagonal line definition shown is as follows: Figure 7 When reading and writing the matrix row by row, the read and write addresses are generated as shown below: Assuming that the "row number" is used as the base address, the read and write addresses of the memory are generated as shown in Table 1: Table 1

[0034] That is, addresses are generated in a cyclical pattern.

[0035] In one embodiment of the present disclosure, Figure 7 In the storage method shown, when reading by row, for example, reading 1, 2, 3, 4, it is read from address 1 of bank0, address 2 of bank1, address 3 of bank2 and address 4 of bank3 respectively, supporting a storage method of full bandwidth reading and writing, which effectively improves the speed of matrix operations.

[0036] exist Figure 7 In terms of storage mode, when reading by column, for example, reading 1, 5, 9, 13, they are read from address 1 of bank0, address 1 of bank1, address 1 of bank2, and address 1 of bank3 respectively, supporting full-bandwidth reading and writing storage mode, which effectively improves the speed of matrix operations.

[0037] In one embodiment of the present disclosure, the original matrix is ​​stored column by column along the diagonal. Figure 8 The original matrix shown is stored in a 4-bank memory as an example. Figure 8 As shown, it is necessary to store diagonally by column. Specifically, based on Figure 5 The calculation matrix diagonal line definition shown in the figure is to calculate the matrix column by column according to Figure 8 The diagonal lines are stored in Fig. 9 On 4 banks. Fig. 9 The four banks shown by the dotted lines are only used to intuitively show the element storage method. For example, the element "14" in bank0 shown by the dotted line means that it is stored at "address 4" of bank0.

[0038] In one embodiment of the present disclosure, Fig. 9 In the storage method shown, when reading by column, for example, reading 1, 5, 9, 13, they are read from address 1 of bank0, address 2 of bank1, address 3 of bank2, and address 4 of bank3 respectively, supporting a storage method of full-bandwidth reading and writing, which effectively improves the speed of matrix operations.

[0039] Right now, Figure 8 and Fig. 9 When reading and writing a matrix by column, the read and write addresses are generated as follows: Assuming that the "column number" is used as the base address, the read and write addresses of the memory are generated as shown in Table 2 below: Table 2

[0040] That is, addresses are generated in a cyclical pattern.

[0041] exist Fig. 9 In terms of storage mode, when reading by row, for example, reading 1, 2, 3, 4, it is read from address 1 of bank0, address 1 of bank1, address 1 of bank2 and address 1 of bank3 respectively, supporting the storage mode of full bandwidth reading and writing, which effectively improves the speed of matrix operation.

[0042] In addition, for non-square matrices, such as NxM matrices, when N is not equal to M, the following processing is used: 1. To achieve full bandwidth reading and writing, the number of memory banks must be greater than or equal to the larger value of M and N; 2. Expand the NxM matrix into a square matrix by filling it with 0 values.

[0043] Therefore, the present disclosure proposes a computing matrix storage processing method based on multiple memory blocks. When reading and writing a matrix by row, the elements of the same row are located on different banks, so there will be no bank read and write conflicts when reading and writing the matrix by row, and full bandwidth reading and writing can be achieved. At the same time, when reading and writing a matrix by column, the elements of the same column are also located on different banks, so there will be no bank conflicts when reading and writing the matrix by column, and full bandwidth reading and writing can be achieved. When the method disclosed in the present disclosure is used to store a matrix, the matrix can be read and written with full bandwidth whether by row or by column, which significantly improves the matrix read and write performance.

[0044] According to a second aspect of an embodiment of the present disclosure, a memory controller is provided, wherein the memory controller is used to store the calculation matrix into multiple memory blocks of the memory according to a calculation matrix storage instruction based on the method described in any one of the above aspects of the present disclosure.

[0045] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a memory, a memory controller and a processor, wherein the memory comprises a plurality of memory blocks; the processor is used to send a calculation matrix storage instruction to the memory controller; the memory controller is used to store the calculation matrix into the plurality of memory blocks of the memory according to the calculation matrix storage instruction and based on the method described in any one of the above aspects of the present disclosure.

[0046] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A computing matrix storage processing method based on multiple memory blocks, characterized in that: include: Each element in the calculation matrix is ​​stored in a preset plurality of memory blocks according to a preset storage method, so that elements in the same row of the calculation matrix are respectively stored in different memory blocks and elements in the same column of the calculation matrix are respectively stored in different memory blocks.

2. The method according to claim 1, characterized in that The preset storage method includes: The elements on the same diagonal line in the calculation matrix are stored in the same memory block, and the elements on different diagonal lines are stored in different memory blocks.

3. The method according to claim 2, characterized in that The storing of each element in the calculation matrix into a plurality of preset memory blocks according to a preset storage method includes: According to a preset storage method, the elements in the calculation matrix are stored row by row in a plurality of preset memory blocks, and adjacent elements in the same row are stored in adjacent memory blocks.

4. The method according to claim 3, characterized in that The first element of each row of the calculation matrix is ​​stored in the first address of the corresponding memory block.

5. The method according to claim 2, characterized in that: The storing of each element in the calculation matrix into a plurality of preset memory blocks according to a preset storage method includes: According to a preset storage method, the elements in the calculation matrix are stored column by column in a plurality of preset memory blocks, and adjacent elements in the same column are stored in adjacent memory blocks.

6. The method according to claim 5, characterized in that The first element of each column of the calculation matrix is ​​stored in the first address of the corresponding memory block.

7. The method according to any one of claims 1 to 6, characterized in that: Before storing each element in the calculation matrix into a plurality of preset memory blocks according to a preset storage method, the method further includes: When the number of rows and columns of the calculation matrix are inconsistent, the row elements or column elements are filled by zero padding so that the number of rows and columns of the calculation matrix are consistent.

8. A memory controller, characterized in that: The memory controller is used to store the calculation matrix into multiple memory blocks of the memory according to the calculation matrix storage instruction based on the method described in any one of claims 1 to 7.

9. An electronic device, characterized in that: The electronic device comprises: A memory, a memory controller and a processor, wherein the memory includes a plurality of memory blocks; The processor is used to send a calculation matrix storage instruction to the memory controller; The memory controller is used to store the calculation matrix into multiple memory blocks of the memory according to the calculation matrix storage instruction based on the method described in any one of claims 1 to 7.

Images