Apparatus and method of processing data, electronic device, computer program and storage medium
By optimizing storage configurations and performing parallel matrix operations, the data processing device addresses inefficiencies in memory-intensive deep learning models, reducing access memory and improving performance.
Patent Information
- Application Number
- JP2024072356
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-04-28
- Filing Date
- 2024-04-26
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-04-26
AI Technical Summary
Existing deep learning models face challenges with memory-intensive operators, such as General Matrix Multiplication (GEMM), where decomposing input matrices for storage in cache units leads to increased access memory and inefficient utilization of storage space.
A data processing device and method that determines optimal storage configurations by dividing matrices into sub-matrices, calculating initial and target access memory amounts based on cache unit capacities, and performing matrix multiplication operations in parallel to enhance cache utilization and reduce memory access.
This approach effectively reduces memory access and improves the efficiency of matrix multiplication operations, optimizing the utilization of cache space and enhancing the overall performance of data processing equipment.
Smart Images

Figure 2024159724000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the field of chip technology. More specifically, the present disclosure provides a data processing device, a data processing method, an electronic device, a computer program, and a storage medium. [Background technology]
[0002] With the development of artificial intelligence technology, the operators of deep learning models can be adjusted based on the hardware resources of artificial intelligence chips. Summary of the Invention [Means for solving the problem]
[0003] The present disclosure provides a data processing device, a data processing method, an electronic device, a computer program, and a storage medium.
[0004] According to one aspect of the present disclosure, a data processing device is provided, the device including: a cache unit including a plurality of storage spaces; and a processor, the processor is configured to determine I storage space groups from the plurality of storage spaces, each including a first storage space and a second storage space; for each of the storage space groups, perform an operation of determining a plurality of first initial scale information based on a scale of a first matrix corresponding to the first storage space and a capacity of the first storage space; an operation of determining at least one second scale information related to a second matrix corresponding to the second storage space based on each of the plurality of first initial scale information; and an operation of determining the plurality of first initial access memory amounts based on the plurality of second scale information and the plurality of first initial scale information, to obtain a plurality of first initial access memory amounts corresponding to each of the storage space groups; and to determine a target access memory amount from all of the first initial access memory amounts of the I storage space groups, where I is an integer greater than or equal to 1.
[0005] According to another aspect of the present disclosure, there is provided an electronic device including a data processing device according to the present disclosure.
[0006] According to another aspect of the present disclosure, there is provided a data processing method, the method including: determining I storage space groups from a plurality of storage spaces of a cache unit, each including a first storage space and a second storage space; for each of the storage space groups, performing an operation of determining a plurality of first initial scale information based on a scale of a first matrix, which is a matrix corresponding to the first storage space, and a capacity of the first storage space; performing an operation of determining, based on each of the plurality of first initial scale information, at least one second scale information related to a second matrix, which is a matrix corresponding to the second storage space, and determining the plurality of first initial access memory amounts based on the plurality of second scale information and the plurality of first initial scale information, to obtain a plurality of first initial access memory amounts corresponding to each of the storage space groups; and determining a target access memory amount from all of the first initial access memory amounts of the I storage space groups, where I is an integer greater than or equal to 1.
[0007] According to another aspect of the present disclosure, there is provided an electronic device comprising at least one processor and a memory communicatively coupled to the at least one processor, the memory storing instructions executable by the at least one processor, the instructions being executed by the at least one processor such that the at least one processor performs a method according to the present disclosure.
[0008] According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having computer instructions stored thereon, the computer instructions causing the computer to perform a method according to the present disclosure.
[0009] According to another aspect of the present disclosure, a computer program is provided which, when executed by a processor, implements a method according to the present disclosure.
[0010] It should be understood that the contents described in this section are not intended to identify key features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will be readily understood from the following specification.
[0011] The drawings are provided for a better understanding of the invention and are not intended to limit the disclosure. [Brief description of the drawings]
[0012] [Figure 1] FIG. 1 is a schematic block diagram of a data processing device according to an embodiment of the present disclosure. [Figure 2A] FIG. 2A is a schematic diagram of first initial scale information according to an embodiment of the present disclosure. [Figure 2B] FIG. 2B is a schematic diagram of second scale information according to an embodiment of the present disclosure. [Figure 2C] FIG. 2C is a schematic diagram of third scale information according to an embodiment of the present disclosure. [Diagram 3] FIG. 3 is a schematic diagram of an electronic device according to an embodiment of the present disclosure. [Figure 4] FIG. 4 is a flowchart of a data processing method according to an embodiment of the present disclosure. [Diagram 5] FIG. 5 is a block diagram of an electronic device to which a data processing method according to an embodiment of the present disclosure can be applied. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the drawings, and various details of the embodiments of the present disclosure will be included for ease of understanding, and it should be understood that these are merely exemplary. Therefore, it should be recognized that those skilled in the art can make various changes and modifications to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, the following description will omit descriptions of known functions and structures.
[0014] Some operators of the deep learning model may be access-memory intensive operators, which may include, for example, a General Matrix Multiplication (GEMM) operator.
[0015] The general matrix multiplication operator can correspond to a multiplicand matrix A, a multiplier matrix B, and a result matrix C. The calculation process of the general matrix multiplication operator can be realized as follows:
number
[0016] The shape of the multiplicand matrix A may be m×k, the shape of the multiplier matrix B may be k×n, and the shape of the result matrix C may be m×n. The multiplicand matrix A and the multiplier matrix B may be two input matrices of a general matrix multiplication operator, respectively.
[0017] In some embodiments, either of the two input matrices may be decomposed to obtain multiple sub-matrices of the input matrix.
[0018] For example, the multiplicand matrix A may be decomposed into multiple sub-matrices. In a level 1 cache unit (Level 1 Cache, L1 Cache), multiple storage spaces are determined to store the sub-matrices of the multiplicand matrix A, the multiplier matrix B, and the result matrix C, respectively. This can improve the utilization rate of the level 1 cache means and reduce the access to the original matrix in the external storage unit (e.g., video memory).
[0019] The size of the sub-matrices of the multiplicand matrix A may be, for example, l_m×k. The sub-matrices of the multiplicand matrix A may be stored in one storage space of a level 1 cache means. A plurality of sub-matrices are respectively multiplied by the multiplier matrix B, and the access memory amount parameter value LS_A may be as follows:
number
[0020] When determining the data amount of each element in the matrix, it is understood that the product of the access memory amount parameter value and the data amount is the access memory amount. The data amount of each element may be, for example, 32 bits.
[0021] For example, the multiplier matrix B may be decomposed into multiple sub-matrices. Multiple storage spaces are determined in the level 1 cache unit to respectively store the multiplicand matrix A, the sub-matrices of the multiplier matrix B, and the result matrix C. This can improve the utilization rate of the level 1 cache unit and reduce the access to the original matrix in the external storage means (e.g., video memory).
[0022] The size of the sub-matrix of the multiplier matrix B may be, for example, k×l_n. The sub-matrix of the multiplier matrix B may be stored in one storage space of a level-one cache unit. The multiple sub-matrices are respectively multiplied by the multiplicand matrix A, and the access memory amount parameter value LS_B may be as follows:
number
[0023] Thus, when decomposing one input matrix, the minimum access memory amount parameter value LS_AorB may be as follows:
number
[0024] However, when only one input matrix is decomposed, the undecomposed matrix may not be easily stored in the level-1 cache unit, and the undecomposed matrix needs to be accessed multiple times from the external storage means, which increases the amount of memory accessed.In addition, the storage space corresponding to the result matrix is not fully utilized, resulting in insufficient utilization of the storage space.
[0025] FIG. 1 is a schematic block diagram of a data processing device according to an embodiment of the present disclosure.
[0026] As shown in FIG. 1, the device 100 may include a cache unit 110 and a processor 120 .
[0027] The cache unit 110 may include multiple storage spaces. In the embodiment of the present disclosure, the multiple storage spaces may correspond to multiple matrices, respectively. For example, the multiple storage spaces may include a storage space L1A, a storage space L1B, and a storage space L1C, which may correspond to the above multiplicand matrix A, the multiplier matrix B, and the result matrix C, respectively.
[0028] The processor 120 may be configured to determine I storage space groups from the plurality of storage spaces. In an embodiment of the present disclosure, each storage space in the I storage space groups may include a first storage space and a second storage space. I may be an integer equal to or greater than 1. For example, when the plurality of matrices is three matrices, I may be six. It is understood that the number of matrices is three is illustrative. The plurality of matrices may be two, four or more matrices.
[0029] The processor 120 may be configured to obtain multiple first initial access memory amounts corresponding to each memory space group by performing, for each memory space group, an operation of determining multiple first initial scale information based on the scale of the first matrix and the capacity of the first memory space, an operation of determining at least one second scale information based on each first initial scale information among the multiple first initial scale information, and an operation of determining multiple first initial access memory amounts based on the multiple second scale information and the multiple first initial scale information.
[0030] In an embodiment of the present disclosure, the first matrix is a matrix corresponding to the first storage space. The matrix corresponding to the first storage space may be, for example, a multiplicand matrix A, and the first storage space may be a storage space L1A. A plurality of first initial scale information may be determined based on the scale of the multiplicand matrix A and the capacity of the storage space L1A. The data amount of the sub-matrix corresponding to the first initial scale information is equal to or less than the capacity of the first storage space. The first initial scale information may include a first initial number of rows and a first initial number of columns. For example, for the multiplicand matrix A, if the number of rows m=20 and the number of columns k=20, at least two pieces of first initial scale information may be determined. The first initial scale information large_A1 may include a first initial number of rows large_am1 and a first initial number of columns large_ak1. The first initial number of rows large_am1 may be 4, and the first initial number of columns large_ak1 may be 5. The first initial scale information large_A2 may include a first initial number of rows large_am2 and a first initial number of columns large_ak2. The first initial number of rows, large_am2, may be two, and the first initial number of columns, large_ak2, may be ten.
[0031] In the embodiment of the present disclosure, the second scale information is related to a second matrix, and the second matrix is a matrix corresponding to a second storage space. Take the matrix corresponding to the second storage space as an example, the multiplier matrix B, and the second storage space may be the storage space L1B. A plurality of second scale information may be determined based on the scale of the multiplier matrix B and the capacity of the storage space L1B. The data amount of the sub-matrix corresponding to the second scale information is equal to or less than the capacity of the second storage space. The second scale information may include a second row number and a second column number. For example, for the multiplier matrix B, if the row number k=20 and the column number n=20, at least two pieces of second scale information can be determined based on the first initial scale information large_A2. The second scale information little_B1 may include a second row number little_bk1 and a second column number little_bn1. The second row number little_bk1 may be 2, and the second column number little_bn1 may be 5. The second scale information little_B2 may include a second number of rows little_bk2 and a second number of columns little_bn2. The second number of rows little_bk2 may be 5, and the second number of columns little_bn2 may be 4.
[0032] In an embodiment of the present disclosure, a first initial access memory amount is obtained based on the scale of the first matrix, the scale of the second matrix, the scale of the third matrix, the first initial scale information, and the second scale information. For example, when the first matrix is the above-mentioned multiplicand matrix A and the second matrix is the above-mentioned multiplier matrix B, the third matrix may be the above-mentioned result matrix C. One access memory amount parameter value LS_A2B1 can be determined based on the scales of the three matrices, the first initial scale information large_A2, and the second scale information little_B1. The access memory amount can be determined based on the access memory amount parameter value.
[0033] The processor 120 may be configured to determine the target access memory amount from all the first initial access memory amounts of the I storage space groups. For example, when I=6, the access memory amounts determined by different storage space groups may be different. The minimum access memory amount may be the target access memory amount.
[0034] According to the embodiments of the present disclosure, at least two matrices among a plurality of matrices associated with the matrix multiplication operator can be divided, which can effectively reduce the amount of access memory, thereby providing the execution efficiency of the matrix multiplication operator and helping to improve the efficiency of the data processing device.
[0035] As will be appreciated, the data processing device of the present disclosure has been described above, and the processor of the present disclosure will now be further described with reference to Figures 2A-2C.
[0036] FIG. 2A is a schematic diagram of first initial scale information according to an embodiment of the present disclosure.
[0037] 2A, the multiplicand matrix A 210 may have a size of m×k, the multiplier matrix B 220 may have a size of k×n, and the result matrix C 230 may have a size of m×n, for example, m=10, n=10, k=10.
[0038] In an embodiment of the present disclosure, the processor may be configured to perform an operation of determining a plurality of first initial scale information for each storage space group based on the scale of the first matrix and the capacity of the first storage space. Taking the first matrix as an example of a multiplicand matrix A210, the first initial scale information large_A3 may include a first initial number of rows large_am3 and a first initial number of columns large_ak3. The first initial number of rows large_am3 may be, for example, 2, and the first initial number of columns large_ak3 may be, for example, 5. The first initial sub-matrix 211 and the first initial sub-matrix 212 corresponding to the first initial scale information large_A3 are shown in FIG. 2A. The data amount of the first initial sub-matrix may be, for example, consistent with the storage space L1A, and may be fully utilized as the cache space set in the original matrix. According to an embodiment of the present disclosure, the data amount can be fully utilized as the cache space set in the first matrix, which is advantageous to improving the utilization rate of the cache space.
[0039] As shown in Fig. 2A, based on the first initial scale information large_A3, the second number of rows of the second initial sub-matrix of the multiplier matrix B220 may be at most large_ak3, and the second number of columns may be at most n. The data amount of the second initial sub-matrix may be greater than the capacity of the second storage space, and in order to more efficiently use the storage resources of the cache unit, second scale information related to the second matrix may be determined, which will be described below with reference to Fig. 2B.
[0040] FIG. 2B is a schematic diagram of second scale information according to an embodiment of the present disclosure.
[0041] In an embodiment of the present disclosure, the processor may be configured to perform an operation of determining at least one second scale information for each storage space group based on each first initial scale information. For example, at least one second scale information may be determined based on each first initial scale information and the capacity of the second storage space. Taking the second matrix as an example that the second matrix is a multiplier matrix B220, the second scale information little_B3 may be determined based on the first initial column number large_ak3 of the first initial scale information large_A3 and the capacity of the second storage space. The second scale information little_B3 may include a second row number little_bk3 and a second column number little_bn3. The second row number little_bk3 may be 5, and the second column number little_bn3 may be 2. A second sub-matrix 221 corresponding to the second scale information little_B3 is shown in FIG. 2B.
[0042] Then, in the embodiment of the present disclosure, the processor may be configured to perform an operation of determining, for each storage space group, a plurality of first initial access memory amounts according to a plurality of second scale information and a plurality of first initial scale information. For example, when the first matrix is a multiplicand matrix A210, when the first sub-matrix is loaded a plurality of times according to the first initial scale information large_A3, the access memory amount of the first matrix does not change, and the access memory amount parameter value Load_A may be as follows:
number
[0043] As described above, m=k=10, and the accessed memory amount parameter value Load_A may be 100.
[0044] When the second matrix is a multiplier matrix B220, when the second sub-matrix is loaded according to the second scale information little_B3, the first initial sub-matrix may be multiplied by the second sub-matrix 221, and the second initial sub-matrix 212 may be multiplied by the second sub-matrix 221 according to the matrix multiplication rule. In this way, the second sub-matrix 221 can be multiplexed. In this case, the access memory amount parameter value Load_B may be as follows:
number
[0045] For example, if little_k is the second number of rows, little_bk3, little_n is the second number of columns, little_bn3, large_m is the first initial number of rows, large_am3, and large_k is the first initial number of columns, large_ak3, then if m=k=n=10, little_bk3=5, little_bn3=2, large_am3=2, and large_ak3=5, the access memory amount parameter value Load_B may be 420.
[0046] When the third matrix is the result matrix C230 and the capacity of the third storage space is sufficient, the total access memory amount of the third matrix does not change, and the access memory amount parameter value Store_C may be as follows:
number
[0047] As described above, m=n=10, and the accessed memory amount parameter value Store_C may be 100.
[0048] For the first matrix A 210, the second matrix B 220 and the third matrix C 230, the total accessed memory amount parameter value LS_ABC may be as follows:
number
[0049] When Load_A=100, Load_B=420, and Store_C=100, the total access memory amount parameter value LS_ABC may be 620. The initial access memory amount can be determined based on the total access memory amount parameter value and the data amount of each element of the matrix.
[0050] Also, for example, by decomposing only the multiplicand matrix A, when m=k=n=10 and l_m=2 according to the above formula 2, it is possible to determine that the total access memory amount parameter value LS_A is 700. This allows at least two matrices to be decomposed, effectively reducing the access memory amount and improving memory access efficiency.
[0051] As can be seen, in the above, the first initial access memory amount is determined based on the first initial scale information large_A3 and the second scale information little_B3. In the embodiment of the present disclosure, a plurality of first initial access memory amounts can be determined based on a plurality of first initial scale information and a plurality of second scale information.
[0052] To further improve the efficiency of performing the matrix multiplication, the first initial sub-matrix may be further divided so as to perform the matrix multiplication operations in parallel, as further described below with reference to FIG.
[0053] FIG. 2C is a schematic diagram of third scale information according to an embodiment of the present disclosure.
[0054] In an embodiment of the present disclosure, the processor is further configured to perform an operation of determining, for each storage space group, a plurality of first target scale information based on a plurality of second scale information and a plurality of first initial scale information. For example, the first initial scale information corresponds to at least one of the first target scale information. Based on the second scale information corresponding to the first initial scale information and the first initial scale information, the first target scale information corresponding to the first initial scale information can be determined. As shown in FIG. 2C, when the second number of rows little_bk3 of the second scale information little_B3 is 5, at least one of the first target scale information of the first initial scale information large_A3 can be determined. The at least one of the first target scale information may include, for example, the first target scale information little_A31 and the first target scale information little_A32. The first target scale information little_A31 may include a first target number of rows little_am31 (for example, 1) and a first target number of columns little_aka31 (for example, 5). The first target size information little_A32 may include a first target number of rows little_am32 (e.g., 2) and a first target number of columns little_ak32 (e.g., 5). A first target sub-matrix 2111 corresponding to the first target size information little_A31 is shown in FIG. 2C. According to an embodiment of the present disclosure, the first sub-matrix can be further divided and the parallelism of matrix multiplication operations can be realized based on a load_compute_store parallel technique, which helps to improve the operation efficiency of the data processing device.
[0055] In an embodiment of the present disclosure, the processor may be further configured to perform an operation of determining, for each storage space group, at least one second initial access memory amount from the plurality of first initial access memory amounts corresponding to each storage space group based on the plurality of first target size information.
[0056] For example, the processor may be configured to perform an operation of determining, for each storage space group, a plurality of third size information based on the plurality of second size information and the plurality of first target size information. The plurality of third size information is associated with a third matrix, and the third matrix is a matrix corresponding to a third storage space in each storage space group. When the third matrix is the result matrix C230, a third size information little_C32 can be determined based on the second size information little_B3 and the first target size information little_A32. A third sub-matrix 2311 corresponding to the third size information little_C32 is shown in FIG. 2C.
[0057] For example, the processor may be configured to perform an operation of determining, for each storage space group, at least one second initial access memory amount from the plurality of first initial access memory amounts based on the plurality of first target size information and the plurality of third size information. At least one second initial access memory amount is determined from the plurality of first initial access memory amounts based on the parameters of the storage means, the first target size information and the plurality of third size information to satisfy the storage alignment constraint, satisfy the storage characteristics and avoid storage channel contention. The storage means may be, for example, a video memory. According to the embodiment of the present disclosure, the size of the matrix corresponding to the second access memory amount can be matched with the storage means, which is helpful in improving the stability of the data processing device.
[0058] As can be understood, the above describes the present disclosure by taking as an example that the first matrix is a multiplicand matrix A and the second matrix is a multiplier matrix B. However, the present disclosure is not limited thereto, and when the first matrix is a multiplicand matrix A, the second matrix may be a result matrix C, which will be further described below.
[0059] In an embodiment of the present disclosure, for example, after the first initial scale information large_A3 is determined, the processor may be configured to perform an operation of determining at least one second scale information for each storage space group based on each of the plurality of first initial scale information. For example, when the second matrix is a result matrix C, based on the first initial number of rows large_am3 of the first initial scale information large_A3, the second number of rows of the second sub-matrix of the result matrix C may be at most large_am3, and the second number of columns may be at most n. Based on the first initial number of rows large_am3 of the first initial scale information large_A3 and the capacity of the second storage space, the second scale information little_C3 may be determined. The second scale information little_C3 may include a second number of rows little_cm3 and a second number of columns little_cn3. The second number of rows little_cm3 may be equal to or less than the first initial number of rows large_am3. The second number of columns little_cn3 may be smaller than n.
[0060] The first initial access memory amount can be determined based on the first initial scale information large_A3 and the second scale information little_C3.
[0061] The first initial sub-matrix may then be further divided to perform matrix multiplication operations in parallel. In an embodiment of the present disclosure, the processor is configured to perform an operation of determining, for each storage space group, a plurality of first target size information based on the plurality of second size information and the plurality of first initial size information. For example, at least one first target size information of the first initial size information large_A3 may be determined based on the second number of rows little_cm3 of the second size information little_C3. The at least one first target size information may include, for example, first target size information little_A33 and first target size information little_A34. The first target size information little_A33 may include a first target number of rows little_am33 and a first target number of columns little_ak33. The first target size information little_A34 may include a first target number of rows little_am34 and a first target number of columns little_ak34. The first target number of rows may be equal to or less than the second number of rows. The first target number of columns may be less than the first initial number of columns.
[0062] In an embodiment of the present disclosure, the processor may be configured to perform an operation of determining, for each storage space group, a plurality of third scale information pieces based on a plurality of second scale information pieces and a plurality of first target scale information pieces. For example, when the third matrix is a multiplier matrix B, the third scale information little_B34 may be determined based on the second scale information little_C3 and the first target scale information little_A34. The third number of rows little_bk34 of the third scale information little_B34 may be, for example, equal to the first target number of columns little_ak34 of the first target scale information little_A34. The third number of columns little_bn34 of the third scale information little_B34 may be, for example, equal to the second number of columns little_cn3 of the second scale information little_C3.
[0063] For the sake of understanding, the above description of the present disclosure takes the first matrix as an example of a multiplicand matrix A. However, the present disclosure is not limited thereto, and the first matrix may be a multiplier matrix B or a result matrix C. Hereinafter, the present disclosure will be further described by taking the first matrix as a multiplier matrix B as an example.
[0064] In an embodiment of the present disclosure, the processor may be configured to perform an operation of determining, for each storage space group, a plurality of first initial scale information based on the scale of the first matrix and the capacity of the first storage space. When the first matrix is a multiplier matrix B, the first initial scale information large_B4 may include a first initial row number large_bk4 and a first initial column number large_bn4. The data amount of the first initial sub-matrix corresponding to the first initial scale information large_B4 may, for example, be consistent with the storage space L1B, and may be fully utilized as a cache space set for the original matrix.
[0065] In an embodiment of the present disclosure, the processor may be configured to perform an operation of determining, for each storage space group, at least one second scale information based on each first initial scale information. For example, at least one second scale information may be determined based on each first initial scale information and the capacity of the second storage space. Taking the second matrix as an example that the second matrix is a multiplicand matrix A, based on the first initial scale information large_B4, the second number of rows of the second sub-matrix of the multiplicand matrix A may be at most m, and the second number of columns may be at most large_bk4. Based on the first initial number of rows large_bk4 of the first initial scale information large_B4 and the capacity of the second storage space, the second scale information little_A4 may be determined. The second scale information little_A4 may include a second number of rows little_am4 and a second number of columns little_ak4. The second number of columns little_ak4 may be equal to or less than the first initial number of rows large_bk4. The second number of rows little_am3 may be less than m.
[0066] The first initial sub-matrix may then be further divided to perform matrix multiplication operations in parallel. In an embodiment of the present disclosure, the processor is configured to perform an operation of determining, for each storage space group, a plurality of first target size information based on the plurality of second size information and the plurality of first initial size information. For example, at least one first target size information of the first initial size information large_B4 may be determined based on the second number of columns little_ak4 of the second size information little_A4. The at least one first target size information may include, for example, first target size information little_B41 and first target size information little_B42. The first target size information little_B41 may include a first target number of rows little_bk41 and a first target number of rows little_bn41. The first target size information little_B42 may include a first target number of rows little_bk42 and a first target number of columns little_bn42. The first target number of columns may be smaller than the first initial number of columns. The first target number of rows may be less than or equal to the second number of columns.
[0067] In an embodiment of the present disclosure, the processor may be configured to perform an operation of determining, for each storage space group, a plurality of third size information based on a plurality of second size information and a plurality of first target size information. For example, when the third matrix is a result matrix C, the third size information little_C41 may be determined based on the second size information little_A4 and the first target size information little_B41. The third number of columns little_cn41 of the third size information little_C41 may be, for example, equal to the first target number of columns little_bn41 of the first target size information little_B41. The third number of rows little_cm41 of the third size information little_C41 may be, for example, equal to the second number of rows little_am4 of the second size information little_A4.
[0068] For understanding, the above is taken as an example that the first matrix is a multiplier matrix B and the second matrix is a multiplicand matrix A, and the present disclosure is further described. However, the present disclosure is not limited thereto, and when the first matrix is a multiplier matrix B, the second matrix may be a result matrix C, which will be further described below.
[0069] In an embodiment of the present disclosure, for example, after the first initial scale information large_B4 is determined, the processor may be configured to perform an operation of determining at least one second scale information for each storage space group based on each of the plurality of first initial scale information. For example, when the second matrix is a result matrix C, the second number of rows of the second sub-matrix of the result matrix C may be at most m and the second number of columns may be at most large_bn4 based on the first initial number of columns large_bn4 of the first initial scale information large_B4. The second scale information little_C4 may be determined based on the first initial number of columns large_bn4 of the first initial scale information large_B4 and the capacity of the second storage space. The second scale information little_C4 may include a second number of rows little_cm4 and a second number of columns little_cn4. The second number of rows little_cm3 may be smaller than m. The second number of columns little_cn3 may be equal to or smaller than the second number of columns little_cn4.
[0070] The first initial access memory amount can be determined based on the first initial scale information large_B4 and the second scale information little_C4.
[0071] The first initial sub-matrix may then be further divided to perform matrix multiplication operations in parallel. In an embodiment of the present disclosure, the processor is configured to perform an operation of determining, for each storage space group, a plurality of first target size information based on the plurality of second size information and the plurality of first initial size information. For example, at least one first target size information of the first initial size information large_B4 may be determined based on the second number of columns little_cn4 of the second size information little_C4. The at least one first target size information may include, for example, the first target size information little_B43 and the first target size information little_B44. The first target size information little_B43 may include a first target number of rows little_bk43 and a first target number of columns little_bn43. The first target size information little_B44 may include a first target number of rows little_bk44 and a first target number of rows little_bn44. The first target number of columns may be equal to or less than the second number of columns. The first target number of rows may be less than the first initial number of rows.
[0072] In an embodiment of the present disclosure, the processor may be configured to perform an operation of determining, for each storage space group, a plurality of third size information pieces based on a plurality of second size information pieces and a plurality of first target size information pieces. For example, when the third matrix is a multiplicand matrix A, the third size information little_A44 may be determined based on the second size information little_C4 and the first target size information little_B44. The third number of rows little_am44 of the third size information little_A44 may be, for example, equal to the second number of rows little_cm4 of the second size information little_C4. The third number of columns little_ak44 of the third size information little_A44 may be, for example, equal to the first target number of rows little_bk44 of the first target size information little_B44.
[0073] As can be understood, the above description of the present disclosure takes the example that the first matrix is a multiplicand matrix A or a multiplier matrix B. However, the present disclosure is not limited thereto, and the first matrix may be a result matrix C, which will be further described below.
[0074] In an embodiment of the present disclosure, the processor may be configured to perform an operation of determining, for each storage space group, a plurality of first initial size information based on the size of the first matrix and the capacity of the first storage space. When the first matrix is a result matrix C, the first initial size information large_C5 may include a first initial row number large_cm5 and a first initial column number large_cnt5. The data amount of the first initial sub-matrix of the result matrix C may, for example, be consistent with the storage space L1C, and may be fully utilized as a cache space set for the original matrix.
[0075] In an embodiment of the present disclosure, the processor may be configured to perform an operation of determining, for each storage space group, at least one second scale information based on each first initial scale information. For example, at least one second scale information may be determined based on each first initial scale information and the capacity of the second storage space. Taking the second matrix as an example that the second matrix is a multiplicand matrix A, based on the first initial scale information large_C5, the second number of rows of the second sub-matrix of the multiplicand matrix A may be at most large_cm5, and the second number of columns may be at most k. Based on the first initial number of rows large_cm5 of the first initial scale information large_C5 and the capacity of the second storage space, the second scale information little_A5 may be determined. The second scale information little_A5 may include a second number of rows little_am5 and a second number of columns little_ak5. The second number of rows little_am5 may be equal to or less than the first initial number of rows large_cm5. The second number of columns little_aka5 may be less than k.
[0076] The first initial sub-matrix may then be further divided to perform matrix multiplication operations in parallel. In an embodiment of the present disclosure, the processor may perform an operation of determining, for each storage space group, a plurality of first target size information based on the plurality of second size information and the plurality of first initial size information. For example, at least one first target size information of the first initial size information large_C5 may be determined based on the second number of rows little_am5 of the second size information little_A5. The at least one first target size information may include, for example, first target size information little_C51 and first target size information little_C52. The first target size information little_C51 may include a first target number of rows little_cm51 and a first target number of columns little_cn51. The first target size information little_C52 may include a first target number of rows little_cm52 and a first target number of rows little_cn52. The first target number of columns may be smaller than the first initial number of columns. The first target number of rows may be less than or equal to the second number of rows.
[0077] In an embodiment of the present disclosure, the processor may be configured to perform an operation of determining, for each storage space group, a plurality of third scale information pieces based on a plurality of second scale information pieces and a plurality of first target scale information pieces. For example, when the third matrix is a multiplier matrix B, the third scale information little_B51 may be determined based on the second scale information little_A5 and the first target scale information little_C51. The third number of columns little_bn51 of the third scale information little_B51 may be, for example, equal to the first target number of columns little_cn51 of the first target scale information little_C51. The third number of rows little_bk51 of the third scale information little_B51 may be, for example, equal to the second number of columns little_ak5 of the second scale information little_A5.
[0078] For understanding, the above takes the example that the first matrix is the result matrix C and the second matrix is the multiplicand matrix A to further describe the present disclosure. However, the present disclosure is not limited thereto, and when the first matrix is the result matrix C, the second matrix may be the multiplier matrix B, which will be further described below.
[0079] In an embodiment of the present disclosure, for example, after the first initial scale information large_C5 is determined, the processor may be configured to perform an operation of determining at least one second scale information for each storage space group based on each of the plurality of first initial scale information. For example, if the second matrix is a multiplier matrix B, the second number of rows of the second sub-matrix of the multiplier matrix B may be at most k and the second number of columns may be at most large_cn5 based on the first initial number of columns large_cn5 of the first initial scale information large_C5. The second scale information little_B5 may be determined based on the first initial number of columns large_cn5 of the first initial scale information large_C5 and the capacity of the second storage space. The second scale information little_B5 may include a second number of rows little_bk5 and a second number of columns little_bn5. The second number of rows little_bk5 may be smaller than k, and the second number of columns little_bn5 may be equal to or smaller than the first initial number of columns large_cn5.
[0080] The first initial access memory amount may be determined based on the first initial scale information large_C5 and the second scale information little_B5.
[0081] The first initial sub-matrix may then be further divided to perform matrix multiplication operations in parallel. In an embodiment of the present disclosure, the processor is configured to perform, for each storage space group, an operation of determining a plurality of first target size information based on the plurality of second size information and the plurality of first initial size information. For example, at least one first target size information of the first initial size information large_C5 may be determined based on the second number of columns little_bn5 of the second size information little_B5. The at least one first target size information may include, for example, first target size information little_C53 and first target size information little_C54. The first target size information little_C53 may include a first target number of rows little_cm53 and a first target number of columns little_cn53. The first target size information little_C54 may include a first target number of rows little_cm54 and a first target number of columns little_cn54. The first target number of columns may be equal to or less than the second number of columns. The first target number of rows may be less than the first initial number of rows.
[0082] In an embodiment of the present disclosure, the processor may be configured to perform an operation of determining, for each storage space group, a plurality of third size information pieces based on a plurality of second size information pieces and a plurality of first target size information pieces. For example, when the third matrix is a multiplicand matrix A, the third size information little_A54 may be determined based on the second size information little_B5 and the first target size information little_C54. The third number of columns little_ak54 of the third size information little_A54 may be, for example, equal to the second number of rows little_bk5 of the second size information little_B5. The third number of rows little_am54 of the third size information little_A54 may be, for example, equal to the first target number of rows little_cm54 of the first target size information little_C54.
[0083] As can be understood, above, several forms for determining the size of each matrix are described, and below, several forms for determining the target access memory amount are described with reference to related embodiments.
[0084] In some embodiments, the processor may be configured to determine the target access memory amount from all the second initial access memory amounts of the I storage space groups. For example, the smallest second initial access memory amount may be set as the target access memory amount.
[0085] As can be appreciated, above several ways of determining the target access memory amount are described, and below several ways of performing the matrix multiplication are described.
[0086] In an embodiment of the present disclosure, a matrix multiplication operation is performed based on first target scale information, second scale information, and third scale information corresponding to a target access memory amount. The first matrix corresponding to the target access memory amount may be a first target matrix. The second matrix corresponding to the target access memory amount may be a second target matrix. The third matrix corresponding to the target access memory amount may be a third target matrix.
[0087] In an embodiment of the present disclosure, the processor may be configured to, when the third target matrix is a result matrix, load a first sub-matrix of the first target matrix into the first storage space according to first target size information corresponding to the target access memory amount; load a second sub-matrix of the second target matrix into the second storage space according to second size information corresponding to the target access memory amount; perform a matrix multiplication operation on the first sub-matrix and the second sub-matrix to obtain a third sub-matrix of the third target matrix; and write the third sub-matrix into the third storage space.
[0088] For example, the first target scale information, the second scale information, and the third scale information corresponding to the target access memory amount are the above-mentioned first target scale information little_A32, the second scale information little_B3, and the third scale information little_C32, respectively. According to the first target scale information little_A32 corresponding to the target access memory amount, a first sub-matrix of the multiplicand matrix A is loaded into the memory space L1A. The scale of the first sub-matrix may be consistent with the first target scale information little_A32. According to the second scale information little_B3 corresponding to the target access memory amount, a second sub-matrix of the second target matrix is loaded into the memory space L1B. The scale of the second sub-matrix may be consistent with the second scale information little_B3. A third sub-matrix of the result matrix C is obtained by performing matrix multiplication on the first sub-matrix and the second sub-matrix, and the scale of the third sub-matrix may be consistent with the third scale information little_C32. The third sub-matrix may be written into the memory space L1C.
[0089] Also, for example, the first target scale information, the second scale information, and the third scale information corresponding to the target access memory amount are the above-mentioned first target scale information little_B41, the second scale information little_A4, and the third scale information little_C41, respectively. According to the first target scale information little_B41 corresponding to the target access memory amount, a first sub-matrix of the multiplier matrix B is loaded into the memory space L1B. The scale of the first sub-matrix may be consistent with the first target scale information little_B41. According to the second scale information little_A4 corresponding to the target access memory amount, a second sub-matrix of the second target matrix is loaded into the memory space L1A. The scale of the second sub-matrix may be consistent with the second scale information little_A4. A third sub-matrix of the result matrix C is obtained by performing a matrix multiplication operation on the first sub-matrix and the second sub-matrix, and the scale of the third sub-matrix may be consistent with the third scale information little_C41. The third sub-matrix may be written into the memory space L1C.
[0090] In an embodiment of the present disclosure, the processor is further configured to: when the first target matrix is a result matrix, load a third sub-matrix of the third target matrix into a third storage space according to third size information corresponding to a target access memory amount; load a second sub-matrix of the second target matrix into a second storage space according to second size information corresponding to a target access memory amount; perform a matrix multiplication operation on the third sub-matrix and the second sub-matrix to obtain a first sub-matrix of the first target matrix; and write the first sub-matrix into the first storage space.
[0091] For example, the first target scale information, the second scale information, and the third scale information corresponding to the target access memory amount are the above-mentioned first target scale information little_C51, the second scale information little_A5, and the third scale information little_B51, respectively. According to the third scale information little_B51 corresponding to the target access memory amount, a third sub-matrix of the multiplier matrix B is loaded into the memory space L1B. The scale of the third sub-matrix may be consistent with the third scale information little_B51. According to the second scale information little_A5 corresponding to the target access memory amount, a second sub-matrix of the multiplicand matrix A is loaded into the memory space L1A. The scale of the second sub-matrix may be consistent with the second scale information little_A5. A matrix multiplication operation is performed on the first sub-matrix and the second sub-matrix to obtain a first sub-matrix of the result matrix C, and the scale of the first sub-matrix may be consistent with the first target scale information little_C51. The first sub-matrix may be written into the memory space L1C.
[0092] For example, the first target scale information, the second scale information, and the third scale information corresponding to the target access memory amount are the above-mentioned first target scale information little_C54, the second scale information little_B5, and the third scale information little_A54, respectively. According to the third scale information little_A54 corresponding to the target access memory amount, a third sub-matrix of the multiplicand matrix A is loaded into the memory space L1A. The scale of the third sub-matrix may be consistent with the third scale information little_A54. According to the second scale information little_B5 corresponding to the target access memory amount, a second sub-matrix of the multiplier matrix B is loaded into the memory space L1B. The scale of the second sub-matrix may be consistent with the second scale information little_B5. A matrix multiplication operation is performed on the first sub-matrix and the second sub-matrix to obtain a first sub-matrix of the result matrix C, and the scale of the first sub-matrix may be consistent with the first target scale information little_C54. The first sub-matrix may be written into the memory space L1C.
[0093] In an embodiment of the present disclosure, the processor may be further configured to, when the second target matrix is a result matrix, load a first sub-matrix of the first target matrix into the first storage space according to first target size information corresponding to the target access memory amount; load a third sub-matrix of the third target matrix into the third storage space according to third size information corresponding to the target access memory amount; perform a matrix multiplication operation on the first sub-matrix and the third sub-matrix to obtain a second sub-matrix of the second target matrix; and write the second sub-matrix into the second storage space.
[0094] For example, the first target scale information, the second scale information, and the third scale information corresponding to the target access memory amount are the above-mentioned first target scale information little_A34, the second scale information little_C3, and the third scale information little_B34, respectively. According to the first target scale information little_A34 corresponding to the target access memory amount, a first sub-matrix of the multiplicand matrix A is loaded into the memory space L1A. The scale of the first sub-matrix may be consistent with the first target scale information little_A34. According to the third scale information little_B34 corresponding to the target access memory amount, a third sub-matrix of the multiplier matrix is loaded into the memory space L1B. The scale of the third sub-matrix may be consistent with the third scale information little_B34. A second sub-matrix of the result matrix C is obtained by performing a matrix multiplication operation on the first sub-matrix and the third sub-matrix, and the scale of the second sub-matrix may be consistent with the second scale information little_C3. The second sub-matrix may be written into the memory space L1C.
[0095] For example, the first target scale information, the second scale information, and the third scale information corresponding to the target access memory amount are the above-mentioned first target scale information little_B44, the second scale information little_C4, and the third scale information little_A44, respectively. According to the first target scale information little_B44 corresponding to the target access memory amount, a first sub-matrix of the multiplier matrix B is loaded into the memory space L1B. The scale of the first sub-matrix may be consistent with the first target scale information little_B44. According to the third scale information little_A44 corresponding to the target access memory amount, a third sub-matrix of the multiplier matrix is loaded into the memory space L1B. The scale of the third sub-matrix may be consistent with the third scale information little_A44. A matrix multiplication operation is performed on the first sub-matrix and the third sub-matrix to obtain a second sub-matrix of the result matrix C, and the scale of the second sub-matrix may be consistent with the second scale information little_C4. The second sub-matrix may be written into the memory space L1C.
[0096] As can be appreciated, above, a data processing device of the present disclosure is described, and below, an electronic device including the data processing device is described.
[0097] FIG. 3 is a schematic diagram of an electronic device according to an embodiment of the present disclosure.
[0098] 3, the electronic device 30 may include a data processing device 300 provided in the present disclosure. The data processing device 300 may be, for example, the device 100 described above.
[0099] For the sake of understanding, the electronic device of the present disclosure has been described above, and the data processing method of the present disclosure will now be described.
[0100] FIG. 4 is a flowchart of a data processing method according to an embodiment of the present disclosure.
[0101] As shown in FIG. 4, the method 400 may include operations S410 to S430.
[0102] In operation S410, a group of I storage spaces is determined from the plurality of storage spaces of the cache unit. In an embodiment of the present disclosure, each of the group of I storage spaces includes a first storage space and a second storage space.
[0103] In operation S420, the following operation is performed for each storage space group to obtain a plurality of first initial-access memory amounts corresponding to each storage space group.
[0104] In operation S421, a plurality of first initial scale information is determined according to a scale of a first matrix and a capacity of a first storage space. In an embodiment of the present disclosure, the first matrix is a matrix corresponding to the first storage space.
[0105] In operation S422, at least one second scale information is determined based on each of the plurality of first initial scale information. In an embodiment of the present disclosure, the second scale information is related to a second matrix, and the second matrix is a matrix corresponding to a second storage space.
[0106] In operation S423, a plurality of first initial access memory amounts are determined based on the plurality of second scale information and the plurality of first initial scale information.
[0107] In operation S430, a target access memory amount is determined from all the first initial access memory amounts of the I storage space groups, where I is an integer equal to or greater than 1.
[0108] As can be appreciated, the method 400 can be performed by the processor 120 described above.
[0109] In some embodiments, the operation performed for each storage space group further includes determining a plurality of first target size information based on the plurality of second size information and the plurality of first initial size information. For example, the first initial size information corresponds to the at least one first target size information. Determine at least one second initial access memory amount from the plurality of first initial access memory amounts corresponding to each storage space group based on the plurality of first target size information.
[0110] In some embodiments, determining at least one second initial access memory amount from the plurality of first initial access memory amounts corresponding to each storage space group includes determining a plurality of third size information based on the plurality of second size information and the plurality of first target size information. For example, the plurality of third size information is associated with a third matrix, and the third matrix is a matrix corresponding to a third storage space in each storage space group. Determining at least one second initial access memory amount from the plurality of first initial access memory amounts based on the plurality of first target size information and the plurality of third size information.
[0111] In some embodiments, determining the target access memory amount from the first initial access memory amounts of all of the I storage space groups includes determining the target access memory amount from the second initial access memory amounts of all of the I storage space groups.
[0112] In some embodiments, the method 400 further includes performing a matrix multiplication operation based on the first target size information, the second size information, and the third size information corresponding to the target access memory amount.
[0113] In some embodiments, the multiple matrices include a multiplier matrix, a multiplicand matrix and a resultant matrix, and the first matrix, the second matrix and the third matrix corresponding to the target access memory amount are a first target matrix, a second target matrix and a third target matrix, respectively.
[0114] In some embodiments, the operation of performing matrix multiplication includes, when the third target matrix is a result matrix, loading a first sub-matrix of the first target matrix into a first storage space according to first target size information corresponding to a target amount of access memory; loading a second sub-matrix of the second target matrix into a second storage space according to second size information corresponding to a target amount of access memory; performing a matrix multiplication operation on the first sub-matrix and the second sub-matrix to obtain a third sub-matrix of the third target matrix; and writing the third sub-matrix into the third storage space.
[0115] In some embodiments, the operation of performing matrix multiplication includes, when the first target matrix is a result matrix, loading a third sub-matrix of the third target matrix into a third storage space according to third size information corresponding to a target access memory amount; loading a second sub-matrix of the second target matrix into a second storage space according to second size information corresponding to a target access memory amount; performing a matrix multiplication operation on the third sub-matrix and the second sub-matrix to obtain a first sub-matrix of the first target matrix; and writing the first sub-matrix into the first storage space.
[0116] In some embodiments, the operation of performing matrix multiplication includes, when the second target matrix is a result matrix, loading a first sub-matrix of the first target matrix into a first storage space according to first target size information corresponding to a target amount of access memory; loading a third sub-matrix of the third target matrix into a third storage space according to third size information corresponding to a target amount of access memory; performing a matrix multiplication operation on the first sub-matrix and the third sub-matrix to obtain a second sub-matrix of the second target matrix; and writing the second sub-matrix into the second storage space.
[0117] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure and other processing of such user personal information shall all comply with the provisions of relevant laws and shall not violate public order and morals.
[0118] According to an embodiment of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium, and a computer program product.
[0119] 5 shows an exemplary block diagram for implementing an exemplary electronic device 500 according to an embodiment of the present disclosure. The electronic device 500 is intended to represent various types of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various types of mobile devices, such as personal digital assistants, mobile phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure as described and / or claimed herein.
[0120] 5, the electronic device 500 includes a computing means 501, which can execute various appropriate operations and processes based on a computer program stored in a read-only memory (ROM) 502 or loaded from a storage means 508 into a random access memory (RAM) 503. The RAM 503 can further store various programs and data necessary for the operation of the electronic device 500. The computing means 501, the ROM 502, and the RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.
[0121] A number of components in the electronic device 500 are connected to an I / O interface 505, and include input means 506, such as a keyboard, a mouse, etc., output means 507, such as various types of displays, speakers, etc., storage means 508, such as a magnetic disk, an optical disk, etc., and communication means 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication means 509 enables the electronic device 500 to exchange information / data with other devices via a computer network, such as the Internet, and / or various types of electric communication networks.
[0122] The computing means 501 may be various general-purpose and / or dedicated processing modules having processing and computing capabilities. Some examples of the computing means 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various machine learning model algorithm computing means, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing means 501 executes each of the methods and processes described above, such as the data processing methods. For example, in some embodiments, the data processing methods may be realized as a computer software program tangibly embodied in a machine-readable medium, such as the storage means 508. In some embodiments, some or all of the computer programs may be loaded and / or installed in the electronic device 500 via the ROM 1002 and / or the communication means 509. When the computer programs are loaded into the RAM 1003 and executed by the computing means 501, one or more steps of the data processing methods described above may be executed. Alternatively, in another embodiment, the computing means 501 may be configured to perform the data processing method in any other suitable manner (eg via firmware).
[0123] Various embodiments of the systems and techniques described herein may be realized in digital electronic circuitry systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may be embodied in one or more computer programs that may be executed and / or interpreted by a programmable system that includes at least one programmable processor, which may be a special purpose or general purpose programmable processor, and may include a processor that is capable of receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
[0124] The program codes for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, a special purpose computer or other programmable data processing apparatus, so that when the program codes are executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are performed. The program codes may be executed entirely on the device, partially on the device, partially on the device as a separate software package and partially on a remote device or entirely on a remote device or server.
[0125] In the context of this disclosure, a machine-readable medium may be a tangible medium, and may include or store a program for use in or in combination with an instruction execution system, apparatus, or electronic device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or electronic device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections through one or more wires, portable computer disks, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), optical fibers, compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0126] A computer may implement the systems and techniques described herein to provide interaction with a user, and may include a display device (e.g., a CRT (cathode ray tube) display or LCD (liquid crystal display)) for displaying information to a user, and a keyboard and pointing device (e.g., a mouse or trackball) through which a user may provide input to the computer. Other types of devices may also provide interaction with a user, for example, the feedback provided to the user may be any form of sensing feedback (e.g., visual feedback, auditory feedback, or tactile feedback) and may receive input from the user in any form (including voice input, speech input, or tactile input).
[0127] The systems and techniques described herein may be implemented in a computing system including background components (e.g., a data server), or a computing system including middleware components (e.g., an application server), or a computing system including front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user may interact with embodiments of the systems and techniques described herein), or a computing system including any combination of such background, middleware, or front-end components. The components of the system may be connected to each other by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks illustratively include a local area network (LAN), a wide area network (WAN), and the Internet.
[0128] The computer system may include clients and servers. The clients and servers are generally remote and typically interact through a communication network. The relationship of client and server is created by computer programs running on the corresponding computers and having a client-server relationship.
[0129] It should be understood that various types of flows shown above may be used, and steps may be rearranged, added, or deleted. For example, each step described in the present invention may be performed in parallel, sequentially, or in a different order, and the present specification is not limited thereto as long as the desired results of the technical solution of the present disclosure can be achieved.
[0130] The specific embodiments described above do not limit the scope of protection of the present disclosure. Those skilled in the art should understand that various modifications, combinations, subcombinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principle of the present disclosure should be included within the scope of protection of the present disclosure.
Claims
1. 1. A data processing device, comprising: a cache unit including a plurality of storage spaces; a processor; The processor, determining I (I is an integer equal to or greater than 1) storage space groups from the plurality of storage spaces, each of the storage space groups including a first storage space and a second storage space; For each of the groups of storage spaces, determining a plurality of first initial scale information based on a scale of a first matrix, the first matrix corresponding to the first storage space, and a capacity of the first storage space; determining at least one second scale information associated with a second matrix, the second matrix corresponding to the second storage space, based on each of the plurality of first initial scale information; Executing an operation of determining the first initial access memory amounts based on the second information and the first initial scale information, obtaining a plurality of first initial access memory amounts corresponding to each of the groups of storage spaces; determining a target access memory amount from all the first initial access memory amounts of the I groups of storage spaces; Data processing device.
2. The processor further performs the following for each of the storage space groups: determining a plurality of pieces of first target scale information based on the plurality of pieces of second scale information and the plurality of pieces of first initial scale information corresponding to at least one piece of first target scale information; and performing an operation of determining at least one second initial access memory amount from a plurality of first initial access memory amounts corresponding to each of the storage space groups based on a plurality of the first target size information.
2. The apparatus of claim 1.
3. The processor further performs the following for each of the storage space groups: determining a plurality of third size information related to a third matrix, the third matrix being a matrix corresponding to a third storage space in each of the storage space groups, based on the plurality of second size information and the plurality of first target size information; and performing an operation of determining the at least one second initial access memory amount from the plurality of first initial access memory amounts based on a plurality of the first target size information and a plurality of the third size information.
3. The apparatus of claim 2.
4. the processor is further configured to determine the target access memory amount from all second initial access memory amounts of the I group of storage spaces.
4. The apparatus of claim 3.
5. The processor further comprises: performing a matrix multiplication operation based on the first target size information, the second size information, and the third size information corresponding to the target access memory amount.
5. The apparatus of claim 4.
6. the plurality of matrices includes a multiplier matrix, a multiplicand matrix, and a result matrix; the first matrix, the second matrix, and the third matrix corresponding to the target access memory amount are a first target matrix, a second target matrix, and a third target matrix, respectively; 6. The apparatus of claim 5.
7. The processor further comprises: When the third target matrix is the result matrix, loading a first sub-matrix of the first target matrix into the first storage space according to the first target size information corresponding to the target amount of accessed memory; loading a second sub-matrix of the second target matrix into the second storage space according to the second size information corresponding to the target access memory amount; performing a matrix multiplication operation on the first sub-matrix and the second sub-matrix to obtain a third sub-matrix of the third target matrix; configured to write the third sub-matrix to the third storage space; 7. The apparatus of claim 6.
8. The processor further comprises: When the first target matrix is the result matrix, loading a third sub-matrix of the third target matrix into the third storage space according to the third size information corresponding to the target access memory amount; loading a second sub-matrix of the second target matrix into the second storage space according to the second size information corresponding to the target access memory amount; performing a matrix multiplication operation on the third sub-matrix and the second sub-matrix to obtain a first sub-matrix of the first target matrix; configured to write the first sub-matrix to the first storage space; 7. The apparatus of claim 6.
9. The processor further comprises: When the second target matrix is the result matrix, loading a first sub-matrix of the first target matrix into the first storage space according to the first target size information corresponding to the target amount of accessed memory; loading a third sub-matrix of the third target matrix into the third storage space according to the third size information corresponding to the target access memory amount; performing a matrix multiplication operation on the first sub-matrix and the third sub-matrix to obtain a second sub-matrix of the second target matrix; configured to write the second sub-matrix to the second storage space; 7. The apparatus of claim 6.
10. An electronic device comprising the data processing device according to any one of claims 1 to 9.
11. 1. A data processing method comprising the steps of: determining I (I is an integer equal to or greater than 1) groups of storage spaces from the plurality of storage spaces of the cache unit, each group including a first storage space and a second storage space; For each of the groups of storage spaces, determining a plurality of first initial scale information based on a scale of a first matrix, the first matrix corresponding to the first storage space, and a capacity of the first storage space; determining at least one second scale information associated with a second matrix, the second matrix corresponding to the second storage space, based on each of the plurality of first initial scale information; Executing an operation to determine the first initial access memory amounts based on the second scale information and the first initial scale information, thereby obtaining the first initial access memory amounts corresponding to each of the storage space groups; determining a target access memory amount from all of the first initial access memory amounts of the I groups of storage spaces. Data processing methods.
12. For each of the groups of storage spaces, determining a plurality of pieces of first target scale information based on the plurality of pieces of second scale information and the plurality of pieces of first initial scale information corresponding to at least one piece of first target scale information; and determining at least one second initial access memory amount from a plurality of first initial access memory amounts corresponding to each of the storage space groups based on a plurality of first target size information. The method of claim 11.
13. Determining at least one second initial access memory amount from the plurality of first initial access memory amounts corresponding to each of the groups of storage spaces includes: determining a plurality of third size information related to a third matrix, the third matrix being a matrix corresponding to a third storage space in each of the storage space groups, based on the plurality of second size information and the plurality of first target size information; determining the at least one second initial access memory amount from the plurality of first initial access memory amounts based on a plurality of the first target size information and a plurality of the third size information; The method of claim 12.
14. Determining a target access memory amount from all first initial access memory amounts of the I groups of storage spaces includes: determining the target access memory amount from all second initial access memory amounts of the I group of storage spaces; The method of claim 13.
15. performing a matrix multiplication operation based on the first target size information, the second size information, and the third size information corresponding to the target access memory amount. The method of claim 14.
16. the plurality of matrices include a multiplier matrix, a multiplicand matrix, and a result matrix, and the first matrix, the second matrix, and the third matrix corresponding to the target access memory amount are a first target matrix, a second target matrix, and a third target matrix, respectively; The method of claim 15.
17. The operation to perform matrix multiplication is When the third target matrix is the result matrix, loading a first sub-matrix of the first target matrix into the first storage space according to the first target size information corresponding to the target amount of accessed memory; loading a second sub-matrix of the second target matrix into the second storage space according to the second size information corresponding to the target amount of accessed memory; performing a matrix multiplication operation on the first sub-matrix and the second sub-matrix to obtain a third sub-matrix of the third target matrix; writing the third sub-matrix to the third storage space.
17. The method of claim 16.
18. The operation to perform matrix multiplication is When the first target matrix is the result matrix, loading a third sub-matrix of the third target matrix into the third storage space according to the third size information corresponding to the target amount of accessed memory; loading a second sub-matrix of the second target matrix into the second storage space according to the second size information corresponding to the target amount of accessed memory; performing a matrix multiplication operation on the third sub-matrix and the second sub-matrix to obtain a first sub-matrix of the first target matrix; writing the first sub-matrix into the first storage space; 17. The method of claim 16.
19. The operation for performing the matrix multiplication is When the second target matrix is the result matrix, loading a first sub-matrix of the first target matrix into the first storage space according to the first target size information corresponding to the target amount of accessed memory; loading a third sub-matrix of the third target matrix into the third storage space according to the third size information corresponding to the target amount of accessed memory; performing a matrix multiplication operation on the first sub-matrix and the third sub-matrix to obtain a second sub-matrix of the second target matrix; writing the second sub-matrix to the second storage space; 17. The method of claim 16.
20. At least one processor; a memory in communication with the at least one processor; The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor such that the at least one processor performs the method of any one of claims 11 to 19. electronic equipment.
21. A non-transitory computer-readable storage medium having computer instructions stored thereon, comprising: The computer instructions cause the computer to carry out a method according to any one of claims 11 to 19. A non-transitory computer-readable storage medium.
22. A computer program which, when executed by a processor, implements the method according to any one of claims 11 to 19.
Citation Information
Patent Citations
Method and device for processing two-dimensional data matrix by artificial intelligence processor
CN111125628A
Method and apparatus for neural network optimized matrix-matrix multiplication (NNMM)
CN113282879A
Arithmetic processing device and arithmetic processing method
JP2021060748A
Information processing method and terminal device
US20210182077A1
