Chip computing method, device, medium and product based on register offset control

Abstract

Embodiments of the present application disclose a chip computing method and device based on register offset control, a medium and a product. The method comprises: performing matrix cutting on a left matrix and a right matrix of a matrix multiplication operation to obtain a left matrix block and a right matrix block; carrying the left matrix to a level one cache, and determining a target left matrix block and a target right matrix block in the matrix multiplication operation; carrying the target right matrix block to the level one cache, reading the target left matrix block and the target right matrix block from the level one cache to perform matrix multiplication to obtain a block operation result; determining a register offset address according to block information of the matrix block, and storing the block operation result to a register according to the register offset address; and when the matrix multiplication operation is determined to be completed, carrying the block operation result in the register to a level three cache to obtain a matrix multiplication result, so that an intermediate operation result is resident in the register and does not occupy the level one cache, the level one cache can cache the left matrix, and frequent data carrying overhead is avoided.

Description

Technical Field

[0001] This invention relates to the field of chip technology, and in particular to a chip computing method, device, medium, and product based on register offset control. Background Technology

[0002] With the development of artificial intelligence technology, the performance requirements for chip-accelerated computing are becoming increasingly demanding. Chip computing devices typically employ multi-level storage hierarchies. Matrix multiplication operations are frequently required in deep learning inference and training.

[0003] In existing matrix multiplication techniques, each block needs to be multiplied by the complete left matrix. Therefore, in each block operation, the left matrix needs to be loaded into the L1 cache level by level. When the matrix dimension is large, the left matrix is ​​repeatedly loaded multiple times, generating significant redundant data transfer overhead within the chip. Furthermore, in existing techniques, intermediate operation results from block operations need to be frequently written back to the L1 cache for accumulation, resulting in redundant L1 cache read / write operations.

[0004] Therefore, there is an urgent need to improve the existing chip computing methods to reduce data transfer overhead and the occupation of intermediate results in the L1 cache, so as to improve chip computing efficiency. Summary of the Invention

[0005] This invention provides a chip computing method, device, medium, and product based on register offset control, which reduces data transfer overhead, reduces data occupancy in the L1 cache, and improves chip computing efficiency.

[0006] According to one aspect of the present invention, a chip computing method based on register offset control is provided, the method comprising: The left and right matrices in the chip to be multiplied are split into multiple left matrix blocks and multiple right matrix blocks. The left matrix is ​​moved to the first-level cache, and the target left matrix block and target right matrix block to be operated on are determined in the matrix multiplication operation; The target right matrix block is moved to the first-level cache, and the target left matrix block and the target right matrix block are read from the first-level cache and matrix multiplication is performed to obtain the block operation result; Based on the block information of the target left matrix block and the target right matrix block, determine the register offset address, and store the block operation result in the register according to the register offset address; When the matrix multiplication operation is completed based on the block information of the target left matrix block and the target right matrix block, the block operation result in the register is moved to the L3 cache to obtain the matrix multiplication result.

[0007] According to another aspect of the present invention, a chip computing device based on register offset control is provided, the device comprising: The matrix segmentation module is used to segment the left and right matrices in the chip that are to be multiplied, respectively, to obtain multiple left matrix blocks and multiple right matrix blocks. The matrix block determination module is used to move the left matrix to the first-level cache and determine the target left matrix block and target right matrix block to be operated on in the matrix multiplication operation; The block operation result calculation module is used to move the target right matrix block to the first-level cache, and read the target left matrix block and the target right matrix block from the first-level cache to perform matrix multiplication to obtain the block operation result; The block operation result storage module is used to determine the register offset address based on the block information of the target left matrix block and the target right matrix block, and store the block operation result in the register according to the register offset address; The matrix multiplication result determination module is used to move the block operation result in the register to the three-level cache to obtain the matrix multiplication result when the matrix multiplication operation is determined to be completed based on the block information of the target left matrix block and the target right matrix block.

[0008] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and a memory communicatively connected to said at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the chip computing method based on register offset control as described in any embodiment of the present invention.

[0009] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the chip computing method based on register offset control as described in any embodiment of the present invention.

[0010] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the chip calculation method based on register offset control as described in any embodiment of the present invention.

[0011] The technical solution of this invention involves matrix partitioning the left and right matrices to be multiplied in the chip to obtain multiple left matrix blocks and multiple right matrix blocks; moving the left matrix to a level 1 cache, and determining the target left and target right matrix blocks to be multiplied during the matrix multiplication operation; moving the target right matrix blocks to a level 1 cache, and reading the target left and target right matrix blocks from the level 1 cache to perform matrix multiplication operations to obtain the block operation results; and determining the register offset address based on the block information of the target left and target right matrix blocks. The block operation result is stored in the register according to the register offset address. When the matrix multiplication operation is completed based on the block information of the target left matrix block and the target right matrix block, the block operation result in the register is moved to the L3 cache to obtain the matrix multiplication result. This solves the problem of redundant data movement in chip computing. By adopting a register offset control mechanism in chip computing, the intermediate operation result resides in the register and does not occupy the L1 cache. Thus, there is storage space in the L1 cache to cache the left matrix, avoiding the overhead caused by frequent movement of the left matrix in chip computing and improving chip computing efficiency.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of a chip calculation method based on register offset control according to Embodiment 1 of the present invention; Figure 2 This is a flowchart of a chip calculation method based on register offset control according to Embodiment 2 of the present invention; Figure 3 This is a flowchart of another chip calculation method based on register offset control according to Embodiment 2 of the present invention; Figure 4 This is a schematic diagram of the structure of a chip computing device based on register offset control according to Embodiment 3 of the present invention; Figure 5This is a schematic diagram of the structure of an electronic device that implements the chip computing method based on register offset control according to the embodiments of the present invention. Detailed Implementation

[0015] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0016] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0017] Example 1 Figure 1 This is a flowchart of a chip computing method based on register offset control according to Embodiment 1 of the present invention. This embodiment is applicable to situations where matrix block operations are performed in chips through multi-level caches and registers. For example, in deep learning network inference, when performing QKV projection, attention calculation, or linear layer calculations such as feedforward networks, the chip computing method based on register offset control provided by any embodiment of the present invention can be used to accelerate chip computing. This method can be executed by a chip computing device based on register offset control, which can be implemented in hardware and / or software. This chip computing device based on register offset control can be configured in electronic devices such as chips, computers, mobile phones, processors, or servers. Figure 1 As shown, the method includes: Step 110: Divide the left and right matrices in the chip into multiple left matrix blocks and multiple right matrix blocks.

[0018] Matrix multiplication can be performed in linear layers of deep learning networks, such as QKV projection, attention calculation, or feedforward networks. Matrix multiplication can be represented as Output[M,N] = LHS[M,K] × RHS[K,N]. Here, Output[M,N] is the output matrix multiplication result, an M×N matrix; LHS[M,K] is the left matrix, an M×K matrix; and RHS[K,N] is the right matrix, a K×N matrix.

[0019] There are several ways to partition a left and right matrix. For example, the left matrix can be partitioned along one or more dimensions, and the right matrix can be partitioned along one or more dimensions. For instance, the left matrix can be partitioned by the number of columns, and the right matrix by the number of rows; or, the left matrix can be partitioned by both the number of rows and columns, and the right matrix by both the number of rows and columns. Alternatively, the right matrix can be partitioned only by the number of columns, and so on.

[0020] In an optional embodiment of the present invention, the left and right matrices to be multiplied in the chip are respectively partitioned to obtain multiple left matrix blocks and multiple right matrix blocks, including: partitioning the left matrix according to a first dimension parameter and a second dimension parameter to obtain multiple left matrix blocks; partitioning the right matrix according to a second dimension parameter and a third dimension parameter to obtain multiple right matrix blocks; wherein the first dimension parameter is the number of rows in the left matrix, the second dimension parameter is the number of columns in the left matrix and the number of rows in the right matrix, and the third dimension parameter is the number of columns in the right matrix.

[0021] For example, the left matrix LHS[M,K] is divided into M_loop = along the first dimension parameter M. The sub-blocks are divided into K_loops along the second dimension parameter K. This results in multiple left matrix blocks. For the right matrix RHS[K,N], it is divided into K_loops along the second dimension parameter K. The number of sub-blocks is divided into N_loops along the third dimension parameter N. This process generates multiple right matrix blocks. Here, M_loop represents the number of sub-blocks along the first dimension parameter M, i.e., the number of blocks in the first dimension; sip_m represents the size of each sub-block along the first dimension parameter M, i.e., the length of the first dimension block. K_loop represents the number of sub-blocks along the second dimension parameter K, i.e., the number of blocks in the second dimension; sip_k represents the size of each sub-block along the second dimension parameter K, i.e., the length of the second dimension block. N_loop represents the number of sub-blocks along the third dimension parameter N, i.e., the number of blocks in the third dimension; sip_n represents the size of each sub-block along the third dimension parameter N, i.e., the length of the third dimension block. This indicates rounding up to the nearest integer.

[0022] By dividing the left and right matrices in a matrix multiplication operation into blocks, the size of a single operation in the chip and the size of data movement in multi-level caches can be reduced, making it suitable for matrix multiplication operations where the matrix size is larger than a preset size.

[0023] Step 120: Move the left matrix to the first-level cache, and determine the target left matrix block and target right matrix block to be operated on in the matrix multiplication operation.

[0024] In this embodiment of the invention, the subsequent register offset mechanism can avoid occupying the storage space of the Level 1 cache (L1), thus allowing the left matrix to reside permanently in the L1 cache. In matrix multiplication operations, when performing calculations along the third dimension parameter, each matrix block along the third dimension needs to be multiplied by the complete left matrix. By residing the left matrix in the L1 cache, this embodiment avoids the need to reload the left matrix from the chip's global storage, such as the Level 3 cache (L3), into the L1 cache every time a new iteration loop calculation with a new third dimension parameter is entered. This avoids the excessive redundant data transfer overhead generated by repeatedly loading the left matrix from the L3 cache to the L1 cache when the third dimension parameter is large.

[0025] In this embodiment of the invention, moving the left matrix to the L1 cache can be a one-time, complete move of the left matrix. For example, the left matrix can be moved from the L3 cache to the L1 cache in one go, raising the entire left matrix to the on-chip shared layer, thus creating the premise for all sub-blocks in the third dimension to reuse the left matrix.

[0026] By moving the left matrix to the L1 cache, the layout of the left matrix can be transformed to enable linear address access when reading data from the left matrix in blocks during chip computing, thus avoiding cross-step data retrieval.

[0027] Optionally, moving the left matrix to the first-level cache includes: moving the left matrix from the third-level cache to the second-level cache, generating a left matrix middle layout with the order of first-dimensional block number, first-dimensional block length, second-dimensional block number, and second-dimensional block length; during the process of moving the left matrix middle layout from the second-level cache to the first-level cache, rearranging it according to the dimension information to form a target left matrix with the order of first-dimensional block number, second-dimensional block number, first-dimensional block length, and second-dimensional block length, so that each left matrix block is stored contiguously in the first-level cache; wherein, the product of the first-dimensional block number and the first-dimensional block length is the first-dimensional parameter, and the product of the second-dimensional block number and the second-dimensional block length is the second-dimensional parameter.

[0028] The left matrix can be moved from the L3 cache to the L2 cache, and then from the L2 cache to the L1 cache. For example, during the process of moving the left matrix from the L3 cache to the L2 cache, an intermediate layout [M_loop, sip_m, K_loop, sip_k] can be generated from the left matrix [M,K]. Essentially, the two are equivalent. M_loop represents the number of blocks in the first dimension, sip_m represents the length of each block in the first dimension, K_loop represents the number of blocks in the second dimension, and sip_k represents the length of each block in the second dimension.

[0029] When moving the middle layout of the left matrix from L2 cache to L1 cache, a layout transformation can be performed on the middle layout. For example, the middle layout [M_loop, sip_m, K_loop, sip_k] can be rearranged according to the dimensions {0,2,1,3} to become [M_loop,K_loop, sip_m, sip_k]. This layout transformation allows each left matrix sub-block [sip_m, sip_k] to be stored contiguously in L1 cache, enabling linear address access for two-dimensional reads and avoiding cross-step data fetching. Continuous data fetching within the chip improves bandwidth utilization during data acquisition, significantly enhancing the chip's data processing capabilities and accelerating chip computation.

[0030] In matrix multiplication, the determination of the target left and right matrix blocks depends on the matrix partitioning method and the loop control method for the matrix partitioning operation. Under different matrix partitioning methods and different loop control methods, the target left and right matrix blocks determined in each calculation will be different. For example, when partitioning the left matrix according to the first and second dimension parameters, and partitioning the right matrix according to the second and third dimension parameters, the order of loop calculations for the first, second, and third dimension parameters can be set. Based on the specific loop method, the target left and right matrix blocks are determined in each calculation.

[0031] Optionally, determining the target left matrix block and target right matrix block to be operated on in the matrix multiplication operation includes: determining the target left matrix block and target right matrix block to be operated on in the matrix multiplication operation according to the nested order of the outer loop to the inner loop as the third dimension parameter, the second dimension parameter and the first dimension parameter.

[0032] In other words, during matrix calculations, a loop for the second dimension parameter K is performed within the loop for the third dimension parameter N; and a loop for the first dimension parameter M is performed within the loop for the second dimension parameter K. Specifically, when the loop variable m_idx of M reaches its upper limit, the loop variable k_idx of K is updated; when k_idx reaches its upper limit, the loop variable n_idx of N is updated; and the calculation ends when n_idx reaches its upper limit. During the loop, the target left matrix block is determined based on m_idx and k_idx; and the target right matrix block is determined based on k_idx and n_idx.

[0033] In practical applications, the loop mode can be set according to specific computational needs. The loop mode is related to the method of subsequently determining the register offset address and the way the block operation results are stored in the register. Specifically, it can be set according to the relationship between the block operation results and the matrix multiplication result when performing block operations.

[0034] Step 130: Move the target right matrix block to the first-level cache, and read the target left matrix block and the target right matrix block from the first-level cache to perform matrix multiplication to obtain the block operation result.

[0035] Moving the target right matrix block to the L1 cache can be done by moving the target right matrix block from the L3 cache to the L1 cache step by step. During the target right matrix block movement, a double-buffer prefetching method can be used. That is, the target right matrix block is moved while the previous right matrix block operation is being performed; and the next right matrix block is moved while the target right matrix block operation is being performed. This achieves time hiding of right matrix data movement, further improving chip computing efficiency.

[0036] When the data corresponding to the target left matrix block [sip_m, sip_k] and the target right matrix block [sip_k, sip_n] are obtained, matrix multiplication can be performed to obtain the block operation result of the matrix block operation.

[0037] Step 140: Determine the register offset address based on the block information of the target left matrix block and the target right matrix block, and store the block operation result in the register according to the register offset address.

[0038] The block information for the target left and right matrix blocks can be information associated with the matrix blocks in the loop operation, such as loop variables and dimension block lengths. For example, the block information for the target left and right matrix blocks can include loop variables of one or more of the first dimension parameter, the second dimension parameter, and the third dimension parameter, i.e., one or more of m_idx, k_idx, and n_idx. The block information for the target left and right matrix blocks can also include one or more of the first dimension block length sip_m, the second dimension block length sip_k, and the third dimension block length sip_n.

[0039] Specifically, the method for determining the register offset address can be related to the loop control method and matrix partitioning method of the matrix block operation. In practical applications, the block operation results can be stored in the storage space corresponding to the register offset address in the register, based on the logic of the matrix block calculation and the block information of the target left and right matrix blocks. The width of a single register can also be considered when storing the block operation results. When storing the block operation results in the register, data can be stored directly, or the corresponding data can be accumulated in the register according to the specific logic of the matrix block calculation.

[0040] For example, when partitioning the left matrix according to the first and second dimension parameters, and the right matrix according to the second and third dimension parameters, and performing calculations in a nested order from the outer loop to the inner loop (third dimension parameter, second dimension parameter, and first dimension parameter), the block operation results corresponding to the sub-blocks on different first dimension parameters can be stored in different locations in the registers, ensuring that different sub-blocks on the first dimension parameter and different functional register areas do not overlap. Therefore, based on the condition that different sub-blocks on the first dimension parameter and different functional register areas do not overlap, combined with the block information of the target left and target right matrix blocks, the register offset address is determined.

[0041] The actual storage address of the block operation result in the register can be determined by the sum of the register base address and the register offset address. By storing the block operation result in the register, the storage space of the L1 cache can be avoided, allowing the left matrix to reside completely in the L1 cache, reducing redundant data transfer overhead and improving chip computing efficiency.

[0042] Step 150: When the matrix multiplication operation is completed based on the block information of the target left matrix block and the target right matrix block, the block operation result in the register is moved to the level 3 cache to obtain the matrix multiplication operation result.

[0043] During the process of moving the results of block operations in registers, the result accumulation and movement can be performed according to the matrix multiplication operation logic. When moving the results of block operations in registers to the three-level cache, a step-by-step movement method can be adopted. For example, the results of block operations in registers are moved from the register to the L1 cache, then to the L2 cache, and finally to the L3 cache to obtain the matrix multiplication result.

[0044] The technical solution of this embodiment involves dividing the left and right matrices to be multiplied in the chip into multiple left and right matrix blocks. The left matrix is ​​moved to the L1 cache, and the target left and right matrix blocks are determined during the matrix multiplication operation. The target right matrix blocks are moved to the L1 cache, and the target left and right matrix blocks are read from the L1 cache for matrix multiplication to obtain the block operation results. Based on the block information of the target left and right matrix blocks, the register offset address is determined, and... The block operation results are stored in the register according to the register offset address. When the matrix multiplication operation is completed based on the block information of the target left matrix and the target right matrix, the block operation results in the register are moved to the L3 cache to obtain the matrix multiplication result. This solves the problem of redundant data movement in chip computing. By adopting the register offset control mechanism in chip computing, the intermediate operation results are kept in the register and do not occupy the L1 cache. Thus, there is storage space in the L1 cache to cache the left matrix, avoiding the overhead caused by frequent movement of the left matrix in chip computing and improving chip computing efficiency.

[0045] Example 2 Figure 2 This is a flowchart of a chip calculation method based on register offset control according to Embodiment 2 of the present invention. This embodiment is a further refinement of the above technical solution, and the technical solution in this embodiment can be combined with various optional solutions in one or more of the above embodiments. Figure 2 As shown, the method includes: Step 210: Divide the left matrix according to the first dimension parameter and the second dimension parameter to obtain multiple left matrix blocks.

[0046] Step 220: Divide the right matrix according to the second and third dimension parameters to obtain multiple right matrix blocks.

[0047] The first dimension parameter is the number of rows in the left matrix, the second dimension parameter is the number of columns in the left matrix and the number of rows in the right matrix, and the third dimension parameter is the number of columns in the right matrix.

[0048] Step 230: Move the left matrix from the level 3 cache to the level 2 cache to generate a left matrix middle layout with the order of first dimension block number, first dimension block length, second dimension block number, and second dimension block length.

[0049] Step 240: During the process of moving the middle layout of the left matrix from the second-level cache to the first-level cache, it is rearranged according to the dimension information to form a target left matrix with the arrangement order of the number of blocks in the first dimension, the number of blocks in the second dimension, the length of the first dimension block, and the length of the second dimension block, so that each left matrix block is stored continuously in the first-level cache.

[0050] The product of the number of blocks in the first dimension and the length of the blocks in the first dimension is the first dimension parameter, and the product of the number of blocks in the second dimension and the length of the blocks in the second dimension is the second dimension parameter.

[0051] Step 250: In the matrix multiplication operation, determine the target left matrix block and target right matrix block to be operated on by following the nested order of the outer loop to the inner loop, which is the third dimension parameter, the second dimension parameter, and the first dimension parameter.

[0052] Step 260: Move the target right matrix block to the first-level cache, and read the target left matrix block and the target right matrix block from the first-level cache to perform matrix multiplication to obtain the block operation result.

[0053] Step 270: Determine the register offset address based on the current loop variable in the first dimension, the block length in the third dimension, and the width of a single register.

[0054] For example, the register offset address is determined according to vab_offset = m_idx × sip_n / register_width, where vab_offset is the register offset address, m_idx is the current loop variable of the first dimension, which increments by the length of the first dimension block, and the product of the number of first dimension blocks and the length of the first dimension block is the first dimension parameter; sip_n is the length of the third dimension block, and the product of the number of third dimension blocks and the length of the third dimension block is the third dimension parameter; register_width is the width of a single register.

[0055] By determining the register offset address based on the current loop variable in the first dimension and the block length in the third dimension, sub-blocks on different first-dimensional parameters M and different functional register areas can be prevented from overlapping. That is, when the block operation result is obtained under the same m_idx, the block operation result can be stored at the same address in the register. Specific storage methods can include direct storage, cumulative storage, or clearing before storage. When the block operation result is obtained under different m_idx values, the block operation result can be stored at different addresses in the register. Determining the register offset address based on the current loop variable in the first dimension and the block length in the third dimension ensures that sub-blocks on different first-dimensional parameters M and different functional register areas do not overlap, satisfying the logic of matrix block operations and facilitating the subsequent obtaining of matrix multiplication results.

[0056] Step 280: When the current loop variable in the second dimension is the start of the loop, clear the storage area of ​​the register offset address in the register to zero, and then store the block operation result; when the current loop variable in the second dimension is not the start of the loop, accumulate the storage area of ​​the register offset address in the register according to the block operation result.

[0057] For example, when the current loop variable k_idx for the second dimension is 0, it indicates the start of the loop for the second dimension parameter. The current loop variable for the second dimension increments by the length of the second dimension block, and the product of the number of second dimension blocks and the length of the second dimension block is the second dimension parameter. When k_idx is 0, the storage area at the register offset address in the register is cleared, and then the block operation result is stored. This avoids errors in the matrix multiplication result caused by accumulating and storing the block operation results with the same m_idx in different iterations of the second dimension parameter loop.

[0058] If the current loop variable for the second dimension is not at the start of the loop (e.g., k_idx is greater than 0), the storage area at the register offset address in the register is accumulated based on the block operation results. That is, when k_idx is greater than 0, accumulation storage is performed in the same register area, ensuring that the block operation results of the same m_idx are accumulated and stored in the current iteration of the second dimension parameter's loop, guaranteeing the correctness of the matrix multiplication results.

[0059] Step 290: When the current loop variable in the second dimension is the end of the loop, move the accumulated block operation results of the storage area of ​​the register offset address in the register to the L3 cache; when the current loop variable in the third dimension is the end of the loop, obtain the matrix multiplication operation result.

[0060] The current loop variable for the second dimension indicates the end of the loop. If k_idx is greater than or equal to K, the accumulated block operation results from the register offset address storage area in the register are moved to the L3 cache. The result at this time is the value of the m_idx row and n_idx column in the matrix multiplication result. The current loop variable for the third dimension indicates the end of the loop. If n_idx is greater than or equal to N, the operation is completed, and the matrix multiplication result is obtained. The current loop variable for the third dimension increments with the length of the third-dimensional block, and the product of the number of third-dimensional blocks and the length of the third-dimensional block is the third-dimensional parameter. The block operation results in the registers are moved to the L3 cache in a step-by-step manner: the result is moved from the register cache to the L1 cache, then from the L1 cache to the L2 cache, and finally from the L2 cache to the L3 cache.

[0061] During the execution of steps 210 to 290 above, when the splitting method of the left and right matrices changes, or the nested order of matrix block operations changes, the method of determining the register offset address, the method of storing the block operation result in the register, and the method of moving the block operation result can be adaptively adjusted to ensure the correctness of the matrix multiplication result.

[0062] Figure 3 This is a flowchart of another chip calculation method based on register offset control according to Embodiment 2 of the present invention. Figure 3 As shown, in chip computing, LHS[M,K] and RHS[K,N] are input, and matrix multiplication C[M,N] = LHS[M,K] × RHS[K,N] is performed. The left and right matrices are then partitioned, and M_loop = ... K_loop= and N_loop = The size allocation is performed on the L1, L2, L3 caches and register working areas in the chip. The entire LHS is moved from L3 to L2, only once. The principle is to first promote the entire left matrix to the on-chip shared layer, creating the premise for the reuse of all subsequent N sub-blocks. During the L2 to L1 transfer, a matrix layout transformation is performed, changing the intermediate layout [M_loop, sip_m, K_loop, sip_k] to [M_loop, K_loop, sip_m, sip_k] by rearranging the dimensions {0,2,1,3}. The layout transformation mechanism ensures that each [sip_m, sip_k] sub-block is stored contiguously in L1, and two-dimensional reading is linear address access, avoiding cross-step data fetching.

[0063] Entering the outer third-dimensional parameter N loop, the loop variable n_idx increments from 0 by sip_n to N, processing the current N sub-blocks. At the start of each iteration, RHS double-buffered prefetching (Ping-Pong) is triggered, batching the data corresponding to the current N window of RHS into L1. When n_idx + sip_n < N, matrix block multiplication is performed in the loop; when n_idx + sip_n ≥ N, the N loop ends.

[0064] With a fixed n_idx, the loop variable k_idx, representing the second dimension parameter, increments from 0 to K by sip_k, sequentially processing K-dimensional blocks. The mechanism of this step is to break down the long K-dimensional structure into multiple accumulative parts and stages. When k_idx + sip_k < K, matrix block multiplication is performed cyclically; when k_idx + sip_k ≥ K, the K-dimensional accumulation of the current N blocks is complete.

[0065] With fixed n_idx and k_idx, the loop variable m_idx, representing the first dimension parameter, increments from 0 by sip_m to M. LHS sub-blocks [sip_m, sip_k] are read continuously from L1, and RHS sub-blocks [sip_k, sip_n] are read from L1 for matrix multiplication. When m_idx + sip_m < M, matrix block multiplication is performed cyclically. Before, during, or after matrix block multiplication, the register offset address can be determined using vab_offset = m_idx × sip_n / register_width. When m_idx + sip_m < M, if k_idx = 0, the storage space at the register offset address in the register is cleared and the block operation result is written; if k_idx > 0, the storage space at the register offset address in the register is incremented; if k_idx + sip_k < K, the increment continues in the register without writing back to the L1 cache; if k_idx + sip_k ≥ K, the last block of the K dimension is reached, triggering the final result to be written from the register to the L1 cache. When m_idx + sip_m ≥ M, the M blocks under fixed n_idx and k_idx are processed, and k_idx is updated. When k_idx + sip_k ≥ K, the K blocks under fixed n_idx are processed, the block operation results in the L1 cache are moved to the L3 cache, and n_idx is updated. When n_idx + sip_n ≥ N, the matrix multiplication result is obtained.

[0066] In practical applications, taking M=512, K=1024, N=2048 as an example, and assuming sip_m=256, sip_k=256, sip_n=128, then M_loop=2, K_loop=4, and N_loop=16. Following this... Figure 3 The process shown involves the LHS being moved from L3 to L2 at the beginning, and then residing in L1 after a layout transformation. Within each n_idx (16 rounds in total), 4 rounds of k_idx are executed; within each k_idx, 2 rounds of m_idx are executed. For each m_idx, register offset control fixes the partial sum of the m blocks to an independent register area. The first 3 k blocks are only accumulated in the registers and not written to L1. Only after the 4th k ​​block is completed is the final result of the corresponding m block written to L1 output and then written back to L3. The matrix multiplication result is obtained after the n_idx rounds are completed. Therefore, within the entire N=2048 range, the LHS no longer repeatedly loads from L3 with N; at the same time, the intermediate results of the K loops do not travel back and forth to L1, significantly reducing the L3 transport and L1 read / write pressure.

[0067] The technical solution of this invention achieves loading from global storage to shared storage by moving the LHS matrix from L3 to L2 in one go, forming an intermediate layout of [M, K], namely [[M_loop, sip_m, K_loop, sip_k]]. When moving from L2 to L1, a layout transformation operation is performed to rearrange the data: for example, using the chip's transpose operation in conjunction with the dimensional arrangement {0,2,1,3} to complete the equivalent transformation, ensuring that the final L1 layout is uniformly [M_loop, K_loop, sip_m, sip_k]. This sub-block [sip_m, sip_k] is completely contiguous in L1, and the chip's computing unit can directly read it through a simple linear address without needing to perform cross-step access or data acquisition operations, maximizing the read bandwidth utilization from L1 to the register. Register offset control ensures that the accumulation results of different M sub-blocks are stored in different numbered areas of the register, without occupying L1 cache storage. By transforming the matrix layout, the LHS can be loaded only once from L3 and reused throughout all N-dimensional iterations. LHS data is stored in L1 in a computationally friendly, contiguous block format, avoiding the need to concatenate data from different K segments in row M for each access to the sip_m × sip_k sub-block, thus preventing non-contiguous access. After the LHS resides permanently in L1, the remaining space in L1 is used to store the double-buffered data and output results of the RHS. If the intermediate parts of the K loop are also written back to L1, it will occupy a large amount of L1 space, potentially preventing the LHS from fully residing in L1. This would force the LHS to be reloaded in N iterations, negating the advantages of the layout transformation. This embodiment of the invention uses register offset control to ensure that the intermediate results of each M sub-block reside in an independent area of ​​the register file. The K loop does not write back to L1 throughout (only the final result is written out), thereby freeing up L1 space for the LHS to reside permanently. Ultimately, this achieves LHS residing permanently in L1, intermediate results residing in registers, sufficient L1 space, efficient access, minimized L3 bandwidth, and chip computational efficiency approaching peak computing power.

[0068] Example 3 Figure 4 This is a schematic diagram of a chip computing device based on register offset control according to Embodiment 3 of the present invention. Figure 4 As shown, the device includes: a matrix segmentation module 410, a matrix block determination module 420, a block operation result calculation module 430, a block operation result storage module 440, and a matrix multiplication operation result determination module 450. Wherein: The matrix segmentation module 410 is used to segment the left and right matrices in the chip to be multiplied, respectively, to obtain multiple left matrix blocks and multiple right matrix blocks. The matrix block determination module 420 is used to move the left matrix to the first-level cache and determine the target left matrix block and target right matrix block to be operated on in the matrix multiplication operation; The block operation result calculation module 430 is used to move the target right matrix block to the first-level cache, and read the target left matrix block and the target right matrix block from the first-level cache to perform matrix multiplication to obtain the block operation result; The block operation result storage module 440 is used to determine the register offset address based on the block information of the target left matrix block and the target right matrix block, and to store the block operation result in the register according to the register offset address; The matrix multiplication result determination module 450 is used to move the block operation result in the register to the three-level cache to obtain the matrix multiplication result when the matrix multiplication operation is completed based on the block information of the target left matrix block and the target right matrix block.

[0069] Optionally, the matrix partitioning module 410 includes: The left matrix partitioning unit is used to partition the left matrix according to the first dimension parameter and the second dimension parameter to obtain multiple left matrix blocks; The right matrix partitioning unit is used to partition the right matrix according to the second and third dimension parameters to obtain multiple right matrix blocks; The first dimension parameter is the number of rows in the left matrix, the second dimension parameter is the number of columns in the left matrix and the number of rows in the right matrix, and the third dimension parameter is the number of columns in the right matrix.

[0070] Optionally, the matrix block determination module 420 includes: The left matrix middle layout generation unit is used to move the left matrix from the third-level cache to the second-level cache and generate a left matrix middle layout with the arrangement order of the first dimension block number, the first dimension block length, the second dimension block number, and the second dimension block length. The layout transformation unit is used to rearrange the middle layout of the left matrix according to the dimension information during the process of moving the middle layout of the left matrix from the second-level cache to the first-level cache, forming a target left matrix with the arrangement order of the number of blocks in the first dimension, the number of blocks in the second dimension, the length of the first dimension block, and the length of the second dimension block, so that each left matrix block is stored continuously in the first-level cache. The product of the number of blocks in the first dimension and the length of the blocks in the first dimension is the first dimension parameter, and the product of the number of blocks in the second dimension and the length of the blocks in the second dimension is the second dimension parameter.

[0071] Optionally, the matrix block determination module 420 includes: The matrix block determination unit is used to determine the target left matrix block and target right matrix block to be operated on in matrix multiplication operations, according to the nested order of the outer loop to the inner loop as the third dimension parameter, the second dimension parameter and the first dimension parameter.

[0072] Optionally, the block operation result storage module 440 includes: The register offset address determination unit is used to determine the register offset address based on the current loop variable in the first dimension, the block length in the third dimension, and the width of a single register. In this context, the current loop variable of the first dimension increments with the length of the first dimension block, the product of the number of first dimension blocks and the length of the first dimension blocks is the first dimension parameter, and the product of the number of third dimension blocks and the length of the third dimension blocks is the third dimension parameter.

[0073] Optionally, the block operation result storage module 440 includes: The first result storage unit is used to clear the storage area of ​​the register offset address in the register when the current loop variable in the second dimension is the start of the loop, and then store the block operation result. The second result storage unit is used to accumulate the storage area of ​​the register offset address in the register according to the block operation result when the current loop variable in the second dimension is not the start of the loop. In this context, the current loop variable of the second dimension increments with the length of the second dimension block, and the product of the number of second dimension blocks and the length of the second dimension blocks is the second dimension parameter.

[0074] Optionally, the matrix multiplication result determination module 450 includes: The block operation result transport unit is used to transport the block operation result accumulated in the storage area of ​​the register offset address in the register to the level 3 cache when the current loop variable in the second dimension is the end of the loop. The matrix multiplication result determination unit is used to obtain the matrix multiplication result when the current loop variable in the third dimension is at the end of the loop. The current loop variable in the third dimension increments with the length of the third dimension block, and the product of the number of third dimension blocks and the length of the third dimension block is the third dimension parameter.

[0075] The chip computing device based on register offset control provided in the embodiments of the present invention can execute the chip computing method based on register offset control provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0076] Example 4 Figure 5 This is a schematic diagram of the structure of an electronic device implementing the chip computing method based on register offset control according to embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0077] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) or random access memory (RAM), communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from the storage unit 18. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. Input / output (I / O) interfaces are also connected to the bus 14.

[0078] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0079] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as chip computing methods based on register offset control.

[0080] In some embodiments, the register offset control-based chip computing method can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the register offset control-based chip computing method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the register offset control-based chip computing method by any other suitable means (e.g., by means of firmware).

[0081] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0082] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0083] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0084] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0085] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0086] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0087] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0088] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A chip computing method based on register offset control, characterized in that, include: The left and right matrices in the chip to be multiplied are split into multiple left matrix blocks and multiple right matrix blocks. The left matrix is ​​moved to the first-level cache, and the target left matrix block and target right matrix block to be operated on are determined in the matrix multiplication operation; The target right matrix block is moved to the first-level cache, and the target left matrix block and the target right matrix block are read from the first-level cache and matrix multiplication is performed to obtain the block operation result; Based on the block information of the target left matrix block and the target right matrix block, determine the register offset address, and store the block operation result in the register according to the register offset address; When the matrix multiplication operation is completed based on the block information of the target left matrix block and the target right matrix block, the block operation result in the register is moved to the L3 cache to obtain the matrix multiplication result.

2. The method according to claim 1, characterized in that, The left and right matrices in the chip to be multiplied are partitioned to obtain multiple left matrix blocks and multiple right matrix blocks, including: The left matrix is ​​partitioned according to the first dimension parameter and the second dimension parameter to obtain multiple left matrix blocks; The right matrix is ​​partitioned according to the second and third dimension parameters to obtain multiple right matrix blocks; Wherein, the first dimension parameter is the number of rows in the left matrix, the second dimension parameter is the number of columns in the left matrix and the number of rows in the right matrix, and the third dimension parameter is the number of columns in the right matrix.

3. The method according to claim 2, characterized in that, Moving the left matrix to the first-level cache includes: The left matrix is ​​moved from the third-level cache to the second-level cache to generate a middle layout of the left matrix with the order of first-dimensional block number, first-dimensional block length, second-dimensional block number, and second-dimensional block length. During the process of moving the middle layout of the left matrix from the second-level cache to the first-level cache, it is rearranged according to the dimension information to form a target left matrix with the arrangement order of the number of blocks in the first dimension, the number of blocks in the second dimension, the length of the first dimension block, and the length of the second dimension block, so that each left matrix block is stored continuously in the first-level cache. The product of the number of blocks in the first dimension and the length of the blocks in the first dimension is the first dimension parameter, and the product of the number of blocks in the second dimension and the length of the blocks in the second dimension is the second dimension parameter.

4. The method according to claim 2, characterized in that, In matrix multiplication, determining the target left and right matrix blocks to be multiplied includes: In matrix multiplication, the target left and right matrix blocks are determined by following the nested order of the outer loop to the inner loop, which is the third dimension parameter, the second dimension parameter, and the first dimension parameter.

5. The method according to claim 4, characterized in that, Based on the block information of the target left matrix block and the target right matrix block, the register offset address is determined, including: Determine the register offset address based on the current loop variable in the first dimension, the block length in the third dimension, and the width of a single register; In this context, the current loop variable of the first dimension increments with the length of the first dimension block, the product of the number of first dimension blocks and the length of the first dimension blocks is the first dimension parameter, and the product of the number of third dimension blocks and the length of the third dimension blocks is the third dimension parameter.

6. The method according to claim 5, characterized in that, Storing the block operation result into the register according to the register offset address includes: When the current loop variable in the second dimension is the start of the loop, the storage area of ​​the register offset address in the register is cleared to zero, and then the block operation result is stored. When the current loop variable in the second dimension is not the start of the loop, the storage area of ​​the register offset address in the register is accumulated according to the block operation result; In this context, the current loop variable of the second dimension increments with the length of the second dimension block, and the product of the number of second dimension blocks and the length of the second dimension blocks is the second dimension parameter.

7. The method according to claim 6, characterized in that, When the matrix multiplication operation is determined to be complete based on the block information of the target left matrix block and the target right matrix block, the block operation result in the register is moved to the L3 cache to obtain the matrix multiplication result, including: When the current loop variable in the second dimension ends, the block operation result accumulated in the storage area of ​​the register offset address in the register is moved to the L3 cache; When the current loop variable in the third dimension is the end of the loop, the result of the matrix multiplication operation is obtained; The current loop variable in the third dimension increments with the length of the third dimension block, and the product of the number of third dimension blocks and the length of the third dimension block is the third dimension parameter.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to said at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the chip computing method based on register offset control as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the chip computing method based on register offset control as described in any one of claims 1-7.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the chip calculation method based on register offset control according to any one of claims 1-7.

Images