A method and device for accelerating training and reasoning of artificial intelligence models

Abstract

The present invention provides an artificial intelligence model accelerated training and inference method and device, belonging to the field of artificial intelligence. The present invention first uses a stacking grouping technology to group and disassemble large-dimensional matrices in the artificial intelligence model, converting them into a stacked block matrix structure; then uses the ACA-SVD matrix decomposition technology to compress and decompose the non-diagonal block matrix, expressing it as the multiplication of two small matrices; then, the diagonal matrix blocks of the obtained stacked block matrix structure are extracted in sequence, and the matrix is expressed as the multiplication of multiple block diagonal matrices, and each block diagonal matrix is stored separately; when performing model training and inference, the matrix multiplication process can be converted into the multiplication of multiple block diagonal matrices, and the matrix inversion process can be converted into the inversion of multiple block diagonal matrices. Each block diagonal matrix is independent of each other, and there is no recursive relationship during the operation process, which can achieve highly parallel operation.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a method and device for accelerating the training and inference of an artificial intelligence model. Background Art

[0002] With the rapid development of technologies such as the internet, big data, and cloud computing, artificial intelligence (AI) has gained unprecedented access to data resources and computing power. Breakthroughs in key AI technologies, such as machine learning, deep learning, reinforcement learning, and transfer learning, have enabled significant advances in speech recognition, image recognition, and natural language processing. This progress has not only manifested itself in significant theoretical breakthroughs but has also been realized in practical applications across multiple industries. AI models have gradually permeated fields such as healthcare, transportation, finance, education, and manufacturing. However, in practical applications, in addition to the effectiveness of the model itself, the cost of its application must also be considered, primarily due to the use of storage and computing resources. AI models require the storage of large matrices and the performance of numerous matrix operations. This is particularly true when processing large-scale data applications, where matrices are large in dimensionality. The high storage requirements and computational complexity of matrix operations significantly increase resource usage during model training and inference, further increasing application costs. Summary of the Invention

[0003] In response to the urgent need to reduce the resource costs occupied during training and reasoning processes when promoting and applying artificial intelligence models, the present invention provides a method for accelerating training and reasoning of artificial intelligence models. It uses multi-layer matrix low-rank compression technology to compress, decompose and transform large-dimensional matrices such as covariance matrices and weight matrices in artificial intelligence models, and finally represents the matrix as the multiplication of multiple block diagonal matrices. Each block diagonal matrix is stored separately to reduce storage resource usage, and the computational complexity of matrix inversion operations in the model training and reasoning process is reduced and highly parallelized, thereby lowering the computing power threshold for model use.

[0004] The technical solution of the present invention is:

[0005] A method for accelerating training and reasoning of an artificial intelligence model comprises the following steps:

[0006] S1. Use the stacking grouping technology to group and decompose the large-dimensional matrix W in the artificial intelligence model into a stacked block matrix structure.

[0007] S2. Use ACA-SVD matrix decomposition technology to decompose the non-diagonal block matrix Perform compression decomposition and express it as two small matrices and Multiply, that is , where i and j represent the group number of the interaction group in the layer where the non-diagonal block matrix is located, and are also the row and column numbers of the matrix block in the corresponding layer.

[0008] S3. Repeat step S2 to complete the compressed decomposition of all non-diagonal block matrices.

[0009] S4. Extract the diagonal matrix blocks of the stacked block matrix obtained after the compression decomposition in sequence, and perform corresponding matrix processing, and finally represent the stacked block matrix as the multiplication of multiple block diagonal matrices.

[0010] S5. Each diagonal block matrix Further decomposition, then according to the Sherman-Morrison-Woodbury formula The inverse matrix of , realizes the dimensionality reduction in the process of matrix inversion.

[0011] S6. Get the matrix The inverse of , which can then be expressed as a block diagonal matrix The inverse of , so the inverse matrix of matrix W can be expressed as the multiplication of multiple block diagonal inverse matrices.

[0012] Furthermore, in step S1, the specific steps of using the stacking grouping technology to group and decompose the large-dimensional matrix W in the artificial intelligence model into a stacked block matrix structure are as follows:

[0013] S1.1. Set the minimum matrix block dimension threshold during group decomposition to min_n.

[0014] S1.2. Split the matrix W into equal-dimensional rows and columns to obtain the first-level grouping matrix. Determine whether the dimension of each matrix block is less than the dimension threshold min_n. If so, stop the operation; otherwise, proceed to the next step.

[0015] S1.3. Perform equal-dimensional row and column splits on the diagonal blocks of the previous grouping matrix to obtain the next grouping matrix. Determine whether the dimension of each matrix block is less than the dimension threshold min_n. If so, stop the operation; otherwise, proceed to the next step.

[0016] S1.4. Repeat step S1.3 until the minimum matrix block dimension is less than the dimension threshold min_n. Stop the operation. For example, the two-layer matrix is obtained. The corresponding matrix W becomes:

[0017] ,

[0018] In the formula, the superscript represents the layer number, and the subscript represents the group number of the interaction group in the corresponding layer.

[0019] Furthermore, in step S2, the non-diagonal block matrix is decomposed using the ACA-SVD matrix decomposition technique. The specific steps for compression decomposition are:

[0020] S2.1. Using ACA algorithm to calculate the matrix During compression, an allowable precision (e) needs to be set for the ACA algorithm, which can make the matrix Decompose into two matrices with smaller dimensions and .

[0021] S2.2. The matrix and The transposed matrix of Use QR decomposition technology to decompose them respectively and get:

[0022] ,

[0023] ,

[0024] So the matrix It can be expressed as:

[0025] ,

[0026] p represents the layer number, and k represents the group number of the corresponding layer.

[0027] Furthermore, in step S5, each diagonal block matrix The specific features of further decomposition are: each diagonal block matrix Further decomposed into:

[0028] ,

[0029] It can be further expressed as: .

[0030] Furthermore, in step S5, according to the Sherman-Morrison-Woodbury formula The inverse matrix of , realizes the dimension reduction in the process of matrix inversion, and its specific features are: The inverse matrix can be expressed as:

[0031] ,

[0032] in

[0033] ,

[0034] matrix The dimension is The rank of a non-diagonal block matrix is much smaller than The dimension of is reduced, which realizes the dimension reduction in the process of matrix inversion.

[0035] Furthermore, in step S6, the inverse matrix of the matrix W can be expressed as the product of multiple block diagonal inverse matrices. The specific characteristics are: the inverse matrix of the matrix W can be expressed as:

[0036] ,

[0037] Where L represents the total number of layers of the stacked matrix. Each block diagonal matrix Each diagonal block matrix in They are all relatively independent, there is no recursive relationship in the matrix operation process, and highly parallel operations can be achieved.

[0038] In addition, the present invention also provides an artificial intelligence model accelerated training and inference device for implementing the above-mentioned method, including: a matrix stacking grouping module, an ACA-SVD matrix decomposition module, a matrix processing and storage module, a model training module, and a model inference module.

[0039] in,

[0040] The matrix stacking grouping module is used to group and decompose the large-dimensional matrix W in the artificial intelligence model according to the process in step S1 and convert it into a stacked block matrix structure.

[0041] ACA-SVD matrix decomposition module is used to implement the non-diagonal block matrix according to the process in step S2 Perform compression decomposition and express it as two small matrices and Multiply, that is , and complete the compressed decomposition of all non-diagonal block matrices in the matrix W according to the process in step S3.

[0042] The matrix processing and storage module is used to extract the diagonal matrix blocks of the stacked block matrix obtained after the compression decomposition in sequence according to the process in step S4, and perform corresponding matrix processing, and finally represent the stacked block matrix as the multiplication of multiple block diagonal matrices, and then store each block diagonal matrix.

[0043] The model training module is used to train the artificial intelligence model. If the matrix inversion operation is used in this process, the low computational complexity of the matrix inversion is implemented according to steps S5 and S6, and this process can be highly parallelized.

[0044] The model inference module is used to complete the reasoning of the artificial intelligence model. If the matrix inversion operation is used in this process, the low computational complexity of the matrix inversion is implemented according to steps S5 and S6, and this process can be highly parallelized.

[0045] The beneficial effects of the present invention are

[0046] The present invention utilizes multi-layer matrix low-rank compression technology to compress and decompose large-dimensional matrices such as the covariance matrix and weight matrix in the artificial intelligence model and convert them into multiple block diagonal matrix multiplications. Then, the matrix inversion process of the covariance matrix and the weight matrix can be converted into the inversion of multiple block diagonal matrices, which not only reduces the resource occupation of matrix storage during model deployment, but also reduces the computational complexity of matrix inversion operations during model training and inference. Moreover, each block diagonal matrix is independent of each other, and there is no recursive relationship during the operation process, which can achieve highly parallel operations and fully reduce the computing power threshold for model use. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a schematic diagram of accelerating training and inference of artificial intelligence models based on multi-layer matrix low-rank compression;

[0048] Figure 2 This is a schematic diagram of a matrix stacking structure (taking a 2-layer matrix as an example);

[0049] Figure 3 This is a schematic diagram of the ACA decomposition;

[0050] Figure 4 It is a schematic diagram showing the representation of a stacked block matrix as the multiplication of multiple block diagonal matrices. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0052] In order to reduce the demand for storage resources and computing resources during the training and reasoning of artificial intelligence models, the present invention uses multi-layer matrix low-rank compression technology to compress and decompose large-dimensional matrices such as the covariance matrix and weight matrix in the artificial intelligence model and convert them into multiple block diagonal matrix multiplications. The matrix inversion process of the covariance matrix and the weight matrix can be converted into the inversion of multiple block diagonal matrices, which not only reduces the resource occupation of matrix storage during model deployment, but also reduces the computational complexity of matrix inversion operations during model training and reasoning. Moreover, each block diagonal matrix is independent of each other, and there is no recursive relationship during the operation process, which can achieve highly parallel operations and fully reduce the computing power threshold for model use.

[0053] The present invention first uses the stacked grouping technology to group and disassemble the large-dimensional matrix in the artificial intelligence model, and converts it into a stacked block matrix structure; then uses the ACA-SVD matrix decomposition technology to compress and decompose the non-diagonal block matrix, and express it as the multiplication of two small matrices; then the diagonal matrix blocks of the obtained stacked block matrix structure are extracted in sequence, and the matrix is expressed as the multiplication of multiple block diagonal matrices, and each block diagonal matrix is stored separately; when performing model training and inference, the matrix multiplication process can be converted into the multiplication of multiple block diagonal matrices, and the matrix inversion process can be converted into the inversion of multiple block diagonal matrices. Each block diagonal matrix is independent of each other, and there is no recursive relationship during the operation process, which can achieve highly parallel operations.

[0054] like Figure 1 The specific steps are as follows:

[0055] Step 1: Use the stacking grouping technology to group and decompose the large-dimensional matrix W in the artificial intelligence model into a stacked block matrix structure. The specific steps are as follows:

[0056] Step 1.1: Set the minimum matrix block dimension threshold during group decomposition to min_n.

[0057] Step 1.2: Split the matrix W into equal-dimensional rows and columns to obtain the first-layer grouping matrix. Determine whether the dimension of each matrix block is less than the dimension threshold min_n. If so, stop the operation, otherwise proceed to the next step.

[0058] Step 1.3: Perform equal-dimensional row and column splits on the diagonal blocks of the previous grouping matrix to obtain the next grouping matrix. Determine whether the dimension of each matrix block is less than the dimension threshold min_n. If so, stop the operation; otherwise, proceed to the next step.

[0059] Step 1.4: Repeat step 1.3 until the minimum matrix block dimension is less than the dimension threshold min_n, then stop the operation. Take the 2-layer matrix as an example, and get Figure 2 For the stacked structure shown in Figure 1, the corresponding matrix W becomes:

[0060] ,

[0061] In the formula, the superscript represents the layer number, and the subscript represents the group number of the interaction group in the corresponding layer.

[0062] Step 2: Use ACA-SVD matrix decomposition technique to decompose the non-diagonal block matrix Perform compression decomposition and express it as two small matrices and Multiply, where i and j represent the group number of the interaction group in the layer where the non-diagonal block matrix is located, and are also the row and column numbers of the matrix block in the corresponding layer. The specific steps are as follows:

[0063] Step 2.1: Use ACA algorithm to adjust the matrix During the compression decomposition process, it is necessary to set an allowable precision (e) for the ACA algorithm, which can make the matrix Decompose into two matrices with smaller dimensions and , its decomposition diagram is as follows Figure 3 shown.

[0064] Step 2.2: Convert the matrix and The transposed matrix of Use QR decomposition technology to decompose them respectively and get:

[0065] ,

[0066] ,

[0067] So the matrix It can be expressed as:

[0068] ,

[0069] in and is an orthogonal matrix.

[0070] Step 2.3: The composed matrix is further compressed and decomposed using the SVD algorithm with truncated precision (t):

[0071] ,

[0072] Then the matrix It can be expressed as:

[0073] ,

[0074] Step 2.4: Perform matrix multiplication and merging, that is and Final off-diagonal block matrix It can be approximately expressed as:

[0075] ,

[0076] Step 3: Repeat step 2 to complete the compression decomposition of all non-diagonal block matrices. Taking a 2-layer matrix as an example, the corresponding matrix W with a stacked block structure becomes:

[0077] ,

[0078] Step 4: Extract the diagonal matrix blocks of the stacked block matrix obtained after compression decomposition in sequence, so that the diagonal blocks of the stacked block matrix become the unit matrix. At the same time, in order to keep the matrices equal, all U matrices need to be multiplied by the inverse of the corresponding diagonal block matrix and updated as Therefore, the stacked block matrix can be expressed as the multiplication of multiple block diagonal matrices, such as Figure 4 As shown, taking a 2-layer matrix as an example, it is expressed as:

[0079] ,

[0080] , , ,

[0081] In the above formula, represents the identity matrix, and the diagonal matrix block can be uniformly expressed as:

[0082] ,

[0083] p represents the layer number, and k represents the group number of the corresponding layer.

[0084] Step 5: Each diagonal block matrix Further decomposed into:

[0085] ,

[0086] It can be further expressed as , then according to the Sherman-Morrison-Woodbury formula, The inverse matrix can be expressed as:

[0087] ,

[0088] in,

[0089] ,

[0090] matrix The dimension is The rank of a non-diagonal block matrix is much smaller than The dimension of is reduced, which realizes the dimension reduction in the process of matrix inversion.

[0091] Step 6: Get the matrix The inverse of , which can then be expressed as a block diagonal matrix The inverse of matrix W can be expressed as:

[0092] ,

[0093] Where L represents the total number of layers of the stacked matrix. Each block diagonal matrix Each diagonal block matrix in They are all relatively independent, there is no recursive relationship in the matrix operation process, and highly parallel operations can be achieved.

[0094] in addition,

[0095] The device of the present invention comprises: a matrix stacking grouping module, an ACA-SVD matrix decomposition module, a matrix processing and storage module, a model training module, and a model reasoning module.

[0096] The matrix stacking grouping module is used to group and decompose the large-dimensional matrix W in the artificial intelligence model according to the process in the aforementioned step 1, and convert it into a stacked block matrix structure.

[0097] ACA-SVD matrix decomposition module is used to implement the non-diagonal block matrix according to the process in step 2 Perform compression decomposition and express it as two small matrices and Multiply, that is , and complete the compressed decomposition of all non-diagonal block matrices in matrix W according to the process in step 3.

[0098] The matrix processing and storage module is used to extract the diagonal matrix blocks of the stacked block matrix obtained after the compression decomposition in sequence according to the process in step 4, and perform corresponding matrix processing, and finally represent the stacked block matrix as the multiplication of multiple block diagonal matrices, and then store each block diagonal matrix.

[0099] The model training module is used to train the artificial intelligence model. If the matrix inversion operation is used in this process, the low computational complexity of the matrix inversion is achieved according to steps 5 and 6, and this process can be highly parallelized.

[0100] The model inference module is used to complete the reasoning of the artificial intelligence model. If the matrix inversion operation is used in this process, the low computational complexity of the matrix inversion is implemented according to steps 5 and 6, and this process can be highly parallelized.

[0101] The above description is only a preferred embodiment of the present invention and is only used to illustrate the technical solution of the present invention, and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention are included in the scope of protection of the present invention.

Claims

1. A method for accelerating the training and reasoning of an artificial intelligence model, characterized in that: This method is suitable for accelerating the training and inference of artificial intelligence models to achieve speech recognition, image recognition, and natural language processing. It uses multi-layer matrix low-rank compression technology to compress and decompose the large-dimensional matrices of the covariance matrix and weight matrix in the artificial intelligence model and convert them into several block diagonal matrix multiplications. Then, the matrix inversion process of the covariance matrix and weight matrix can be converted into the inversion of several block diagonal matrices, which not only reduces the resource usage of matrix storage during model deployment, but also reduces the computational complexity of matrix inversion operations during model training and inference. In addition, each block diagonal matrix is independent of each other, and there is no recursive relationship during the operation process, which can achieve highly parallel operations. The steps include: S1. Use the stacking grouping technique to group and decompose the large-dimensional matrix W in the artificial intelligence model into a stacked block matrix structure; S2. Use ACA-SVD matrix decomposition technology to decompose the non-diagonal block matrix W ij Perform compression decomposition, represented as two matrices U ij and V ij Multiplication, that is, W ij =U ij ·V ij , where i and j represent the group number of the interaction group in the layer where the non-diagonal block matrix is located, and are also the row and column numbers of the matrix block in the corresponding layer; S3. Loop step S2 to complete the compressed decomposition of all non-diagonal block matrices; S4. The diagonal matrix blocks of the stacked block matrix obtained after the compression decomposition are extracted in sequence and subjected to corresponding matrix processing, and finally the stacked block matrix is represented as a multiplication of several block diagonal matrices; S5. Each diagonal block matrix Decompose and then express it according to the Sherman-Morrison-Woodbury formula The inverse matrix of , realizes the dimension reduction in the process of matrix inversion; S6. Get the matrix The inverse of , which then expresses the block diagonal matrix D p The inverse of the matrix W is represented by the multiplication of several block diagonal inverse matrices.

2. The method according to claim 1, characterized in that In step S1, the large-dimensional matrix W in the artificial intelligence model is grouped and decomposed using the stacking grouping technology, and the specific steps of converting it into a stacked block matrix structure are as follows: S1.

1. Set the minimum matrix block dimension threshold during group decomposition to min_n; S1.

2. Split the matrix W into equal-dimensional rows and columns to obtain the first-level grouping matrix. Determine whether the dimension of each matrix block is less than the dimension threshold min_n. If so, stop the operation; otherwise, proceed to the next step. S1.

3. Perform equal-dimensional row and column splits on the diagonal blocks of the previous grouping matrix to obtain the next grouping matrix. Determine whether the dimension of each matrix block is less than the dimension threshold min_n. If so, stop the operation; otherwise, proceed to the next step. S1.

4. Repeat step S1.3 until the minimum matrix block dimension is less than the dimension threshold min_n. Stop the operation and assume that a 2-layer matrix is obtained. The corresponding matrix W becomes: In the formula, the superscript represents the layer number, and the subscript represents the group number of the interaction group in the corresponding layer.

3. The method according to claim 1, characterized in that In step S2, the non-diagonal block matrix W is decomposed using the ACA-SVD matrix decomposition technique. ij The specific steps for compression decomposition are: S2.

1. Using ACA algorithm to calculate the matrix W ij During the compression process, it is necessary to set an allowable precision (ε) for the ACA algorithm so that the matrix X ij Decompose into two matrices X ij and Y ij ; S2.

2. Transform the matrix X ij and Y ij The transposed matrix of Use QR decomposition technology to decompose them respectively and get: X ij =Q1·R1 So the matrix W ij Expressed as: Among them, Q1 and is an orthogonal matrix; S2.

3. The matrix is compressed and decomposed using the SVD algorithm with truncated precision (τ): Then the matrix W ij Expressed as: S2.

4. Perform matrix multiplication and merging, that is, and The final off-diagonal block matrix W ij Expressed as: W ij =U ij ·V ij 。 4. The method according to claim 1, wherein In step S3, the compressed decomposition of all non-diagonal block matrices is completed. Specifically, after the compressed decomposition of all non-diagonal block matrices is completed, a two-layer matrix is assumed, and the corresponding matrix W with a stacked block structure is expressed as:

5. The method according to claim 1, wherein In step S4, the stacked block matrix is represented as the multiplication of several block diagonal matrices. Specifically, the diagonal matrix blocks of the stacked block matrix obtained after the compressed decomposition are extracted in sequence, so that the diagonal blocks of the stacked block matrix become the unit matrix. At the same time, in order to keep the matrices equal, all U matrices need to be multiplied by the inverse of the corresponding diagonal block matrix to update to U′. Therefore, the stacked block matrix is represented as the multiplication of several block diagonal matrices. Assume that there are two layers of matrices, which are represented as follows: In the above formula, I represents the identity matrix, and the diagonal matrix block is uniformly expressed as: p represents the layer number, and k represents the group number of the corresponding layer.

6. The method according to claim 1, characterized in that In step S5, each diagonal block matrix Decomposition, specifically: each diagonal block matrix Decompose into Expressed as 7. The method according to claim 1, characterized in that In step S5, according to the Sherman-Morrison-Woodbury formula, The inverse matrix of , realizes the dimension reduction in the process of matrix inversion, specifically: The inverse matrix of in matrix The dimension is The rank of a non-diagonal block matrix is less than The dimension of is reduced, which realizes the dimension reduction in the process of matrix inversion.

8. The method according to claim 1, characterized in that In step S6, the inverse matrix of the matrix W is represented by multiplying several block diagonal inverse matrices. Specifically, the inverse matrix of the matrix W is represented as: W -1 =(D 0 ) -1 ·(D 1 ) -1 ...(D p ) -1 ...(D L-1 ) -1 ·(D L ) -1 Where L represents the total number of layers of the stacked matrix; each block diagonal matrix D p Each diagonal block matrix in They are all relatively independent, there is no recursive relationship in the matrix operation process, and highly parallel operations can be achieved.

9. An artificial intelligence model accelerated training and inference device, used to implement the method of claim 1. It is characterized by: Includes: matrix stacking grouping module, ACA-SVD matrix decomposition module, matrix processing and storage module, model training module, and model inference module; in, A matrix stacking grouping module is used to group and decompose the large-dimensional matrix W in the artificial intelligence model according to the process in step S1 and convert it into a stacked block matrix structure; The ACA-SVD matrix decomposition module is used to implement the non-diagonal block matrix W according to the process in step S2. ij Perform compression decomposition, represented as two matrices U ij and V ij Multiplication, that is, W ij =U ij V ij , and complete the compressed decomposition of all non-diagonal block matrices in the matrix W according to the process in step S3; a matrix processing and storage module, configured to sequentially extract the diagonal matrix blocks of the stacked block matrix obtained after the compression decomposition according to the process in step S4, perform corresponding matrix processing, and ultimately represent the stacked block matrix as a product of several block diagonal matrices, and then store each block diagonal matrix; The model training module is used to train the artificial intelligence model. If matrix inversion is required in this process, the matrix inversion is implemented with low computational complexity according to steps S5 and S6, and this process can be highly parallelized. The model inference module is used to complete the reasoning of the artificial intelligence model. If the matrix inversion operation is used in this process, the low computational complexity of the matrix inversion is implemented according to steps S5 and S6, and this process can be highly parallelized.

Images