Artificial intelligence model accelerated training reasoning method and device

Abstract

The invention provides an artificial intelligence model accelerated training reasoning method and device, and belongs to the field of artificial intelligence, and the method comprises the steps: firstly carrying out the grouping disassembly of a large-dimension matrix in an artificial intelligence model through employing a lamination grouping technology, and converting the large-dimension matrix into a lamination block matrix structure; then, carrying out compression decomposition on the non-diagonal block matrix by utilizing an ACA-SVD matrix decomposition technology, and representing that two small matrixes are multiplied; then diagonal matrix blocks of the obtained laminated block matrix structure are extracted in sequence, the matrix is expressed as multiplication of a plurality of block diagonal matrixes, and the block diagonal matrixes are stored respectively; when model training and reasoning are carried out, a matrix multiplication process can be converted into multiplication of a plurality of block diagonal matrixes, a matrix inversion process can be converted into inversion of the plurality of block diagonal matrixes, the block diagonal matrixes are mutually independent, a recursive relationship does not exist in an operation process, and highly parallel operation can be realized.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a method and device for accelerating the training and reasoning of an artificial intelligence model. Background Art

[0002] With the rapid development of technologies such as the Internet, big data, and cloud computing, artificial intelligence has gained unprecedented data resources and computing power. Breakthroughs in key artificial intelligence technologies such as machine learning, deep learning, reinforcement learning, and transfer learning have enabled artificial intelligence to make significant progress in speech recognition, image recognition, and natural language processing. This progress is not only reflected in major theoretical breakthroughs, but also in practical applications in multiple industrial fields. The application of artificial intelligence models has gradually penetrated into multiple fields such as medical care, transportation, finance, education, and manufacturing. However, in practical applications, in addition to considering the application effect of the model itself, it is also necessary to consider the cost of model application, which mainly comes from the occupation of storage resources and computing resources. In artificial intelligence models, a large number of matrices need to be stored and a large number of matrix operations need to be performed. Especially when processing large-scale data applications, the dimensions of the matrices are large. The high storage occupied by deployment and the high computational complexity of matrix operations will greatly increase the resource usage of the model during training and reasoning, thereby increasing the application cost. Summary of the invention

[0003] In response to the urgent need to reduce the resource cost occupied during training and reasoning processes faced by artificial intelligence models when they are promoted and applied, the present invention provides a method for accelerating training and reasoning of artificial intelligence models. Multi-layer matrix low-rank compression technology is used to compress, decompose and transform large-dimensional matrices such as covariance matrices and weight matrices in artificial intelligence models. Finally, the matrix is ​​represented as the multiplication of multiple block diagonal matrices, and each block diagonal matrix is ​​stored separately to reduce storage resource usage. In addition, the computational complexity of matrix inversion operations in the model training and reasoning process is reduced and highly parallelized, thereby lowering the computing power threshold for model use.

[0004] The technical solution of the present invention is: An artificial intelligence model accelerated training reasoning method comprises the following steps: S1. Use the stacking grouping technology to group and decompose the large-dimensional matrix W in the artificial intelligence model and convert it into a stacked block matrix structure.

[0005] S2. Use ACA-SVD matrix decomposition technology to decompose the non-diagonal block matrix Perform compression decomposition and represent it as two small matrices and Multiply, that is , where i and j represent the group number of the interaction group in the layer where the non-diagonal block matrix is ​​located, which is also the row and column number of the corresponding layer matrix block.

[0006] S3. Loop through step S2 to complete the compression decomposition of all non-diagonal block matrices.

[0007] S4. Extract the diagonal matrix blocks of the stacked block matrix obtained after the compression decomposition in sequence, and perform corresponding matrix processing, and finally represent the stacked block matrix as the multiplication of multiple block diagonal matrices.

[0008] S5. Each diagonal block matrix Further decomposition, then according to the Sherman-Morrison-Woodbury formula The inverse matrix of is used to achieve dimensionality reduction in the process of matrix inversion.

[0009] S6. Get the matrix The inverse of , which can then be expressed as a block diagonal matrix The inverse of W can be expressed as the multiplication of multiple block diagonal inverse matrices.

[0010] Furthermore, in step S1, the specific steps of grouping and decomposing the large-dimensional matrix W in the artificial intelligence model using the stacking grouping technology and converting it into a stacked block matrix structure are as follows: S1.1. Set the minimum matrix block dimension threshold during group decomposition to min_n.

[0011] S1.2. Split the matrix W into rows and columns with equal dimensions to obtain the first-layer grouping matrix. Determine whether the dimension of each matrix block is less than the dimension threshold min_n. If so, stop the operation, otherwise proceed to the next step.

[0012] S1.3. Perform row and column splitting of equal dimensions on the diagonal blocks of the previous layer of grouping matrix to obtain the next layer of grouping matrix. Determine whether the dimension of each matrix block is less than the dimension threshold min_n. If so, stop the operation, otherwise proceed to the next step.

[0013] S1.4. Repeat step S1.3 until the minimum matrix block dimension is less than the dimension threshold min_n, then stop the operation. Taking the 2-layer matrix as an example, the corresponding matrix W becomes: , In the formula, the superscript indicates the layer number, and the subscript indicates the group number of the interaction group in the corresponding layer.

[0014] Furthermore, in step S2, the non-diagonal block matrix is ​​decomposed using the ACA-SVD matrix decomposition technique. The specific steps for compression decomposition are: S2.1. Using ACA algorithm to calculate the matrix During the compression process, the ACA algorithm needs to set an allowable precision (e), which can make the matrix Decompose into two matrices with smaller dimensions and .

[0015] S2.2. The matrix and The transposed matrix of Use QR decomposition technology to decompose them respectively and get: , , So the matrix It can be expressed as: , p represents the layer number, and k represents the group number of the corresponding layer.

[0016] Furthermore, in step S5, each diagonal block matrix The specific characteristics of further decomposition are: each diagonal block matrix Further broken down into: , It can be further expressed as: .

[0017] Furthermore, in step S5, according to the Sherman-Morrison-Woodbury formula The inverse matrix of , realizes the dimension reduction in the process of matrix inversion, and its specific features are: The inverse matrix can be expressed as: , in , matrix The dimension is The rank of a non-diagonal block matrix is ​​much smaller than The dimension of is realized in the process of matrix inversion.

[0018] Furthermore, in step S6, the inverse matrix of the matrix W can be expressed as the multiplication of multiple block diagonal inverse matrices. The specific characteristics are: the inverse matrix of the matrix W can be expressed as: , Where L represents the total number of layers of the stacked matrix. Each block diagonal matrix Each diagonal block matrix in They are all relatively independent, there is no recursive relationship in the matrix operation process, and highly parallel operations can be achieved.

[0019] In addition, the present invention also provides an artificial intelligence model accelerated training and inference device for implementing the above-mentioned method, including: a matrix stacking grouping module, an ACA-SVD matrix decomposition module, a matrix processing and storage module, a model training module, and a model inference module.

[0020] in, The matrix stacking grouping module is used to group and decompose the large-dimensional matrix W in the artificial intelligence model according to the process in step S1 and convert it into a stacked block matrix structure.

[0021] The ACA-SVD matrix decomposition module is used to implement the process in step S2 to decompose the non-diagonal block matrix Perform compression decomposition and represent it as two small matrices and Multiply, that is , and complete the compressed decomposition of all non-diagonal block matrices in the matrix W according to the process in step S3.

[0022] The matrix processing and storage module is used to extract the diagonal matrix blocks of the stacked block matrix obtained after the compression decomposition in sequence according to the process in step S4, and perform corresponding matrix processing, and finally represent the stacked block matrix as the multiplication of multiple block diagonal matrices, and then store each block diagonal matrix.

[0023] The model training module is used to train the artificial intelligence model. If the matrix inversion operation is used in this process, the low computational complexity of the matrix inversion is implemented according to steps S5 and S6, and this process can be highly parallelized.

[0024] The model reasoning module is used to complete the reasoning of the artificial intelligence model. If the matrix inversion operation is used in this process, the low computational complexity of the matrix inversion is implemented according to steps S5 and S6, and this process can be highly parallelized.

[0025] The beneficial effects of the present invention are The present invention utilizes multi-layer matrix low-rank compression technology to compress and decompose large-dimensional matrices such as covariance matrices and weight matrices in artificial intelligence models and convert them into multiple block diagonal matrix multiplications. Then, the matrix inversion process of the covariance matrix and the weight matrix can be converted into the inversion of multiple block diagonal matrices, which not only reduces the resource occupation of matrix storage during model deployment, but also reduces the computational complexity of matrix inversion operations during model training and inference. Moreover, each block diagonal matrix is ​​independent of each other, and there is no recursive relationship during the operation process, which can achieve highly parallel operations and fully reduce the computing power threshold for model use. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a schematic diagram of accelerating training and reasoning of an artificial intelligence model based on multi-layer matrix low-rank compression; Figure 2 It is a schematic diagram of the matrix stacking structure (taking a 2-layer matrix as an example); Figure 3 This is a schematic diagram of the ACA decomposition; Figure 4 It is a schematic diagram showing the representation of a stacked block matrix as the multiplication of multiple block diagonal matrices. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0028] In order to reduce the demand for storage resources and computing resources during the training and reasoning of artificial intelligence models, the present invention utilizes multi-layer matrix low-rank compression technology to compress and decompose large-dimensional matrices such as covariance matrices and weight matrices in artificial intelligence models and convert them into multiple block diagonal matrix multiplications. The matrix inversion process of the covariance matrix and the weight matrix can be converted into the inversion of multiple block diagonal matrices, which not only reduces the resource occupation of matrix storage during model deployment, but also reduces the computational complexity of matrix inversion operations during model training and reasoning. Moreover, each block diagonal matrix is ​​independent of each other, and there is no recursive relationship during the operation process, so highly parallel operations can be achieved, which fully reduces the computing power threshold for model use.

[0029] The present invention firstly utilizes the stacking grouping technology to group and disassemble the large-dimensional matrix in the artificial intelligence model, and converts it into a stacking block matrix structure; then utilizes the ACA-SVD matrix decomposition technology to compress and decompose the non-diagonal block matrix, and expresses it as the multiplication of two small matrices; then extracts the diagonal matrix blocks of the obtained stacking block matrix structure in turn, and expresses the matrix as the multiplication of multiple block diagonal matrices, and stores each block diagonal matrix separately; when performing model training and inference, the matrix multiplication process can be converted into the multiplication of multiple block diagonal matrices, and the matrix inversion process can be converted into the inversion of multiple block diagonal matrices, each block diagonal matrix is ​​independent of each other, there is no recursive relationship in the operation process, and highly parallel operation can be achieved.

[0030] like Figure 1 As shown, the specific steps are as follows: Step 1: Use the stacking grouping technology to group and decompose the large-dimensional matrix W in the artificial intelligence model and convert it into a stacked block matrix structure. The specific steps are as follows: Step 1.1: Set the minimum matrix block dimension threshold during group decomposition to min_n.

[0031] Step 1.2: Split the matrix W into rows and columns with equal dimensions to obtain the first-layer grouping matrix. Determine whether the dimension of each matrix block is less than the dimension threshold min_n. If so, stop the operation, otherwise proceed to the next step.

[0032] Step 1.3: Perform row and column splitting of the diagonal blocks of the previous layer grouping matrix with equal dimensions to obtain the next layer grouping matrix. Determine whether the dimension of each matrix block is less than the dimension threshold min_n. If so, stop the operation, otherwise proceed to the next step.

[0033] Step 1.4: Repeat step 1.3 until the minimum matrix block dimension is less than the dimension threshold min_n, then stop the operation. Take the 2-layer matrix as an example, and get Figure 2 The stacked structure shown, the corresponding matrix W becomes: , In the formula, the superscript indicates the layer number, and the subscript indicates the group number of the interaction group in the corresponding layer.

[0034] Step 2: Use ACA-SVD matrix decomposition technique to decompose the non-diagonal block matrix Perform compression decomposition and represent it as two small matrices and Multiply, where i and j represent the group number of the interaction group of the layer where the non-diagonal block matrix is ​​located, and are also the row and column numbers of the corresponding layer matrix block. The specific steps are as follows: Step 2.1: Use the ACA algorithm to adjust the matrix During the compression decomposition process, the ACA algorithm needs to set an allowable precision (e), which can make the matrix Decompose into two matrices with smaller dimensions and , its decomposition diagram is as follows Figure 3 shown.

[0035] Step 2.2: Convert the matrix and The transposed matrix of Use QR decomposition technology to decompose them respectively and get: , , So the matrix It can be expressed as: , in and is an orthogonal matrix.

[0036] Step 2.3: The composed matrix is ​​further compressed and decomposed using the SVD algorithm with truncated precision (t): , Then the matrix It can be expressed as: , Step 2.4: Perform matrix multiplication and merging, that is and The final off-diagonal block matrix It can be approximately expressed as: , Step 3: Repeat step 2 to complete the compression decomposition of all non-diagonal block matrices. Taking a 2-layer matrix as an example, the corresponding matrix W with a stacked block structure becomes: , Step 4: Extract the diagonal matrix blocks of the stacked block matrix obtained after compression decomposition one by one, so that the diagonal blocks of the stacked block matrix become the unit matrix. At the same time, in order to keep the matrices equal, all U matrices need to be multiplied by the inverse of the corresponding diagonal block matrix and updated as Therefore, the stacked block matrix can be expressed as the multiplication of multiple block diagonal matrices, such as Figure 4 As shown, taking a 2-layer matrix as an example, it is expressed as: , , , , In the above formula, represents the identity matrix, and the diagonal matrix block can be uniformly expressed as: , p represents the layer number, and k represents the group number of the corresponding layer.

[0037] Step 5: Transform each diagonal block matrix Further broken down into: , It can be further expressed as , and then according to the Sherman-Morrison-Woodbury formula, The inverse matrix can be expressed as: , in, , matrix The dimension is The rank of a non-diagonal block matrix is ​​much smaller than The dimension of is realized in the process of matrix inversion.

[0038] Step 6: Get the matrix The inverse of , which can then be expressed as a block diagonal matrix The inverse of matrix W can be expressed as: , Where L represents the total number of layers of the stacked matrix. Each block diagonal matrix Each diagonal block matrix in They are all relatively independent, there is no recursive relationship in the matrix operation process, and highly parallel operations can be achieved.

[0039] in addition, The device of the invention comprises: a matrix stacking grouping module, an ACA-SVD matrix decomposition module, a matrix processing storage module, a model training module and a model reasoning module.

[0040] The matrix stacking grouping module is used to group and decompose the large-dimensional matrix W in the artificial intelligence model according to the process in the above step 1, and convert it into a stacked block matrix structure.

[0041] The ACA-SVD matrix decomposition module is used to implement the process in step 2 to decompose the non-diagonal block matrix Perform compression decomposition and represent it as two small matrices and Multiply, that is , and complete the compressed decomposition of all non-diagonal block matrices in the matrix W according to the process in step 3.

[0042] The matrix processing and storage module is used to extract the diagonal matrix blocks of the stacked block matrix obtained after the compression decomposition in sequence according to the process in step 4, and perform corresponding matrix processing, and finally represent the stacked block matrix as the multiplication of multiple block diagonal matrices, and then store each block diagonal matrix.

[0043] The model training module is used to train the artificial intelligence model. If the matrix inversion operation is used in this process, the low computational complexity of the matrix inversion is implemented according to steps 5 and 6, and this process can be highly parallelized.

[0044] The model reasoning module is used to complete the reasoning of the artificial intelligence model. If the matrix inversion operation is used in this process, the low computational complexity of the matrix inversion is implemented according to steps 5 and 6, and this process can be highly parallelized.

[0045] The above description is only a preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, and is not used to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for accelerating training and reasoning of an artificial intelligence model, characterized in that: The steps include: S1. Use the stacking grouping technology to group and disassemble the large-dimensional matrix W in the artificial intelligence model into a stacked block matrix structure; S2. Use ACA-SVD matrix decomposition technology to decompose the non-diagonal block matrix Perform compression decomposition and represent it as two matrices and Multiply, that is , where i and j represent the group number of the interaction group in the layer where the non-diagonal block matrix is ​​located, and are also the row and column numbers of the corresponding layer matrix block; S3. Loop through step S2 to complete the compression decomposition of all non-diagonal block matrices; S4. Extract the diagonal matrix blocks of the stacked block matrix obtained after the compression decomposition in sequence, and perform corresponding matrix processing, and finally represent the stacked block matrix as a multiplication of several block diagonal matrices; S5. Each diagonal block matrix Decomposed, and then expressed according to the Sherman-Morrison-Woodbury formula The inverse matrix of , realizes the dimension reduction in the process of matrix inversion; S6. Get the matrix The inverse of The inverse of W is thus represented by the multiplication of several block diagonal inverse matrices.

2. The method according to claim 1, characterized in that In step S1, the specific steps of grouping and decomposing the large-dimensional matrix W in the artificial intelligence model using the stacking grouping technology and converting it into a stacked block matrix structure are as follows: S1.

1. Set the minimum matrix block dimension threshold in the grouping and decomposition process to min_n; S1.

2. Perform row and column splitting of equal dimensions on the matrix W to obtain the first-layer grouping matrix. Determine whether the dimension of each matrix block is less than the dimension threshold min_n. If so, stop the operation, otherwise proceed to the next step. S1.

3. Perform row and column splitting of the diagonal blocks of the previous grouping matrix with equal dimensions to obtain the next grouping matrix. Determine whether the dimension of each matrix block is less than the dimension threshold min_n. If so, stop the operation, otherwise proceed to the next step. S1.

4. Repeat step S1.3 until the minimum matrix block dimension is less than the dimension threshold min_n, then stop the operation. Suppose a 2-layer matrix is ​​obtained, and the corresponding matrix W becomes: ， In the formula, the superscript indicates the layer number, and the subscript indicates the group number of the interaction group in the corresponding layer.

3. The method according to claim 1, characterized in that In step S2, the non-diagonal block matrix is ​​decomposed by using the ACA-SVD matrix decomposition technique. The specific steps for compression decomposition are: S2.

1. Using ACA algorithm to calculate the matrix During the compression process, the ACA algorithm needs to set an allowable precision (e) so that the matrix Decompose into two matrices and ; S2.

2. The matrix and The transposed matrix of Use QR decomposition technology to decompose them respectively and get: ， ， So the matrix It is expressed as: ， in and is an orthogonal matrix; S2.

3. The matrix is ​​compressed and decomposed using the SVD algorithm with truncated precision (t): ， Then the matrix It is expressed as: ， S2.

4. Perform matrix multiplication and merging, that is and The final off-diagonal block matrix It is expressed as: 。 4. The method according to claim 1, characterized in that In the step S3, the compression decomposition of all non-diagonal block matrices is completed, specifically: after the compression decomposition of all non-diagonal block matrices is completed, assuming two layers of matrices, the corresponding matrix W with a stacked block structure is expressed as: 。 5. The method according to claim 1, characterized in that In step S4, the specific content of expressing the stacked block matrix as the multiplication of several block diagonal matrices is as follows: the diagonal matrix blocks of the stacked block matrix obtained after compression decomposition are extracted in sequence, so that the diagonal blocks of the stacked block matrix become unit matrices. At the same time, in order to keep the matrices equal, all U matrices need to be multiplied by the inverse of the corresponding diagonal block matrix and updated as follows: ; Therefore, the stacked block matrix is ​​represented by the multiplication of several block diagonal matrices. Suppose there are two layers of matrices, which can be represented as: ， ， ， ， In the above formula, represents the identity matrix, and the diagonal matrix block is uniformly represented as: ， p represents the layer number, and k represents the group number of the corresponding layer.

6. The method according to claim 1, characterized in that In step S5, each diagonal block matrix Decomposition, specifically: each diagonal block matrix This breaks down into: ， Expressed as .

7. The method according to claim 1, characterized in that In step S5, according to the Sherman-Morrison-Woodbury formula, The inverse matrix of , realizes the dimension reduction in the process of matrix inversion, specifically: The inverse matrix of is expressed as: ， in, ， matrix The dimension is The rank of a non-diagonal block matrix is ​​less than The dimension of is realized in the process of matrix inversion.

8. The method according to claim 1, characterized in that In step S6, the inverse matrix of the matrix W is represented by multiplying a number of block diagonal inverse matrices. Specifically, the inverse matrix of the matrix W is represented by: ， Where L represents the total number of layers of the stacked matrix; each block diagonal matrix Each diagonal block matrix in They are all relatively independent, there is no recursive relationship in the matrix operation process, and highly parallel operations can be achieved.

9. An artificial intelligence model accelerated training and reasoning device, used to implement the method of claim 1, It is characterized in that Including: matrix stacking grouping module, ACA-SVD matrix decomposition module, matrix processing storage module, model training module, model reasoning module; in, A matrix stacking grouping module is used to group and decompose the large-dimensional matrix W in the artificial intelligence model according to the process in step S1, and convert it into a stacked block matrix structure; The ACA-SVD matrix decomposition module is used to implement the process in step S2 to decompose the non-diagonal block matrix Perform compression decomposition and represent it as two matrices and Multiply, that is , and complete the compressed decomposition of all non-diagonal block matrices in the matrix W according to the process in step S3; A matrix processing and storage module is used to extract the diagonal matrix blocks of the stacked block matrix obtained after the compression decomposition in sequence according to the process in step S4, and perform corresponding matrix processing, and finally represent the stacked block matrix as a multiplication of several block diagonal matrices, and then store each block diagonal matrix; The model training module is used to train the artificial intelligence model. If the matrix inversion operation is used in this process, the low computational complexity of the matrix inversion is implemented according to steps S5 and S6, and this process can be highly parallelized; The model reasoning module is used to complete the reasoning of the artificial intelligence model. If the matrix inversion operation is used in this process, the low computational complexity of the matrix inversion is implemented according to steps S5 and S6, and this process can be highly parallelized.

Images