Model fine tuning training method, device, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a model fine tuning training method and device, equipment and a medium. When fine tuning training is carried out on the model, a to-be-adjusted matrix in the fine tuning model is decomposed based on a singular value decomposition algorithm to obtain a plurality of decomposition matrixes, a target number is determined based on the plurality of decomposition matrixes, and an increment matrix is constructed based on the determined target number and the plurality of decomposition matrixes. The incremental matrix is constructed based on knowledge contained in the to-be-adjusted matrix instead of being completely independent of the to-be-adjusted matrix, the incremental matrix is added into the fine adjustment model to obtain the target model, and fine adjustment training is performed on the target model, so that the problems of long fine adjustment training time and poor effect of the model are avoided. The technical scheme protected by the invention has the characteristics of reliability, generalization and robustness, and accords with the trustworthy characteristic.

Description

[0001] This application claims the priority of the Chinese patent application filed with the China Patent Office on July 26, 2023, with application number 202310919300.9 and application name “A model fine-tuning training method, device, equipment and medium”, all contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of artificial intelligence technology, and in particular to a model fine-tuning training method, device, equipment and medium. Background Art

[0003] With the development of artificial intelligence technology, more and more models are applied to various industries. In particular, generative natural language models represented by the Chat Generative Pre-trained Transformer (ChatGPT) model have attracted widespread attention in recent years. The core principle behind technologies such as the ChatGPT model is the large-scale language model. Large-scale language models usually use the transformer model (Transformer) as the basic structure, involving tens to hundreds of billions of parameters. Training such a large-scale parameter model requires huge computing resources, such as more than 1,000 graphics cards, and the training time is as high as 1-2 months.

[0004] In order to reduce the cost of model development, technicians in this field have explored various efficient model parameter fine-tuning solutions, that is, based on the existing large model, they fine-tune the model by adjusting some of the model's parameters to make the large model suitable for specific downstream tasks.

[0005] In related technologies, reparameterization technology is usually used to fine-tune some parameters of a trained model. In large-scale language models, the parameters that need to be adjusted are usually matrices. Reparameterization technology is to add an incremental matrix to the matrix to be adjusted. When fine-tuning the model, the matrix to be adjusted is not adjusted, but only the added incremental matrix is ​​adjusted, thereby achieving the purpose of efficient fine-tuning.

[0006] In the related art, the incremental matrix added to the matrix to be adjusted is usually determined based on low-rank adaptation (LoRA). However, in practice, it is found that since the incremental matrix determined based on LoRA is completely independent of the matrix to be adjusted, that is, the knowledge contained in the matrix to be adjusted is not considered when determining the incremental matrix, the fine-tuning training time is long and the effect is poor.

[0007] Therefore, how to improve the efficiency and effectiveness of model fine-tuning training has become an urgent problem to be solved. Summary of the invention

[0008] The embodiments of the present application provide a model fine-tuning training method, device, equipment and medium to solve the problem of long time and poor effect of model fine-tuning training in the prior art.

[0009] The present application provides a model fine-tuning training method, the method comprising:

[0010] Get the matrix to be adjusted in the fine-tuning model;

[0011] Decomposing the matrix to be adjusted based on a singular value decomposition algorithm to obtain a plurality of decomposition matrices; determining a target quantity based on the plurality of decomposition matrices;

[0012] An incremental matrix is ​​constructed based on the target quantity and the multiple decomposition matrices; the incremental matrix is ​​added to the fine-tuning model to obtain a target model, and fine-tuning training is performed on the target model.

[0013] Further, the determining the target quantity based on the multiple decomposition matrices includes:

[0014] Based on the multiple decomposition matrices, singular values ​​are determined, the singular values ​​are sorted in order of size, a minimum sum when a first sum of a maximum singular value and other adjacent singular values ​​is greater than a preset threshold is determined, and a target number of target singular values ​​is included.

[0015] Furthermore, the determination of the minimum sum when the first sum of the maximum singular value and other adjacent singular values ​​is greater than a preset threshold value includes a target number of target singular values:

[0016] determining targets and values ​​of singular values ​​included in the singular value matrix;

[0017] Among the sorted singular values, determine the second sum of the maximum singular value and at least one other adjacent singular value one by one; and determine in turn whether the ratio of the second sum to the target sum is greater than the preset threshold;

[0018] When a second sum value is identified whose ratio to the target sum value is greater than the preset threshold, the second sum value is used as the target sum value, the singular value of the target sum value is determined, the determined singular value is used as the target singular value, and the number of the target singular values ​​is determined as the target number.

[0019] Furthermore, the preset threshold is any value greater than 0 and less than 1.

[0020] Further, constructing an incremental matrix based on the target quantity and the multiple decomposition matrices includes:

[0021] constructing a diagonal matrix using the target number of singular values;

[0022] In a first target matrix among the multiple decomposition matrices, select the target number of rows and / or columns of data to obtain a second target matrix;

[0023] A delta matrix is ​​constructed using the diagonal matrix and the second target matrix.

[0024] Furthermore, selecting the target number of rows and / or columns of data from a first target matrix in the multiple decomposition matrices to obtain a second target matrix includes:

[0025] Obtaining a first target matrix from among the multiple decomposition matrices, wherein the first target matrix includes a left singular value vector matrix and a right singular value vector matrix;

[0026] The target number of column data is selected from the left singular value vector matrix as the first matrix; and the target number of row data is selected from the right singular value vector matrix as the second matrix; the first matrix and the second matrix are determined as the second target matrix.

[0027] Further, the constructing an incremental matrix using the diagonal matrix and the second target matrix includes:

[0028] A matrix obtained by multiplying the first matrix, the diagonal matrix and the second matrix is ​​determined as the incremental matrix.

[0029] The present application also provides a model fine-tuning training device, the device comprising:

[0030] An acquisition module is used to obtain the matrix to be adjusted in the fine-tuning model;

[0031] A decomposition determination module, used to decompose the matrix to be adjusted based on a singular value decomposition algorithm to obtain multiple decomposition matrices; and determine the target quantity based on the multiple decomposition matrices;

[0032] A construction module, configured to construct an incremental matrix based on the target quantity and the plurality of decomposition matrices;

[0033] A training module is used to add the incremental matrix to the fine-tuning model to obtain a target model, and perform fine-tuning training on the target model.

[0034] An embodiment of the present application also provides an electronic device, which includes at least a processor and a memory, and the processor is used to implement the steps of the model fine-tuning training method as described in any one of the above items when executing a computer program stored in the memory.

[0035] An embodiment of the present application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the model fine-tuning training method as described in any one of the above items.

[0036] Because in an embodiment of the present application, when fine-tuning the model, the matrix to be adjusted in the fine-tuning model is decomposed based on the singular value decomposition algorithm to obtain multiple decomposition matrices, the target number is determined based on the multiple decomposition matrices, and an incremental matrix is ​​constructed based on the determined target number and the multiple decomposition matrices. The incremental matrix is ​​constructed based on the knowledge contained in the matrix to be adjusted, rather than being completely independent of the matrix to be adjusted. The incremental matrix is ​​added to the fine-tuning model to obtain the target model, and the target model is fine-tuned, thereby avoiding the problem of long time and poor effect of fine-tuning training of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 A flowchart of a model fine-tuning training method provided in an embodiment of the present application;

[0039] Figure 2 A schematic diagram of the model fine-tuning training principle provided in the embodiment of the present application;

[0040] Figure 3 A flowchart of another model fine-tuning training method provided in an embodiment of the present application;

[0041] Figure 4 A schematic diagram of the structure of a model fine-tuning training device provided in an embodiment of the present application;

[0042] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the embodiment of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiment is only a part of the embodiment of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of this application.

[0044] The embodiments of the present application provide a model fine-tuning training method, apparatus, device and medium, in which a matrix to be adjusted in a fine-tuning model is obtained; the matrix to be adjusted is decomposed based on a singular value decomposition algorithm to obtain multiple decomposition matrices; a target number is determined based on the multiple decomposition matrices; an incremental matrix is ​​constructed based on the target number and the multiple decomposition matrices; the incremental matrix is ​​added to the fine-tuning model to obtain a target model, and fine-tuning training is performed on the target model.

[0045] Figure 1 A flow chart of a model fine-tuning training method provided in an embodiment of the present application, the process includes the following steps:

[0046] S101: Obtain the matrix to be adjusted in the fine-tuning model.

[0047] The model fine-tuning training method provided in the embodiment of the present application is applied to an electronic device, which may be a server, a PC, etc.

[0048] In order to improve the fine-tuning training efficiency of the fine-tuning model, in an embodiment of the present application, the matrix to be adjusted of the fine-tuning model to be fine-tuned can be obtained, and the incremental matrix can be determined according to the matrix to be adjusted, and the incremental matrix can be added to the fine-tuning model, so as to fine-tune the fine-tuning model with the incremental matrix added. In an embodiment of the present application, the fine-tuning model can be a model that has been pre-trained, and the fine-tuning model can be stored locally in the electronic device, or it can be sent to the electronic device by the user of the electronic device. The matrix to be adjusted in the fine-tuning matrix can be understood as the training parameters that have been fixed in the pre-training of the fine-tuning model, that is, the fine-tuning model will adjust some training parameters in the fine-tuning model according to the results output during the training of the fine-tuning model during the pre-training process, and the fine-tuning model will fix the adjusted training parameters after the training is completed. In order to make the fine-tuning model that has been trained applicable to some specific application scenarios, it is necessary to fine-tune the fine-tuning model that has been trained. Then, when fine-tuning the training, it is necessary to adjust the fixed training parameters again, and the fixed training parameters are the matrix to be adjusted. In the embodiment of the present application, the matrix to be adjusted may be pre-identified in the running code of the fine-tuning model, or may be input by a user of the electronic device, and the matrix to be adjusted may be a matrix of any dimension.

[0049] Specifically, large-scale language models are mainly based on the Transformer architecture, that is, each large language model is composed of multiple stacked Transformer layers, and each Transformer layer is composed of a multi-head attention structure and a fully connected forward network structure. Among them, the multi-head attention structure is usually calculated as follows:

[0050] Q=XWq , K = XW k , V = XW v

[0051]

[0052] Among them, W q , W k and W V All represent the matrix parameters to be adjusted; X represents the input data of the multi-head attention structure, and the multi-head attention structure multiplies the input data with the matrix parameters to obtain matrices Q, K, and V; H represents the output result of the multi-head attention structure; softmax() represents the normalization function; d k Represents the dimension of matrix K; K T Represents the transposed matrix of matrix K.

[0053] The fully connected forward network structure is usually calculated as follows:

[0054] F=σ(XW1+b1)W2+b2

[0055] Among them, F represents the output result of the fully connected forward network structure; σ represents the weight; X represents the input data of the fully connected forward network structure; W1 and W2 represent the matrix parameters to be adjusted; b1 and b2 represent the bias parameters respectively.

[0056] It should be noted that the above-mentioned introductions to the multi-head attention structure and the fully connected forward network structure are all prior art and are only briefly described in the embodiments of the present application without detailed introduction.

[0057] Through the above analysis of the multi-head attention structure and the fully connected forward network structure, it can be found that the core of both the multi-head attention structure and the fully connected forward network structure is matrix operations, and these matrices involved in the operation constitute the most important parameters of the large-scale language model. In order to improve the efficiency of fine-tuning the model, when determining to fine-tune the fine-tuning model, an incremental matrix can be added to the matrix parameters to be adjusted in the pre-trained large-scale language model. Then, the above Wq, W k , W V , W1 and W2 are determined as matrices to be adjusted.

[0058] S102: Decomposing the matrix to be adjusted based on a singular value decomposition algorithm to obtain a plurality of decomposition matrices.

[0059] After obtaining the matrix to be adjusted of the fine-tuning model, the matrix to be adjusted in the fine-tuning model can be decomposed based on the singular value decomposition algorithm to obtain multiple decomposition matrices, wherein the multiple decomposition matrices may include a left singular value vector matrix, a singular value matrix and a right singular value vector matrix.

[0060] Specifically, assuming that the matrix to be adjusted is W, and the matrix to be adjusted consists of m rows and n columns, then the matrix to be adjusted can be expressed as W∈R m×n When fine-tuning training is performed, the matrix W to be adjusted can be decomposed based on the singular value decomposition algorithm to obtain the left singular value vector matrix, the singular value matrix and the right singular value vector matrix, where the left singular value vector matrix can be expressed as U∈R m×m , the left singular value vector matrix is ​​an m×m symmetric matrix; the right singular value vector matrix can be expressed as V∈R n×n , the right singular value vector matrix is ​​an n×n symmetric matrix; the singular value matrix can be expressed as Σ∈R m×n , the singular value matrix is ​​a semidefinite diagonal matrix. It should be noted that how to decompose the matrix to be adjusted based on the singular value decomposition algorithm is an existing technology and will not be described in detail here.

[0061] S103: Determine a target quantity based on the multiple decomposition matrices.

[0062] After the matrix to be adjusted is decomposed, the target number can be determined based on the multiple decomposition matrices obtained by decomposition. Since the singular values ​​in the decomposition matrix represent the importance of the knowledge contained in the matrix to be adjusted, in the embodiment of the present application, a singular value threshold can be pre-configured, and the singular values ​​are determined according to the multiple decomposition matrices obtained by decomposition, and the number of singular values ​​greater than the singular value threshold is determined in the singular values, and the number is determined as the target number. It is also possible to determine the average singular value of the singular values ​​in the multiple decomposition matrices, and determine the number of singular values ​​greater than the average singular value in the singular values, and determine the number as the target number.

[0063] S104: constructing an incremental matrix based on the target quantity and the multiple decomposition matrices; adding the incremental matrix to the fine-tuning model to obtain a target model, and performing fine-tuning training on the target model.

[0064] After determining the target number based on the decomposition matrix obtained by decomposition, an incremental matrix can be constructed based on the target number and the decomposition matrix obtained by decomposition. In an embodiment of the present application, a decomposition matrix can be randomly selected from the multiple decomposition matrices obtained by decomposition, and the target number of row / column data in the randomly selected decomposition matrix can be used as the incremental matrix.

[0065] After the incremental matrix is ​​determined, the incremental matrix is ​​added to the target model to obtain the target model, and the target model is fine-tuned. When fine-tuning the target model, only the incremental matrix in the target model is adjusted, and the original fixed matrix parameters, that is, the matrix to be adjusted, are not adjusted.

[0066] Specifically, assuming that the matrix to be adjusted is W, the matrix to be adjusted can be expressed as W∈R m×n That is to say, the matrix to be adjusted is composed of m rows and n columns, and an incremental matrix W0 of the same dimension is added to the matrix to be adjusted. The incremental matrix W0 is composed of m rows and n columns. W0 is added to the fine-tuning model to obtain the target model. The parameters to be adjusted in the target model can be expressed as h=X(W+W0)=XW+XW0. In the fine-tuning model, the amount of data involved in the parameters to be adjusted is mn. In the embodiment of the present application, after the incremental matrix is ​​added to the fine-tuning model, when m and n are in the order of 1000-10000, the amount of parameters that need to be adjusted in the fine-tuning training process is much lower than the matrix to be adjusted.

[0067] Because in an embodiment of the present application, when fine-tuning the model, the matrix to be adjusted in the fine-tuning model is decomposed based on the singular value decomposition algorithm to obtain multiple decomposition matrices, the target number is determined based on the multiple decomposition matrices, and an incremental matrix is ​​constructed based on the determined target number and the multiple decomposition matrices. The incremental matrix is ​​constructed based on the knowledge contained in the matrix to be adjusted, rather than being completely independent of the matrix to be adjusted. The incremental matrix is ​​added to the fine-tuning model to obtain the target model, and the target model is fine-tuned, thereby avoiding the problem of long time and poor effect of fine-tuning training of the model.

[0068] In order to further improve the efficiency of fine-tuning training, on the basis of the above embodiments, in an embodiment of the present application, the determining of the target quantity based on the multiple decomposition matrices includes:

[0069] Based on the multiple decomposition matrices, singular values ​​are determined, the singular values ​​are sorted in order of size, a minimum sum when a first sum of a maximum singular value and other adjacent singular values ​​is greater than a preset threshold is determined, and a target number of target singular values ​​is included.

[0070] After decomposing the matrix to be adjusted in the fine-tuning model, the singular value may be determined based on the multiple decomposition matrices obtained. In the embodiment of the present application, a singular value matrix in the multiple decomposition matrices may be obtained, and the singular values ​​included in the singular value matrix may be obtained. Each element of the diagonal of the singular value matrix represents a singular value. The singular value matrix may be expressed as:

[0071]

[0072] Among them, σ i , i=1,…,k are singular values; the number of singular values ​​is k, and the number of singular values ​​is the minimum value of the number of rows m and the number of columns n of the matrix to be adjusted. The singular value matrix described above is described by taking k=min(m,n)=m as an example.

[0073] It can be known from the matrix theory that the size of the singular value represents the importance of the knowledge contained in the matrix to be adjusted. Therefore, in the embodiment of the present application, the acquired singular values ​​can be sorted in order of size, and the first sum of the maximum singular value and other adjacent singular values ​​can be determined one by one. When the first sum is greater than the preset threshold, the first sum is determined as the candidate sum. After determining each first sum of the maximum singular value and other adjacent singular values, the minimum value among the determined candidate sums is determined as the target minimum sum, the singular values ​​involved in determining the target minimum sum are determined as the target singular values, and the number of singular values ​​involved is determined as the target number, which can be used as a low-rank parameter for the subsequent construction of an incremental matrix. Among them, the preset threshold can be pre-configured by the staff based on a large number of statistics.

[0074] In order to further improve the efficiency of fine-tuning training, on the basis of the above embodiments, in the embodiment of the present application, the minimum sum when the first sum of the maximum singular value and other adjacent singular values ​​is greater than the preset threshold is determined, and the target number of target singular values ​​included includes:

[0075] determining targets and values ​​of singular values ​​included in the singular value matrix;

[0076] Among the sorted singular values, determine the second sum of the maximum singular value and at least one other adjacent singular value one by one; and determine in turn whether the ratio of the second sum to the target sum is greater than the preset threshold;

[0077] When a second sum value is identified whose ratio to the target sum value is greater than the preset threshold, the second sum value is used as the target sum value, the singular value of the target sum value is determined, the determined singular value is used as the target singular value, and the number of the target singular values ​​is determined as the target number.

[0078] Since different pre-trained models have different parameters to be adjusted, the singular values ​​of different pre-trained models are also different. Therefore, in order to further improve the efficiency of fine-tuning training, when determining the target singular value and the target number, the targets and values ​​of all singular values ​​included in the singular value matrix can be determined.

[0079] Among the sorted singular values, the second sum of the largest singular value and at least one other adjacent singular value is determined one by one. Assuming that the singular values ​​are sorted in descending order, when determining the second sum, the second sum of the first-ranked singular value and the second-ranked singular value among the sorted singular values ​​is first determined; then the second sum of the first-ranked singular value, the second-ranked singular value and the third-ranked singular value among the sorted singular values ​​is determined, and so on.

[0080] After each second sum value is determined, it is determined in turn whether the ratio of the second sum value to the target sum value is greater than a preset threshold value, wherein the preset threshold value is any value greater than 0 and less than 1, for example, the preset threshold value may be 0.8, and the preset threshold value may be understood as a parameter indicating the degree to which information needs to be retained. When a second sum value having a ratio greater than the preset threshold value to the target sum value is identified, the second sum value may be used as the target sum value, the singular value involved in determining the target sum value may be determined as the target singular value, and the number of the target singular values ​​may be determined as the target number.

[0081] Specifically, all the singular values ​​included in the singular value matrix are arranged in order from large to small, and the sorted singular value sequence is [σ1,σ2,…,σ k ],σ1≥σ2…≥σ k , for this singular value sequence, the target number r can be determined based on the following formula:

[0082]

[0083] Among them, ε represents the preset threshold; σ i represents the i-th singular value in the singular value sequence; r represents the target number; k represents the total number of singular values ​​in the singular value sequence; st. represents the constraints that the equation must satisfy.

[0084] According to the above formula, when the ratio of the sum of the first r singular values ​​in the singular value sequence to the target sum is greater than or equal to the preset threshold ε, the first r singular values ​​are the target singular values, r is the target number, and r is the minimum value that meets the above conditions.

[0085] In an embodiment of the present application, the process of determining the target singular value and the target number can be determined based on a greedy algorithm, that is, traversing the first k values, and determining the target singular value and the target number based on the first value that meets the conditions.

[0086] In order to further improve the efficiency of fine-tuning training, on the basis of the above embodiments, in an embodiment of the present application, constructing an incremental matrix based on the target quantity and the multiple decomposition matrices includes:

[0087] constructing a diagonal matrix using the target number of singular values;

[0088] In a first target matrix among the multiple decomposition matrices, select the target number of rows and / or columns of data to obtain a second target matrix;

[0089] A delta matrix is ​​constructed using the diagonal matrix and the second target matrix.

[0090] When constructing an incremental matrix, the determined target number of singular values ​​can be used to construct a diagonal matrix. In an embodiment of the present application, singular values ​​can be determined based on the multiple decomposition matrices obtained by decomposition, and the determined singular values ​​can be sorted in order from large to small, and the first target number of singular values ​​in the sorting can be selected as the target singular values, and the diagonal matrix can be constructed using the target singular values.

[0091] After the diagonal matrix is ​​constructed, a first target matrix among the multiple decomposition matrices may be obtained, wherein the target matrix may be any one of the multiple decomposition matrices or any multiple decomposition matrices among the multiple decomposition matrices.

[0092] In an embodiment of the present application, a target number of rows and / or columns of data may be selected from the acquired first target matrix to obtain a second target matrix, wherein the second target matrix may be one or more.

[0093] When there is only one first target matrix, the target number of row data can be selected in the first target matrix as the second target matrix, or the target number of column data can be selected as the second target matrix. When selecting the target number of row data and column data in the first target matrix, any selection can be made. For example, the first target number of row data or column data in the first target matrix can be selected, or the last target number of row data or column data in the first target matrix can be selected. The target number of row data or column data at any position in the first target matrix can also be selected.

[0094] When there are multiple first target matrices, illustratively, a target number of row or column data can be selected from target matrix A in the first target matrix as the second target matrix, and a target number of row or column data can be selected from target matrix B in the first target matrix as the second target matrix.

[0095] After obtaining the diagonal matrix and the second target matrix, the diagonal matrix and the second target matrix can be used to construct an incremental matrix. When constructing the incremental matrix, it can be determined whether to add the diagonal matrix and the second target matrix to obtain the incremental matrix, or to multiply the diagonal matrix and the second target matrix to obtain the incremental matrix, based on the number of rows and columns of the diagonal matrix and the number of rows and columns of the second target matrix. The embodiment of the present application not only has row restrictions on how to construct the incremental matrix based on the diagonal matrix and the second target matrix, but technicians in this field can make adjustments based on the determined number of rows and columns of the diagonal matrix and the second target matrix.

[0096] In order to further improve the efficiency of fine-tuning training, on the basis of the above embodiments, in an embodiment of the present application, in the first target matrix among the multiple decomposition matrices, selecting the target number of rows and / or columns of data to obtain the second target matrix includes:

[0097] Obtaining a first target matrix from among the multiple decomposition matrices, wherein the first target matrix includes a left singular value vector matrix and a right singular value vector matrix;

[0098] The target number of column data is selected from the left singular value vector matrix as the first matrix; and the target number of row data is selected from the right singular value vector matrix as the second matrix; the first matrix and the second matrix are determined as the second target matrix.

[0099] When determining the second target matrix, a first target matrix from among multiple decomposition matrices obtained based on a singular value decomposition algorithm may be obtained, wherein the first target matrix may be a left singular value vector matrix and a right singular value vector matrix. In an embodiment of the present application, a front target number of column data may be selected from the left singular value vector matrix as a first matrix, and a front target number of row data may be selected from the right singular value vector matrix as a second matrix, wherein the first matrix and the second matrix are the determined second target matrix.

[0100] In an embodiment of the present application, constructing an incremental matrix using the diagonal matrix and the second target matrix includes:

[0101] A matrix obtained by multiplying the first matrix, the diagonal matrix and the second matrix is ​​determined as the incremental matrix.

[0102] After the first matrix and the second matrix determine the second target matrix, the first matrix, the diagonal matrix and the second matrix can be multiplied to obtain an incremental matrix.

[0103] Specifically, the diagonal matrix constructed using the target singular values ​​is Σ r , the diagonal matrix can be expressed as Σ r =diag([σ1,…,σ r ])∈R r×r , where diag() means constructing a diagonal matrix, σ1,…,σ r represents the singular value of the number of targets, r represents the number of targets, that is, the low-rank parameter, and the diagonal matrix is ​​Σ r That is, a matrix with r rows and r columns; select the column data of the previous target number r in the left singular value vector matrix, and the first matrix obtained is U r , the first matrix can be expressed as U r =U[:,:r]∈R m ×r ; Select the row data of the previous target number r in the right singular value vector matrix, and the second matrix obtained is V r , the second matrix can be expressed as V r =V[:r,:]∈R r×nThe incremental matrix constructed according to the first matrix, the diagonal matrix and the second matrix can be expressed as W0=U r Σ r V r .

[0104] The model fine-tuning process is described below in conjunction with a specific embodiment. The incremental matrix for determining the matrix to be adjusted provided in the embodiment of the present application is applicable to each layer of the fine-tuning model.

[0105] Assume that the fine-tuning model has been pre-trained and the weight parameters are fixed as W∈R m×n , that is, the fixed weight parameter is a matrix W with m rows and n columns. When fine-tuning the model, the weight parameter W∈R m×n Determined as the matrix to be adjusted, after decomposing the matrix to be adjusted based on the singular value decomposition algorithm, the left singular value vector matrix U∈R m×m , the right singular value vector matrix V∈R n×n and the singular value matrix Σ∈R m×n According to the singular values ​​included in the singular value matrix, the target singular values ​​and the target number r are determined, where the target number is the low-rank parameter. After the low-rank parameter is determined, the initial low-rank matrix of the low-rank dimension is determined based on the low-rank parameter, that is, the incremental matrix is ​​determined. Specifically, the diagonal matrix Σ is constructed using the target singular values. r ∈R r×r , in the left singular value vector matrix U∈R m×m Select the column data of the first target number r from the above, and get the first matrix U r ∈R m×r , in the right singular value vector matrix V∈R n×n Select the row data of the previous target number r from the matrix, and get the second matrix V r ∈R r×n An incremental matrix is ​​constructed according to the determined first matrix, second matrix and diagonal matrix, and the incremental matrix is ​​added to the fine-tuning model to obtain a target model. Figure 2 A schematic diagram of the model fine-tuning training principle provided in the embodiment of the present application, such as Figure 2 As shown, Figure 2 It can be regarded as the fine-tuning training process involved in a certain layer of the target model. Figure 2 The bottom rectangle (upper and lower in the diagram) can be regarded as the m-dimensional feature vector x. The feature vector x is used as the input of this layer. This layer processes the feature vector x based on the pre-trained weights, i.e., the matrix to be adjusted W, to obtain x1, where the matrix to be adjusted can be expressed as W∈R m ×n, that is, the matrix to be adjusted is a matrix with m rows and n columns; while processing the eigenvector x based on the matrix to be adjusted, the layer also processes the eigenvector x based on the determined incremental matrix to obtain x2, where the incremental matrix is ​​based on the matrix V r ∈R r×n ,Σ r ∈R r×r and U r ∈R m×r Determined, the matrix V r ∈R r×n ,Σ r ∈R r×r and U r ∈R m×r is based on the decomposition matrix V∈R n×n ,Σ∈R m×n and U∈R m×m The decomposition matrix is ​​constructed by decomposing the pre-trained weight W based on the singular value decomposition algorithm. How to determine the incremental matrix based on the decomposition matrix has been described in detail in the above embodiments and will not be repeated here. After obtaining x1 and x2, x1 and x2 are fused to obtain Figure 2 The topmost strip matrix h (the top and bottom in the diagram) is the eigenvector after the fusion of x1 and x2.

[0106] Large-scale language model training mainly involves matrix operation parameters. The embodiment of the present application mainly proposes to achieve the purpose of efficient fine-tuning by adding an additional incremental matrix W0. The embodiment of the present application performs singular value decomposition on the matrix to be adjusted W, and uses the decomposed left singular value vector matrix, singular value matrix and right singular value vector matrix to construct the initialized incremental matrix W0, thereby achieving the purpose of efficient parameter fine-tuning.

[0107] Compared with the widely used LoRA technology, the embodiment of the present application selects a reasonable number of targets as the low-rank dimension from a theoretical level through singular value decomposition, reducing the difficulty of manually designing low-rank coefficients; at the same time, the decomposed matrix is ​​used to construct the incremental matrix, which contains more model knowledge than the random Gaussian initialization matrix, making the model easier to train and containing more domain information.

[0108] The model fine-tuning training process is described below in conjunction with another specific embodiment. Figure 3 A flow chart of another model fine-tuning training method provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, the following steps are included:

[0109] S301: Receive the storage location of the fine-tuning model and the matrix to be adjusted in the fine-tuning model.

[0110] S302: Decompose the matrix to be adjusted in the fine-tuning model based on a singular value decomposition algorithm to obtain a left singular value vector matrix, a singular value matrix, and a right singular value vector matrix.

[0111] S303: Determine a low-rank parameter, that is, a target number.

[0112] Specifically, the diagonal elements of the singular value matrix are sorted in order from large to small, the target sum values ​​of all singular values ​​are determined, and among the sorted singular values, the second sum value of the maximum singular value and at least one other adjacent singular value is determined one by one; and it is judged in turn whether the ratio of the second sum value to the target sum value is greater than a preset threshold value; when a second sum value whose ratio to the target sum value is greater than the preset threshold value is identified, the second sum value is used as the target sum value, the singular value of the target sum value is determined, the determined singular value is used as the target singular value, and the number of target singular values ​​is determined as the target number.

[0113] S304: Construct an initial increment matrix according to the determined target quantity.

[0114] Specifically, a diagonal matrix is ​​constructed using the target singular values; and the column data of the previous target number is selected from the left singular value vector matrix as the first matrix; and the row data of the previous target number is selected from the right singular value vector matrix as the second matrix; and the incremental matrix is ​​constructed using the first matrix, the diagonal matrix, and the second matrix. How to construct the incremental matrix has been described in detail in the above embodiments, and will not be repeated here.

[0115] S305: searching and acquiring the fine-tuning model at the storage location where the received fine-tuning model is stored, and adding the constructed incremental matrix to the fine-tuning model to obtain a target model, and performing fine-tuning training on the target model.

[0116] The technical solution protected by this application has the characteristics of reliability, generalizability, and robustness, and meets the trustworthy characteristics.

[0117] Figure 4 A schematic diagram of the structure of the model fine-tuning training device provided in the embodiment of the present application, such as Figure 4 As shown, the device comprises:

[0118] An acquisition module 401 is used to acquire a matrix to be adjusted in a fine-tuning model;

[0119] A decomposition determination module 402 is used to decompose the matrix to be adjusted based on a singular value decomposition algorithm to obtain multiple decomposition matrices; and determine the target quantity based on the multiple decomposition matrices;

[0120] A construction module 403 constructs an incremental matrix based on the target quantity and the multiple decomposition matrices;

[0121] The training module 404 is used to add the incremental matrix to the fine-tuning model to obtain a target model, and perform fine-tuning training on the target model.

[0122] In one possible implementation, the decomposition determination module 402 is specifically used to determine singular values ​​based on the multiple decomposition matrices, sort the singular values ​​in order of size, determine the minimum sum when the first sum of the maximum singular value and other adjacent singular values ​​is greater than a preset threshold, and include the target number of target singular values.

[0123] In a possible implementation, the decomposition determination module 402 is specifically used to determine the target sum of the singular values ​​included in the singular value matrix; among the sorted singular values, determine the second sum of the maximum singular value and at least one other adjacent singular value one by one; and determine in turn whether the ratio of the second sum to the target sum is greater than the preset threshold; when a second sum whose ratio to the target sum is greater than the preset threshold is identified, use the second sum as the target sum, determine the singular value of the target sum, use the determined singular value as the target singular value, and determine the number of the target singular values ​​as the target number.

[0124] In a possible implementation manner, the preset threshold is any value greater than 0 and less than 1.

[0125] In one possible implementation, the construction module 403 is specifically used to construct a diagonal matrix using the target number of singular values; in a first target matrix among the multiple decomposition matrices, select the target number of row and / or column data to obtain a second target matrix; and use the diagonal matrix and the second target matrix to construct an incremental matrix.

[0126] In a possible implementation, the acquisition module 401 is further used to acquire a first target matrix from the multiple decomposition matrices, wherein the first target matrix includes a left singular value vector matrix and a right singular value vector matrix;

[0127] The construction module 403 is specifically used to select the target number of column data in the left singular value vector matrix as the first matrix; and select the target number of row data in the right singular value vector matrix as the second matrix; and determine the first matrix and the second matrix as the second target matrix.

[0128] In a possible implementation, the construction module 403 is specifically configured to determine a matrix obtained by multiplying the first matrix, the diagonal matrix, and the second matrix as the incremental matrix.

[0129] Figure 5A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Based on the above embodiments, the present application further provides an electronic device, such as Figure 5 As shown, it includes: a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504;

[0130] The memory 503 stores a computer program. When the program is executed by the processor 501, the processor 501 performs the following steps:

[0131] Get the matrix to be adjusted in the fine-tuning model;

[0132] Decomposing the matrix to be adjusted based on a singular value decomposition algorithm to obtain a plurality of decomposition matrices; determining a target quantity based on the plurality of decomposition matrices;

[0133] An incremental matrix is ​​constructed based on the target quantity and the multiple decomposition matrices; the incremental matrix is ​​added to the fine-tuning model to obtain a target model, and fine-tuning training is performed on the target model.

[0134] In one possible implementation, the processor 501 is also used to determine singular values ​​based on the multiple decomposition matrices, sort the singular values ​​in order of size, determine the minimum sum when the first sum of the maximum singular value and other adjacent singular values ​​is greater than a preset threshold, and include a target number of target singular values.

[0135] In a possible implementation manner, the processor 501 is further configured to determine targets and values ​​of singular values ​​included in the singular value matrix;

[0136] Among the sorted singular values, determine the second sum of the maximum singular value and at least one other adjacent singular value one by one; and determine in turn whether the ratio of the second sum to the target sum is greater than the preset threshold;

[0137] When a second sum value is identified whose ratio to the target sum value is greater than the preset threshold, the second sum value is used as the target sum value, the singular value of the target sum value is determined, the determined singular value is used as the target singular value, and the number of the target singular values ​​is determined as the target number.

[0138] In a possible implementation, the processor 501 is further configured to construct a diagonal matrix using the target number of singular values;

[0139] In a first target matrix among the multiple decomposition matrices, select the target number of rows and / or columns of data to obtain a second target matrix;

[0140] A delta matrix is ​​constructed using the diagonal matrix and the second target matrix.

[0141] In a possible implementation, the processor 501 is further configured to obtain a first target matrix from the multiple decomposition matrices, wherein the first target matrix includes a left singular value vector matrix and a right singular value vector matrix;

[0142] The target number of column data is selected from the left singular value vector matrix as the first matrix; and the target number of row data is selected from the right singular value vector matrix as the second matrix; the first matrix and the second matrix are determined as the second target matrix.

[0143] In a possible implementation, the processor 501 is further configured to determine a matrix obtained by multiplying the first matrix, the diagonal matrix, and the second matrix as the incremental matrix.

[0144] On the basis of the above embodiments, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program executable by a processor, and when the program runs on the processor, the processor implements the following steps when executing:

[0145] Get the matrix to be adjusted in the fine-tuning model;

[0146] Decomposing the matrix to be adjusted based on a singular value decomposition algorithm to obtain a plurality of decomposition matrices; determining a target quantity based on the plurality of decomposition matrices;

[0147] An incremental matrix is ​​constructed based on the target quantity and the multiple decomposition matrices; the incremental matrix is ​​added to the fine-tuning model to obtain a target model, and fine-tuning training is performed on the target model.

[0148] In a possible implementation manner, determining the target quantity based on the multiple decomposition matrices includes:

[0149] Based on the multiple decomposition matrices, singular values ​​are determined, the singular values ​​are sorted in order of size, a minimum sum when a first sum of a maximum singular value and other adjacent singular values ​​is greater than a preset threshold is determined, and a target number of target singular values ​​is included.

[0150] In a possible implementation manner, the step of determining the minimum sum when the first sum of the maximum singular value and other adjacent singular values ​​is greater than a preset threshold value includes a target number of target singular values:

[0151] determining targets and values ​​of singular values ​​included in the singular value matrix;

[0152] Among the sorted singular values, determine the second sum of the maximum singular value and at least one other adjacent singular value one by one; and determine in turn whether the ratio of the second sum to the target sum is greater than the preset threshold;

[0153] When a second sum value is identified whose ratio to the target sum value is greater than the preset threshold, the second sum value is used as the target sum value, the singular value of the target sum value is determined, the determined singular value is used as the target singular value, and the number of the target singular values ​​is determined as the target number.

[0154] In a possible implementation manner, the preset threshold is any value greater than 0 and less than 1.

[0155] In a possible implementation manner, constructing an incremental matrix based on the target quantity and the multiple decomposition matrices includes:

[0156] constructing a diagonal matrix using the target number of singular values;

[0157] In a first target matrix among the multiple decomposition matrices, select the target number of rows and / or columns of data to obtain a second target matrix;

[0158] A delta matrix is ​​constructed using the diagonal matrix and the second target matrix.

[0159] In a possible implementation, selecting the target number of rows and / or columns of data from a first target matrix in the multiple decomposition matrices to obtain a second target matrix includes:

[0160] Obtaining a first target matrix from among the multiple decomposition matrices, wherein the first target matrix includes a left singular value vector matrix and a right singular value vector matrix;

[0161] The target number of column data is selected from the left singular value vector matrix as the first matrix; and the target number of row data is selected from the right singular value vector matrix as the second matrix; the first matrix and the second matrix are determined as the second target matrix.

[0162] In a possible implementation, constructing an incremental matrix using the diagonal matrix and the second target matrix includes:

[0163] A matrix obtained by multiplying the first matrix, the diagonal matrix and the second matrix is ​​determined as the incremental matrix.

[0164] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A model fine-tuning training method, characterized in that: The method comprises: Get the matrix to be adjusted in the fine-tuning model; Decomposing the matrix to be adjusted based on a singular value decomposition algorithm to obtain a plurality of decomposition matrices; determining a target quantity based on the plurality of decomposition matrices; An incremental matrix is ​​constructed based on the target quantity and the multiple decomposition matrices; the incremental matrix is ​​added to the fine-tuning model to obtain a target model, and fine-tuning training is performed on the target model.

2. The method according to claim 1, characterized in that The determining the target quantity based on the multiple decomposition matrices comprises: Based on the multiple decomposition matrices, singular values ​​are determined, the singular values ​​are sorted in order of size, a minimum sum when a first sum of a maximum singular value and other adjacent singular values ​​is greater than a preset threshold is determined, and a target number of target singular values ​​is included.

3. The method according to claim 2, characterized in that The method of determining the minimum sum when the first sum of the maximum singular value and other adjacent singular values ​​is greater than a preset threshold value includes the target number of target singular values: determining targets and values ​​of singular values ​​included in the singular value matrix; Among the sorted singular values, determine the second sum of the maximum singular value and at least one other adjacent singular value one by one; and determine in turn whether the ratio of the second sum to the target sum is greater than the preset threshold; When a second sum value is identified whose ratio to the target sum value is greater than the preset threshold, the second sum value is used as the target sum value, the singular value of the target sum value is determined, the determined singular value is used as the target singular value, and the number of the target singular values ​​is determined as the target number.

4. The method according to claim 3, characterized in that The preset threshold is any value greater than 0 and less than 1.

5. The method according to claim 1, characterized in that The constructing of an incremental matrix based on the target quantity and the multiple decomposition matrices comprises: constructing a diagonal matrix using the target number of singular values; In a first target matrix among the multiple decomposition matrices, select the target number of rows and / or columns of data to obtain a second target matrix; A delta matrix is ​​constructed using the diagonal matrix and the second target matrix.

6. The method according to claim 5, characterized in that The step of selecting the target number of rows and / or columns of data from a first target matrix in the plurality of decomposition matrices to obtain a second target matrix comprises: Obtaining a first target matrix from among the multiple decomposition matrices, wherein the first target matrix includes a left singular value vector matrix and a right singular value vector matrix; The target number of column data is selected from the left singular value vector matrix as the first matrix; and the target number of row data is selected from the right singular value vector matrix as the second matrix; the first matrix and the second matrix are determined as the second target matrix.

7. The method according to claim 6, characterized in that The constructing an incremental matrix using the diagonal matrix and the second target matrix comprises: A matrix obtained by multiplying the first matrix, the diagonal matrix and the second matrix is ​​determined as the incremental matrix.

8. A model fine-tuning training device, characterized in that: The device comprises: An acquisition module is used to obtain the matrix to be adjusted in the fine-tuning model; A decomposition determination module, used to decompose the matrix to be adjusted based on a singular value decomposition algorithm to obtain multiple decomposition matrices; and determine the target quantity based on the multiple decomposition matrices; A construction module, configured to construct an incremental matrix based on the target quantity and the plurality of decomposition matrices; A training module is used to add the incremental matrix to the fine-tuning model to obtain a target model, and perform fine-tuning training on the target model.

9. An electronic device, characterized in that: The electronic device includes at least a processor and a memory, and the processor is used to implement the steps of the model fine-tuning training method as described in any one of claims 1 to 7 when executing a computer program stored in the memory.

10. A computer-readable storage medium, characterized in that: It stores a computer program, which, when executed by a processor, implements the steps of the model fine-tuning training method as described in any one of claims 1 to 7.

Images