A text inference method, product, device and storage medium

Abstract

The application discloses a text reasoning method, product, equipment and storage medium, and relates to the technical field of computers, and comprises the following steps: through the orthogonal triangular decomposition mode and in combination with the judgment mechanism whether the pre-trained language model has been trained on the fine-tuning data set, if the pre-trained language model has been trained on the fine-tuning data set, the orthogonal triangular decomposition is performed on the to-be-fine-tuned parameter matrix, the triangular matrix obtained after the decomposition is fine-tuned and trained on the fine-tuning data set, and the fine-tuned language model is obtained; if the language model has not been trained on the fine-tuning data set, the orthogonal fine-tuning is performed on the language model starting from a preset unit matrix, the fine-tuned language model is obtained, and the text reasoning is performed by using the fine-tuned language model, through the above mode, the robustness of the language model to different data sets is improved, and the accuracy of the text reasoning result is improved, and meanwhile, the problem that the robustness of the traditional pre-trained language model parameter after fine-tuning is poor and the performance is reduced is solved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a text reasoning method, product, device and storage medium. Background Technology

[0002] Currently, to improve the accuracy of text reasoning tasks, a pre-training-fine-tuning approach is commonly used to generate LLM (Large Language Model) models suitable for specific reasoning tasks. However, fine-tuned models exhibit fragile robustness when faced with input distribution shifts. For example, when the test data contains slight adversarial perturbations, style transfer, or long-tailed distributions, the model often produces high-confidence incorrect predictions on tasks it was originally good at; furthermore, the fine-tuning process can easily amplify biases already present in the pre-training stage. Simultaneously, the downstream datasets upon which fine-tuning relies are often limited in size, making it difficult to provide sufficient distribution coverage for the model, resulting in a high risk of overfitting.

[0003] Currently, in order to improve the robustness of LLM models to unknown perturbations, concept drift, and potential biases while maintaining fine-tuning efficiency, an orthogonal fine-tuning method has been proposed. By orthogonally constraining the parameter matrix, the invariance of semantic association is guaranteed, which can effectively alleviate the overfitting problem. However, the above method does not fully consider the impact of orthogonal fine-tuning on the robustness of the model, and it is difficult to improve the inherent robustness weakness of the model, thereby reducing the inference accuracy of the inference task. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a text reasoning method, product, device, and storage medium that can improve the robustness of language models to different datasets and increase the accuracy of text reasoning results, while solving the problems of poor robustness and performance degradation after fine-tuning of parameters in traditional pre-trained language models. The specific solution is as follows:

[0005] Firstly, this application discloses a text reasoning method, including:

[0006] Identify the pre-trained target language model corresponding to the target text reasoning task, and obtain the fine-tuning dataset for fine-tuning the target language model;

[0007] Determine whether the target language model has been trained on the fine-tuning dataset;

[0008] If the target language model has been trained on the fine-tuning dataset, then determine the parameters to be fine-tuned in the target language model, generate the parameter matrix to be fine-tuned, and perform orthogonal triangular decomposition on the parameter matrix to be fine-tuned to obtain the orthogonal matrix and the triangular matrix.

[0009] The parameters in the triangular matrix are fine-tuned on the training set of the fine-tuning dataset to obtain the fine-tuned language model.

[0010] If the target language model has not been trained on the fine-tuning dataset, the target language model is orthogonally fine-tuned starting from the preset identity matrix to obtain the fine-tuned language model.

[0011] The fine-tuned language model is used to perform reasoning operations on the target text reasoning task to obtain the text reasoning result.

[0012] Secondly, this application also discloses a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-mentioned text reasoning methods.

[0013] Thirdly, this application discloses an electronic device, including a processor and a memory; wherein the processor executes a computer program stored in the memory to implement the aforementioned text reasoning method.

[0014] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned text reasoning method.

[0015] This application employs orthogonal triangular decomposition combined with a mechanism to determine whether the pre-trained language model has been trained on a fine-tuning dataset. If the pre-trained language model has been trained on the fine-tuning dataset, the parameter matrix to be fine-tuned is decomposed into orthogonal triangles, and the resulting triangular matrix is ​​then fine-tuned on the fine-tuning dataset to obtain the fine-tuned language model. If the language model has not been trained on the fine-tuning dataset, it is orthogonally fine-tuned starting from a preset identity matrix to obtain the fine-tuned language model. This approach allows for further fine-tuning of the pre-trained language model while maintaining orthogonality during the fine-tuning process. It also considers both cases of training and non-training on the fine-tuning dataset, thereby improving the robustness of the language model to different datasets and increasing the accuracy of text inference results. Furthermore, it addresses the issues of poor robustness and performance degradation after parameter fine-tuning in traditional pre-trained language models. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0017] Figure 1 This is a flowchart of a text reasoning method disclosed in this application;

[0018] Figure 2 This is a specific orthogonal triangular decomposition diagram disclosed in this application;

[0019] Figure 3 This application discloses a specific text reasoning flowchart;

[0020] Figure 4 This is a specific text reasoning effect diagram disclosed in this application;

[0021] Figure 5 This is a flowchart of a specific text reasoning method disclosed in this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0024] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] This application discloses a text reasoning method, see [link to relevant documentation]. Figure 1 As shown, the method includes:

[0026] Step S11: Determine the pre-trained target language model corresponding to the target text reasoning task, and obtain the fine-tuning dataset for fine-tuning the target language model.

[0027] In this embodiment, the pre-trained target language model (LLM) corresponding to the target text reasoning task to be performed (such as intelligent question answering task, geometric image understanding task, medical question answering task, factual question answering task, or logical reasoning task, etc.) is first determined, such as a multimodal language model. Specifically, a base model that has been trained on massive general data, such as BERT (Bidirectional Encoder Representations from Transformers, a pre-trained language model), GPT (Generative Pre-trained Transformer), ViT (Vision Transformer, a computer vision model based on Transformer architecture), LLaMA (Large Language Model Meta AI), etc., can be downloaded or loaded as the target language model, and all model parameters are retained as initial parameters. After obtaining the target language model O corresponding to the current target text reasoning task, a fine-tuning dataset D (such as a geometric image understanding dataset, historical common sense dataset, medical question answering dataset, code generation dataset, etc.) can be further obtained for fine-tuning the target language model O. The fine-tuning dataset D can be divided, such as into a training set, a test set, and a validation set.

[0028] Step S12: Determine whether the target language model has been trained on the fine-tuning dataset.

[0029] In this embodiment, after obtaining the fine-tuning dataset D for fine-tuning the target language model O, it is further possible to determine whether the target language model O has been trained on the fine-tuning dataset D by following a preset number of attacks (e.g., 9 rounds of attacks) and by using Membership Inference Attack (MIA).

[0030] In one specific implementation, determining whether a target language model has been trained on a fine-tuning dataset through a member reasoning attack can specifically include: synthesizing a sample set similar to the fine-tuning dataset to obtain a similar sample set; inputting the similar sample set and the fine-tuning dataset into the target language model for text reasoning, and calculating the perplexity of the similar sample set and the fine-tuning dataset respectively during the reasoning process to obtain similar perplexity values ​​and fine-tuning perplexity values; calculating the ratio of the fine-tuning perplexity value to the similar perplexity value to obtain a perplexity ratio, and determining whether the perplexity ratio is less than a preset perplexity threshold; if the perplexity ratio is less than the preset perplexity threshold, it is determined that the target language model has been trained on the fine-tuning dataset; if the perplexity ratio is not less than the preset perplexity threshold, it is determined that the target language model has not been trained on the fine-tuning dataset. In this embodiment, firstly, a similar sample set is synthesized based on the fine-tuning dataset D to obtain a similar sample set F (the similar sample set F is a sample dataset with the same task type as the fine-tuning dataset D). Then, the synthesized similar sample set F and the fine-tuning dataset D are input into the target language model O for text reasoning. During the reasoning process, the perplexity (PPL) of the similar sample set F and the fine-tuning dataset O is calculated respectively to obtain the corresponding similarity perplexity value PPL-F and the fine-tuning perplexity value PPL-D. Next, the ratio of the fine-tuning perplexity value PPL-D to the similarity perplexity value PPL-F is calculated to obtain the perplexity ratio. Then, it is determined whether the perplexity ratio is less than a preset perplexity threshold ε (e.g., 0.5). If the perplexity ratio is less than the preset perplexity threshold ε (e.g., 0.5), it is determined that the target language model O has been trained on the fine-tuning dataset D; otherwise (i.e., the perplexity ratio is not less than the preset perplexity threshold ε), it is determined that the target language model O has not been trained on the fine-tuning dataset D. It should be noted that perplexity is typically used to characterize the model's predictive ability on the true sequence. Its core idea is that the higher the model's prediction probability of the true sequence, the lower the perplexity, and the better the model. However, the perplexity used in this application characterizes the probability that the model has encountered a specified sample; this process can be called a member inference attack. By determining whether the language model has been trained on the fine-tuning dataset, a basis can be provided for selecting subsequent model fine-tuning strategies for different dataset scenarios. This ensures that the fine-tuned language model can achieve good inference performance not only on already trained datasets but also on unused datasets, thereby improving the model's robustness on those datasets. Furthermore, by synthesizing similar sample sets, sufficient distribution coverage can be provided to the model, reducing the risk of overfitting.

[0031] Specifically, the formula for calculating the perplexity value PPL is as follows:

[0032] ;

[0033] In the formula, It is the i-th word. It refers to all words preceding the i-th word (the context). It is the model's prediction in a given context. The probability is given by N, where N is the total number of words in the sample sequence.

[0034] Specifically, synthesizing a sample set similar to the fine-tuned dataset to obtain a similar sample set can include: inputting the fine-tuned dataset into an initial fallback language model for training to obtain a trained fallback language model; the initial fallback language model is a language model based on a fallback mechanism; and using the trained fallback language model to synthesize a sample set similar to the fine-tuned dataset to obtain a similar sample set. In this embodiment, the fine-tuned dataset D can first be input into an initial fallback language model (such as an N-gram fallback language model) for training to obtain a trained fallback language model N, and then the trained fallback language model N can be used to synthesize a sample set similar to the fine-tuned dataset D, i.e., a similar sample set. It is understood that since the N-gram fallback language model samples word by word, the probability of entire sentences being repeated is extremely low, thus making it suitable for synthesizing a similar sample set.

[0035] Step S13: If the target language model has been trained on the fine-tuning dataset, determine the parameters to be fine-tuned in the target language model, generate the parameter matrix to be fine-tuned, and perform orthogonal triangular decomposition on the parameter matrix to be fine-tuned to obtain the orthogonal matrix and the triangular matrix.

[0036] In this embodiment, if the target language model O has been trained on the fine-tuning dataset D, the parameters to be fine-tuned in the target language model are determined (which may be some or all of the parameters of the target language model O), thereby generating a parameter matrix A to be fine-tuned. Then, the parameter matrix A to be fine-tuned is decomposed into orthogonal triangular matrix (QR decomposition) to obtain the orthogonal matrix Q and the triangular matrix R. The methods of orthogonal triangular matrix (QR decomposition) include, but are not limited to, Gram-Schmid orthogonalization, Household transformation, and Givens transformation.

[0037] For details, see Figure 2 As shown, the parameter matrix to be fine-tuned Perform orthogonal trigonometric decomposition to obtain (If there are multiple matrices containing parameters to be fine-tuned, QR decomposition is performed separately for each matrix.) Here, m and n correspond to the number of rows and columns of the matrix, respectively. The specific process of QR decomposition is as follows:

[0038] First, the parameter matrix A to be fine-tuned is subjected to Schmitt orthogonalization:

[0039] Equation (1);

[0040] in, , Representing vectors and The inner product of.

[0041] Next, by transforming equation (1), we obtain:

[0042] Equation (2);

[0043] right After normalization, we get:

[0044] Equation (3);

[0045] Substituting equation (3) into equation (2), we get:

[0046] Equation (4);

[0047] Equation (4) can be written in matrix form as follows:

[0048] ;

[0049] in, It is called the Q matrix (i.e., an orthogonal matrix); It is called the R matrix (i.e., the triangular matrix).

[0050] It is understandable that any matrix composed of any linearly independent vector group can be decomposed into QR. For the parameter matrix A to be fine-tuned in the pre-trained language model, the vectors of matrix A can be considered to be linearly independent, so matrix A can be decomposed into QR.

[0051] Step S14: Fine-tune the parameters in the triangular matrix on the training set in the fine-tuning dataset to obtain the fine-tuned language model.

[0052] In this embodiment, after performing orthogonal triangular decomposition (QR decomposition) on the parameter matrix A to be fine-tuned to obtain the orthogonal matrix Q and the triangular matrix R, the parameters in the triangular matrix R can be fine-tuned on the training set in the fine-tuning dataset D to obtain the fine-tuned language model.

[0053] Specifically, the parameters in the triangular matrix are fine-tuned on the training set of the fine-tuning dataset to obtain the fine-tuned language model. This can include: adding constraints containing the triangular matrix to a preset loss function to obtain a target loss function; and then using the target loss function to fine-tune the parameters in the triangular matrix on the training set of the fine-tuning dataset to obtain the fine-tuned language model. In this embodiment, for scenarios where the model has been trained on the fine-tuning dataset D, constraints containing the triangular matrix R can be added to the preset loss function L (such as the cross-entropy loss function) to obtain the target loss function. ;For example, ,in, This is a new constraint added to the preset loss function L, where I is the identity matrix. Next, on the training set in the fine-tuning dataset D, the target loss function is applied... The parameters in the triangular matrix R are fine-tuned to obtain a fine-tuned language model for text reasoning.

[0054] The preset loss function L includes, but is not limited to, the cross-entropy loss function and the negative log-likelihood loss function (NLLLoss). These loss functions are calculated as follows: for a given token sequence (Tokens are generally words, sub-words, phrases, letters, numbers, etc., generated by a specific tokenizer) x = [x1, x2, …, x_T], the model obtains the next predicted distribution through autoregression from left to right. ,in It is the token corresponding to position t. This refers to all tokens before position t. The actual label for each position is the sequence number corresponding to the next token. The formula for calculating the single-sample loss is:

[0055] ;

[0056] Where t is the position of x, and T is the length of the entire sentence. The total loss L is the average of the losses L(θ) of all single samples on the training set.

[0057] Specifically, adding constraints containing a triangular matrix to the preset loss function yields the target loss function. This can include: subtracting corresponding elements of the triangular matrix from the preset identity matrix to obtain the target parameter matrix; calculating the product of the norm of the target parameter matrix and the preset constraint strength coefficient; and summing this product with the preset loss function as the target loss function. In this embodiment, the triangular matrix R can be subtracted from the corresponding elements of the preset identity matrix I first. Then, the norm corresponding to the obtained target parameter matrix is ​​calculated. The product of the preset constraint strength coefficient α and the product The sum of the loss function and the preset loss function is used as the target loss function for model fine-tuning. Specifically, it can be expressed as: By adding constraints to the initial loss function (i.e., the preset loss function) and training the model using the loss function with added constraints, the robustness of the model can be further improved, resulting in better inference performance.

[0058] Step S15: If the target language model has not been trained on the fine-tuning dataset, then orthogonally fine-tune the target language model starting from the preset identity matrix to obtain the fine-tuned language model.

[0059] In this embodiment, if the target language model O has not been trained on the fine-tuning dataset D, the target language model O can be orthogonally fine-tuned starting from the preset identity matrix I, thereby obtaining the fine-tuned language model.

[0060] Specifically, orthogonally fine-tuning the target language model starting from a preset identity matrix to obtain a fine-tuned language model can include: using the target loss function and based on the parameter matrix to be fine-tuned, orthogonally fine-tuning the parameters in the preset identity matrix to obtain a first fine-tuned matrix; and then fine-tuning and training the first fine-tuned matrix on the training set of the fine-tuning dataset to obtain the fine-tuned language model. That is, using the preset identity matrix I as the matrix to be optimized, and using the target loss function... The parameters in the preset identity matrix I are orthogonally fine-tuned to obtain the corresponding fine-tuned matrix M. It is important to note that M must remain orthogonal throughout the orthogonal fine-tuning process. Methods for orthogonal fine-tuning the preset identity matrix I include orthogonal matrix parameterization methods, such as Cayley parameterization (a mathematical transformation that maps antisymmetric matrices to orthogonal matrices) and matrix exponent parameterization. By constraining the orthogonality of the parameter matrices, not only is the invariance of semantic association guaranteed, but overfitting problems can also be effectively mitigated.

[0061] When using the Cayley parameterization method for orthogonal fine-tuning, the specific update formula for the fine-tuned matrix M is as follows:

[0062] ;

[0063] In the formula, It is the weight matrix after the k-th update. I is the identity matrix. Let A be the preset learning rate, A be the parameter matrix to be fine-tuned, and U be the skew matrix (i.e., the skew-symmetric matrix) containing gradient information, which can be represented as:

[0064] ;

[0065] Where G is the gradient matrix, and the element in the i-th row and j-th column of matrix G is... H represents Hermitian transpose.

[0066] Specifically, methods for obtaining gradient information include, but are not limited to, analytical gradients, numerical gradients, and automatic differentiation. Among them, analytical gradients directly calculate the partial derivatives of the function through mathematical derivation; numerical gradients approximate the gradient using finite differences; and automatic differentiation automatically calculates the gradient using a computer program by saving the forward calculation results of the model and combining them with previously determined basic operations and the chain rule.

[0067] When using the matrix exponential parameterization method for orthogonal fine-tuning, the specific update formula for the fine-tuned matrix M is as follows:

[0068] ;

[0069] Where A is the parameter matrix to be fine-tuned, N is the matrix exponential series (which can be a preset value, such as 18, in actual calculation), and B is the adjustable parameter matrix (initially the identity matrix I).

[0070] It should be noted that the fine-tuning training process on the training set of the fine-tuning dataset for the first fine-tuned matrix specifically includes: determining whether the trend of the loss value of the target loss function is less than a preset loss threshold, or determining whether the current training iterations have reached a preset number of epochs; if the trend of the loss value of the preset loss function is less than the preset loss threshold, or the current training iterations have reached a preset number of epochs, then the fine-tuning training stops. That is, during the fine-tuning training of the fine-tuned matrix on the training set of the fine-tuning dataset D, it is determined whether a preset convergence condition has been met (i.e., the trend of the loss function is less than a specified threshold δ, or the training has reached a specified number of epochs r). If the preset convergence condition is met, the fine-tuning training stops. By determining the convergence condition, model convergence can be guaranteed, thereby ensuring the model's inference performance.

[0071] Specifically, the complete fine-tuning training process can be divided into the following steps: First, construct the downstream task structure. This can be done by adding a "task header" to the top of the pre-trained language model, such as adding a linear classification layer for classification tasks or using the same structure as the original model for sequence-to-sequence tasks. Next, prepare the domain / task data. This can be obtained by collecting a small amount of labeled data (thousands to hundreds of thousands of records) consistent with the downstream task, thus obtaining the dataset D to be fine-tuned. Further, preprocessing operations consistent with the pre-training stage can be performed on dataset D, such as word segmentation and resizing. Operations such as e (size adjustment) and Normalize (normalization) are performed, and the preprocessed dataset is divided into training / validation / test sets (train / val / test). Further, a training strategy is set and the model is trained, including setting the learning rate (e.g., 1e-5~5e-4 to avoid destroying pre-trained features), batch size, optimizer (AdamW), warmup parameters, weight decay coefficients, and other hyperparameters. Finally, task metrics (loss function, accuracy, etc.) are monitored on the validation set, and a performance report is generated on the test set. Simultaneously, all parameters of the fine-tuned model are exported.

[0072] Step S16: Perform reasoning operations on the target text reasoning task based on the fine-tuned language model to obtain the text reasoning result.

[0073] In this embodiment, different fine-tuning strategies are adopted for two scenarios: the target language model O has been trained on the fine-tuning dataset D and the target language model O has not been trained on the fine-tuning dataset D. After fine-tuning, the fine-tuned language model can be used to perform text reasoning operations on the target text reasoning task (such as intelligent question answering). Furthermore, the obtained text reasoning results can be sent to the corresponding client.

[0074] In a specific embodiment, if the target language model has been trained on the fine-tuning dataset, then an inference operation is performed on the target text inference task based on the fine-tuned language model to obtain a text inference result, which may specifically include: calculating the accuracy of the fine-tuned language model on the validation set in the fine-tuning dataset to obtain an accuracy value; if the accuracy value is less than the preset accuracy threshold, then orthogonal fine-tuning is performed on the fine-tuned language model starting from the orthogonal matrix to obtain an updated language model; using the updated language model to perform an inference operation on the target text inference task to obtain a text inference result. In this embodiment, after fine-tuning the model, the accuracy of the fine-tuned language model can be further verified. Specifically, first calculate the accuracy of the fine-tuned language model on the validation set in the fine-tuning dataset D, and then determine whether the calculated accuracy value is less than the preset accuracy threshold (such as 60%). If it is less, then orthogonal fine-tuning is performed on the fine-tuned language model starting from the orthogonal matrix Q to obtain an updated language model, and the updated language model is used to perform a text inference operation on the target text inference task. By calculating the accuracy of the fine-tuned language model on the validation set and further optimizing and updating the model parameters when the accuracy does not meet the preset accuracy conditions, the accuracy of the model's text inference can be further improved.

[0075] Specifically, the formula for calculating the accuracy of the language model corresponding to the language generation task and the reasoning fact answering task is:

[0076] ;

[0077] In the formula, T is the length of the whole sentence, t is the position where the current prediction x is located, V represents the vocabulary, and the size of the set of all possible tokens is |V|; v represents the index of a certain token in the vocabulary V; represents that the model parameters are θ, and after obtaining the first t - 1 tokens (denoted as x_<t), the probability distribution given to the next token; represents changing the probability distribution to "the index of the most likely token"; represents the index of the token at the t-th position in the real text; represents the indicator function, which takes 1 when the condition holds and 0 otherwise. represents taking the average over all positions to obtain the token-level accuracy of this sequence.

[0078] Specifically, orthogonally fine-tuning the fine-tuned language model starting from the orthogonal matrix to obtain the updated language model can include: using the target loss function and based on the triangular matrix, orthogonally fine-tuning the parameters in the orthogonal matrix to obtain the second fine-tuned matrix; and fine-tuning the third fine-tuned matrix and the second fine-tuned matrix to obtain the updated language model; wherein the third fine-tuned matrix is ​​the matrix obtained after fine-tuning the triangular matrix using the target loss function. In this embodiment, for the scenario where the target language model O has been trained on the fine-tuning dataset D, if the accuracy value is less than a preset accuracy threshold (e.g., 60%), the Cayley parameterization method can be used to orthogonally fine-tune the parameters in the orthogonal matrix Q and update them using gradient descent. Then, the matrix obtained after fine-tuning the triangular matrix R and the matrix obtained after fine-tuning the orthogonal matrix Q are fine-tuned and trained until convergence (e.g., the trend of the loss function change is less than a specified threshold δ, or the current training number reaches a specified number of rounds r).

[0079] For scenarios where the target language model O has not been trained on the fine-tuning dataset D, it can be used directly. (r is the number of times the last parameter was updated) is used as the parameter matrix A' (i.e. model parameters) for the final text inference model.

[0080] Specifically, fine-tuning the third and second fine-tuned matrices to obtain the updated language model can include: obtaining the third fine-tuned matrix obtained by fine-tuning the triangular matrix using the target loss function; performing a dot product operation on the third and second fine-tuned matrices to merge them into a merged parameter matrix; and fine-tuning the merged parameter matrix on the training set in the fine-tuning dataset to obtain the updated language model. In this embodiment, the third fine-tuned matrix obtained by fine-tuning the triangular matrix using the target loss function is first obtained. The third fine-tuned matrix is ​​obtained after fine-tuning the triangular matrix R. Then, a dot product operation is performed on the third fine-tuned matrix and the second fine-tuned matrix (i.e., the matrix obtained after fine-tuning the orthogonal matrix Q), thereby merging the third and second fine-tuned matrices. Finally, fine-tuning training is performed on the training set in the fine-tuning dataset D. By merging the QR matrices as the new parameter matrix, inference efficiency can be guaranteed.

[0081] This application's embodiments employ orthogonal triangular decomposition combined with a mechanism to determine whether the pre-trained language model has been trained on a fine-tuning dataset. If the pre-trained language model has been trained on the fine-tuning dataset, the parameter matrix to be fine-tuned is subjected to orthogonal triangular decomposition, and the resulting triangular matrix is ​​then fine-tuned on the fine-tuning dataset to obtain the fine-tuned language model. If the language model has not been trained on the fine-tuning dataset, orthogonal fine-tuning is performed on the language model starting from a preset identity matrix to obtain the fine-tuned language model. This approach allows for further fine-tuning of the pre-trained language model while maintaining orthogonality during the fine-tuning process. It also considers both cases of model training on and not on the fine-tuning dataset, thereby improving the robustness of the language model to different datasets and increasing the accuracy of text inference results. Furthermore, it addresses the problems of poor robustness and performance degradation after parameter fine-tuning in traditional pre-trained language models.

[0082] For details, see Figure 3 As shown, first, the pre-trained original model O corresponding to the target text reasoning task is determined, and a fine-tuning dataset D is determined for fine-tuning the original model O. Next, a similar sample set F is synthesized based on the fine-tuning dataset D, and both the fine-tuning dataset D and the similar sample set F are input into the original model O for text reasoning operations. During the reasoning process, the perplexity of the similar sample set F and the fine-tuning dataset O are calculated respectively, obtaining the corresponding similarity perplexity value PPL-F and fine-tuning perplexity value PPL-D. The ratio of the fine-tuning perplexity value PPL-D to the similarity perplexity value PPL-F is calculated. If the ratio is less than a preset threshold ε, orthogonal triangular decomposition (i.e., QR decomposition) is performed to obtain an orthogonal matrix Q and a triangular matrix R, and the triangular matrix R is used as the matrix to be optimized, while setting a loss function. If the ratio is not less than the preset threshold ε, the identity matrix is ​​used as the matrix to be optimized and orthogonalized. Further, the matrix to be optimized is fine-tuned and trained on the fine-tuning dataset D until convergence. Furthermore, for the orthogonal triangular decomposition case, it is determined whether the accuracy is less than a preset target t (e.g., 60%). If it is, the orthogonal matrix Q is updated. Finally, the updated orthogonal matrix Q and triangular matrix R are merged, and the merged matrix is ​​used as the new parameter matrix, thus completing the entire parameter fine-tuning process. Finally, the fine-tuned model is used for text inference. By adopting different fine-tuning strategies, the robustness of the language model to different datasets can be improved, and the accuracy of the text inference results can be increased.

[0083] See Figure 4As shown, the original pre-trained model uses the Qwen-2.5VL 7B model, and the fine-tuning dataset uses the color_and_shape_disambiguation subset of the DoYou See Me dataset. After fine-tuning, the model can clearly distinguish potentially confusing color and shape descriptions in geometric patterns of visual images, thus achieving fine-tuning of a fact-oriented question-answering model. Robustness advantages are reflected by adversarial attacks, with an attack strength alpha=0.01 and a cutoff value esp=1.0. The x-axis represents the number of attack rounds, and the y-axis represents the model's accuracy on the DoYou See Me color_and_shape_disambiguation dataset (this dataset is unseen by the model, thus corresponding to the case where it was not trained on the fine-tuning dataset). Origin is the model after directly fine-tuning the A matrix using traditional methods, Cayley is the model after using Cayley parameterization, and EXP (matrix exponential) is the model after using matrix exponential parameterization. As can be seen from the figure, after 9 rounds of attacks, the accuracy of the traditional method for fine-tuning the model drops to below 20%, while the fine-tuning method of this scheme can keep the accuracy above 40%, with matrix exponential parameterization keeping it above 80%.

[0084] This application discloses a specific text reasoning method in its embodiments. See [link to relevant documentation]. Figure 5 As shown, the method includes:

[0085] Step S21: Determine the pre-trained target language model corresponding to the target text reasoning task, and obtain the fine-tuning dataset for fine-tuning the target language model.

[0086] Step S22: Determine whether the target language model has been trained on the fine-tuning dataset.

[0087] Step S23: If the target language model has been trained on the fine-tuning dataset, determine the parameters to be fine-tuned in the target language model, generate the parameter matrix to be fine-tuned, and perform orthogonal triangular decomposition on the parameter matrix to be fine-tuned to obtain the orthogonal matrix and the triangular matrix.

[0088] Step S24: Fine-tune the parameters in the triangular matrix on the training set in the fine-tuning dataset to obtain the fine-tuned language model.

[0089] Step S25: If the target language model has not been trained on the fine-tuning dataset, then orthogonally fine-tune the target language model starting from the preset identity matrix to obtain the fine-tuned language model.

[0090] Step S26: Evaluate the performance degradation of the fine-tuned language model under various noise intensities and / or various adversarial attack intensities to obtain a robustness score.

[0091] In this embodiment, the performance degradation of the fine-tuned language model O can be evaluated and represented by a score under various noise intensities and / or various adversarial attack intensities to obtain the corresponding robustness score.

[0092] Specifically, the robustness of the model under noise and / or adversarial attacks can be evaluated using various text quality assessment metrics such as Bertscore, BLEU, METEOR, and CIDER. Furthermore, corresponding weight coefficients can be assigned to each metric, and the product of each text quality assessment metric and its corresponding weight coefficient can be calculated. The sum of the products of all text quality assessment metrics is then used as the final robustness score. By using this weighted mixed scoring method to evaluate the performance degradation of the fine-tuned language model, a more accurate robustness score can be obtained, providing a basis for model optimization, resulting in a more robust model and improved inference performance.

[0093] Step S27: Determine whether the robustness score is greater than the preset score. If the robustness score is greater than the preset score, then use the fine-tuned language model to perform reasoning operations on the target text reasoning task to obtain the text reasoning result.

[0094] In this embodiment, after obtaining the robustness score, it is determined whether the robustness score is greater than the preset score. If it is greater, it means that the robustness of the fine-tuned model is good and can be applied to actual text reasoning tasks. At this time, the fine-tuned language model can be directly used to perform reasoning operations on the target text reasoning task.

[0095] If the robustness score is not greater than the preset score, it means that the robustness of the fine-tuned model is poor. At this time, the fine-tuned model can continue to be fine-tuned until it is greater than the preset score. If it is still not greater than the preset score after multiple fine-tunings, the fine-tuning can be ended and the corresponding alarm information can be generated.

[0096] For more detailed processing procedures of steps S21 to S25, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0097] As can be seen, the embodiments of this application, by fine-tuning the language model, not only retain the massive general knowledge learned in the pre-training stage, but also achieve accuracy close to or even exceeding that of full-parameter training in specific text reasoning tasks. Furthermore, by evaluating the performance degradation of the fine-tuned language model under various noise intensities and / or various adversarial attack intensities, the robustness of the model can be determined, thus enabling the model to achieve both high accuracy and robustness. This solves the technical problem of poor robustness and performance degradation after parameter fine-tuning of traditional pre-trained models.

[0098] Accordingly, embodiments of this application also disclose a text reasoning device, which includes:

[0099] The determination module is used to determine the pre-trained target language model corresponding to the target text reasoning task.

[0100] The dataset acquisition module is used to acquire the fine-tuning dataset for fine-tuning the target language model;

[0101] The judgment module is used to determine whether the target language model has been trained on the fine-tuning dataset;

[0102] The orthogonal triangular decomposition module is used to determine the parameters to be fine-tuned in the target language model if the target language model has been trained on the fine-tuning dataset, to generate the parameter matrix to be fine-tuned, and to perform orthogonal triangular decomposition on the parameter matrix to be fine-tuned to obtain the orthogonal matrix and the triangular matrix.

[0103] The fine-tuning training module is used to fine-tune the parameters in the triangular matrix on the training set in the fine-tuning dataset to obtain the fine-tuned language model.

[0104] The orthogonal fine-tuning module is used to orthogonally fine-tune the target language model starting from a preset identity matrix if the target language model has not been trained on the fine-tuning dataset, so as to obtain the fine-tuned language model.

[0105] The text reasoning module is used to perform reasoning operations on the target text reasoning task based on the fine-tuned language model and obtain the text reasoning result.

[0106] The specific workflow of each of the above modules can be found in the relevant content disclosed in the foregoing embodiments, and will not be repeated here.

[0107] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above-described text reasoning method embodiments.

[0108] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described text reasoning method embodiments at runtime.

[0109] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0110] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described text reasoning method embodiments.

[0111] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described text reasoning method embodiments.

[0112] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0113] The foregoing has provided a detailed description of a text reasoning method, product, device, and storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A text-based reasoning method, characterized in that, include: Determine the pre-trained target language model corresponding to the target text reasoning task, and obtain a fine-tuning dataset for fine-tuning the target language model; The fine-tuning dataset is a geometric image understanding dataset, a historical common sense dataset, a medical question-and-answer dataset, or a code generation dataset; Determine whether the target language model has been trained on the fine-tuning dataset; If the target language model has been trained on the fine-tuning dataset, then the parameters to be fine-tuned in the target language model are determined to generate a parameter matrix to be fine-tuned, and the parameter matrix to be fine-tuned is decomposed into orthogonal triangular matrices to obtain orthogonal matrices and triangular matrices. The parameters in the triangular matrix are fine-tuned and trained on the training set in the fine-tuning dataset to obtain the fine-tuned language model; If the target language model has not been trained on the fine-tuning dataset, then the target language model is orthogonally fine-tuned starting from the preset identity matrix to obtain the fine-tuned language model. Based on the fine-tuned language model, inference operations are performed on the target text inference task to obtain text inference results; The step of fine-tuning the parameters in the triangular matrix on the training set in the fine-tuning dataset to obtain a fine-tuned language model includes: adding constraints containing the triangular matrix to a preset loss function to obtain a target loss function; and fine-tuning the parameters in the triangular matrix on the training set in the fine-tuning dataset using the target loss function to obtain a fine-tuned language model. The step of orthogonally fine-tuning the target language model starting from a preset identity matrix to obtain a fine-tuned language model includes: using the target loss function and based on the parameter matrix to be fine-tuned, orthogonally fine-tuning the parameters in the preset identity matrix to obtain a first fine-tuned matrix; and fine-tuning training the first fine-tuned matrix on the training set in the fine-tuning dataset to obtain a fine-tuned language model.

2. The text reasoning method according to claim 1, characterized in that, The step of determining whether the target language model has been trained on the fine-tuning dataset includes: Based on a preset number of attacks and through member inference attacks, determine whether the target language model has been trained on the fine-tuning dataset.

3. The text reasoning method according to claim 2, characterized in that, The method of determining whether the target language model has been trained on the fine-tuning dataset through member reasoning attacks includes: A sample set similar to the fine-tuned dataset is synthesized to obtain a similar sample set; The similar sample set and the fine-tuned dataset are input into the target language model for text reasoning, and the perplexity of the similar sample set and the fine-tuned dataset are calculated respectively during the reasoning process to obtain the similarity perplexity value and the fine-tuning perplexity value. Calculate the ratio of the fine-tuned perplexity value to the similar perplexity value to obtain the perplexity ratio, and determine whether the perplexity ratio is less than a preset perplexity threshold; If the perplexity ratio is less than the preset perplexity threshold, then the target language model is determined to have been trained on the fine-tuning dataset; If the perplexity ratio is not less than the preset perplexity threshold, then it is determined that the target language model has not been trained on the fine-tuning dataset.

4. The text reasoning method according to claim 3, characterized in that, The synthesis of a sample set similar to the fine-tuned dataset yields a similar sample set, including: The fine-tuned dataset is input into the initial fallback language model for training to obtain the trained fallback language model; the initial fallback language model is a language model based on the fallback mechanism. The similar sample set is obtained by synthesizing a sample set similar to the fine-tuned dataset using the trained backtracking language model.

5. The text reasoning method according to claim 1, characterized in that, The step of adding constraints containing the triangular matrix to the preset loss function to obtain the target loss function includes: Subtracting the corresponding elements of the triangular matrix from the preset identity matrix yields the target parameter matrix; Calculate the product of the norm corresponding to the target parameter matrix and the preset constraint strength coefficient, and use the sum of the product and the preset loss function as the target loss function.

6. The text reasoning method according to claim 5, characterized in that, The process of fine-tuning the first fine-tuned matrix on the training set in the fine-tuning dataset further includes: Determine whether the trend of the loss value of the target loss function is less than a preset loss threshold, or determine whether the current training iterations have reached a preset number of rounds; If the trend of the loss value of the preset loss function is less than the preset loss threshold, or if the current training iterations reach the preset number of rounds, then fine-tuning training will stop.

7. The text reasoning method according to claim 1, characterized in that, If the target language model has been trained on the fine-tuned dataset, then the inference operation based on the fine-tuned language model for the target text inference task to obtain the text inference result includes: Calculate the accuracy of the fine-tuned language model on the validation set in the fine-tuned dataset to obtain the accuracy value; If the accuracy value is less than the preset accuracy threshold, the fine-tuned language model is orthogonally fine-tuned starting from the orthogonal matrix to obtain the updated language model. The updated language model is used to perform reasoning operations on the target text reasoning task to obtain the text reasoning result.

8. The text reasoning method according to claim 7, characterized in that, The step of orthogonally fine-tuning the fine-tuned language model using the orthogonal matrix as a starting point to obtain the updated language model includes: Using the target loss function and based on the triangular matrix, the parameters in the orthogonal matrix are orthogonally fine-tuned to obtain the second fine-tuned matrix; The third and second fine-tuned matrices are fine-tuned and trained to obtain the updated language model. The third fine-tuned matrix is ​​the matrix obtained by fine-tuning the triangular matrix using the target loss function.

9. The text reasoning method according to claim 8, characterized in that, The step of fine-tuning and training the third and second fine-tuned matrices to obtain the updated language model includes: Obtain the third fine-tuned matrix obtained after fine-tuning the triangular matrix using the target loss function; Perform a dot product operation on the third fine-tuned matrix and the second fine-tuned matrix to merge the third fine-tuned matrix and the second fine-tuned matrix to obtain a merged parameter matrix; On the training set of the fine-tuning dataset, the merged parameter matrix is ​​fine-tuned to obtain the updated language model.

10. The text reasoning method according to any one of claims 1 to 9, characterized in that, The process of performing reasoning operations on the target text reasoning task based on the fine-tuned language model to obtain text reasoning results includes: The performance degradation of the fine-tuned language model is evaluated under various noise intensities and / or various adversarial attack intensities to obtain a robustness score. Determine whether the robustness score is greater than a preset score. If the robustness score is greater than the preset score, then use the fine-tuned language model to perform reasoning operations on the target text reasoning task to obtain the text reasoning result.

11. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the text reasoning method as described in any one of claims 1 to 10 when executing the computer program.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the text reasoning method as described in any one of claims 1 to 10.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the text reasoning method as described in any one of claims 1 to 10.

Images