Language model training method, device, equipment and storage medium

Abstract

The present application provides a language model training method, apparatus, device, and storage medium, the method comprising: determining a first matrix, a second matrix, and a third matrix based on a pre-trained matrix of the language model; determining a first scaling vector for the second matrix and a second scaling vector for the third matrix; determining the second matrix and the second scaling vector as a target training matrix in response to the norm of the second matrix being greater than or equal to the norm of the third matrix; obtaining a training text and an actual response result of the training text; inputting the training text into the target training matrix to obtain a predicted response result; determining a result error based on the predicted response result and the actual response result; and training the target training matrix based on the result error to obtain an optimized language model. The present application reduces the training parameters of the language model through a singular value decomposition algorithm, thereby reducing the complexity and computational cost of the model.

Description

Technical Field

[0001] The present application relates to the field of natural language processing, and in particular to a method, apparatus, device, and storage medium for training a language model. Background Art

[0002] With the rapid development of artificial intelligence technology, language models have become the core force in the field of natural language processing. For example, in machine translation, text summarization, sentiment analysis, and question-answering systems, these models have powerful text generation, understanding, and reasoning capabilities through deep learning and training with massive amounts of text data.

[0003] However, as the size of language models continues to expand, the number of their parameters also grows exponentially, which not only places higher demands on storage space but also requires expensive computing resources for the training process. Summary of the Invention

[0004] In view of this, the purpose of this application is to propose a language model training method, apparatus, device and storage medium.

[0005] As one aspect of the present application, a method for training a language model is provided, comprising:

[0006] Determine a first matrix, a second matrix, and a third matrix according to the pre-training matrix of the language model; the first matrix is ​​used to represent the high-rank features of the pre-training matrix; the second matrix and the third matrix are used to represent the low-rank features of the pre-training matrix;

[0007] determining a first scaling vector of the second matrix and a second scaling vector of the third matrix;

[0008] In response to a norm of the second matrix being greater than or equal to a norm of the third matrix, determining the second matrix and the second scaling vector as a target training matrix;

[0009] Obtaining training text and actual response results of the training text;

[0010] Inputting the training text into the target training matrix to obtain a predicted answer result;

[0011] determining a result error based on the predicted response result and the actual response result;

[0012] Based on the result error, the target training matrix is ​​trained to obtain the optimized language model.

[0013] Optionally, in response to the norm of the second matrix being smaller than the norm of the third matrix, the third matrix and the first scaling vector are determined as the target training parameters.

[0014] Optionally, the norm is a two-norm.

[0015] Optionally, determining the first matrix, the second matrix, and the third matrix according to the pre-trained matrix of the language model includes:

[0016] Obtaining a pre-trained matrix of the language model;

[0017] Determining a low-rank matrix set of the pre-trained matrix according to a preset rank value set;

[0018] According to an iterative optimization algorithm, sample data is input into the low-rank matrix set to determine a target low-rank matrix; the sample data is a set of data randomly extracted from the training data set of the language model; the sample data is used to characterize the feature distribution of the training data set;

[0019] Determining a target rank value according to the target low-rank matrix;

[0020] Based on the target rank value, singular value decomposition is performed on the pre-training matrix to obtain the first matrix, the second matrix, and the third matrix.

[0021] Optionally, before determining the first matrix, the second matrix, and the third matrix according to the pre-trained matrix of the language model, the method further includes:

[0022] Configure the hyperparameters of the language model.

[0023] Optionally, configuring the hyperparameters of the language model includes:

[0024] Determining a hyperparameter set; the hyperparameter set includes at least one set of hyperparameter combinations;

[0025] Determining a target hyperparameter combination from the hyperparameter set according to a grid search method;

[0026] The language model is configured according to the target hyperparameter combination.

[0027] Optionally, determining a set of hyperparameters includes:

[0028] Inputting the hyperparameter combination into an initial model to obtain the language model;

[0029] Inputting sample data into the language model to obtain a sample result; the sample data is a set of data randomly extracted from the training data set of the language model;

[0030] In response to the sample results satisfying a preset evaluation indicator and a preset verification mechanism, the hyperparameter set is determined.

[0031] As a second aspect of the present application, a language model training device is provided, comprising: a determination module, an acquisition module, a processing module, and a training module;

[0032] The determination module is used to determine a first matrix, a second matrix, and a third matrix according to the pre-training matrix of the language model; the first matrix is ​​used to represent the high-rank features of the pre-training matrix; the second matrix and the third matrix are used to represent the low-rank features of the pre-training matrix;

[0033] The determining module is further configured to determine a first scaling vector of the second matrix and a second scaling vector of the third matrix;

[0034] The determining module is further configured to, in response to a norm of the second matrix being greater than or equal to a norm of the third matrix, determine that the second matrix and the second scaling vector are target training matrices;

[0035] The acquisition module is used to acquire the training text and the actual response result of the training text;

[0036] The processing module is used to input the training text into the target training matrix to obtain a predicted answer result;

[0037] The determining module is further configured to determine a result error based on the predicted response result and the actual response result;

[0038] The training module is used to train the target training matrix based on the result error to obtain the optimized language model.

[0039] Optionally, the determination module is further configured to, in response to a norm of the second matrix being smaller than a norm of the third matrix, determine the third matrix and the first scaling vector as the target training parameters.

[0040] Optionally, the norm is a two-norm.

[0041] Optionally, the determination module is specifically used to obtain the pre-training matrix of the language model; determine the low-rank matrix set of the pre-training matrix based on a preset rank value set; input sample data into the low-rank matrix set according to an iterative optimization algorithm to determine the target low-rank matrix; the sample data is a set of data randomly selected from the training data set of the language model; the sample data is used to characterize the feature distribution of the training data set; determine the target rank value based on the target low-rank matrix; based on the target rank value, perform singular value decomposition on the pre-training matrix to obtain the first matrix, the second matrix and the third matrix.

[0042] Optionally, the language model training device further includes a configuration module.

[0043] The configuration module is used to configure the hyperparameters of the language model.

[0044] Optionally, the configuration module is specifically used to determine a hyperparameter set; the hyperparameter set includes at least one group of hyperparameter combinations; according to a grid search method, determine a target hyperparameter combination in the hyperparameter set; and configure the language model according to the target hyperparameter combination.

[0045] Optionally, the determination module is further specifically used to input the hyperparameter combination into the initial model to obtain the language model; input sample data into the language model to obtain a sample result; the sample data is a set of data randomly selected from the training data set of the language model; and determine the hyperparameter set in response to the sample result satisfying the preset evaluation indicators and the preset verification mechanism.

[0046] As a third aspect of the present application, an electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned language model training method when executing the program.

[0047] As a fourth aspect of the present application, a non-transitory computer-readable storage medium is provided, which stores computer instructions, and the computer instructions are used to enable the computer to execute the above-mentioned language model training method provided in the present application.

[0048] From the above, it can be seen that the training method, apparatus, device and storage medium of the language model provided in the present application decompose the pre-training matrix of the language model into a first matrix of high-rank features and a second matrix and a third matrix of low-rank features through a singular value decomposition algorithm, which helps to retain the key features in the model while significantly reducing the number of parameters that need to be optimized in the model. Furthermore, by determining the scaling vectors of the second matrix and the third matrix and selecting the target training matrix according to their norm, flexible adjustment of the low-rank feature representation is achieved. This dynamic adjustment mechanism can flexibly optimize the training parameters of the model for different situations, focusing on the matrix with a greater impact on the model, that is, the matrix with a larger norm, thereby improving the optimization accuracy. At the same time, by reducing unnecessary training parameter updates, further, by obtaining the training text and its actual response results, and inputting the training text into the target training matrix to obtain the predicted response results, the difference between the predicted results and the actual results can be directly compared. This precise training feedback mechanism helps to accurately evaluate the performance of the model and provides strong support for further optimization of the model. Based on the error between the predicted response results and the actual response results, the language model is optimized by training the target training matrix, thereby improving the accuracy and generalization ability of the model in processing text data and reducing the complexity and computational cost of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in this application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are merely embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0050] Figure 1 A flowchart of a language model training method provided in an embodiment of the present application;

[0051] Figure 2 A flowchart of another language model training method provided in an embodiment of the present application;

[0052] Figure 3 A flowchart of another language model training method provided in an embodiment of the present application;

[0053] Figure 4 A flowchart of another language model training method provided in an embodiment of the present application;

[0054] Figure 5 A flowchart of another language model training method provided in an embodiment of the present application;

[0055] Figure 6A schematic diagram of the composition of a language model training device provided in an embodiment of the present application;

[0056] Figure 7 A schematic diagram of the composition of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] To make the objectives, technical solutions, and advantages of this application more clearly understood, the following describes this application in further detail with reference to specific embodiments and the accompanying drawings. It is apparent that the embodiments described are only a portion of the embodiments of this application, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments in this application without creative effort are intended to fall within the scope of protection of this application.

[0058] It should be noted that, in the embodiments of the present application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or design. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a concrete way. Unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should have the usual meaning understood by people with ordinary skills in the field to which this application belongs. SUMMARY OF THE INVENTION

[0060] The widespread adoption of language models has led to a significant increase in model size and complexity. These models, with their massive parameter count, have demonstrated unprecedented performance advantages across a wide range of tasks. However, this has also led to a surge in storage requirements and significant consumption of computing resources. While full fine-tuning strategies—adjusting all model parameters to suit specific tasks—maximize the model's potential, their high cost makes practical application challenging. This situation is not only challenging in academic research but also a significant obstacle in industrial applications.

[0061] The inventors of this application found that in current technical practices, although methods such as adapters, prompt tuning, and low-rank adaptation (LoRA) can reduce the amount of parameter adjustment, during the fine-tuning process, the selection of optimal hyperparameters is usually significantly different from the hyperparameters used in full fine-tuning. The main reason for this difference is the different number of trainable parameters. For example, in efficient parameter fine-tuning, in order to adapt to fewer trainable parameters, the optimal learning rate often needs to be set higher than in full fine-tuning.

[0062] In order to solve the above problems, the present application provides a training method for a language model. Through the singular value decomposition algorithm, the pre-training matrix of the language model is decomposed into a first matrix of high-rank features and a second matrix and a third matrix of low-rank features, which helps to retain the key features in the model while significantly reducing the number of parameters that need to be optimized in the model. Furthermore, by determining the scaling vectors of the second matrix and the third matrix and selecting the target training matrix according to their norm size, flexible adjustment of the low-rank feature representation is achieved. This dynamic adjustment mechanism can flexibly optimize the training parameters of the model for different situations, focusing on the matrix with a greater impact on the model, that is, the matrix with a larger norm, thereby improving the optimization accuracy. At the same time, by reducing unnecessary training parameter updates, further, by obtaining the training text and its actual response results, and inputting the training text into the target training matrix to obtain the predicted response results, the difference between the predicted results and the actual results can be directly compared. This precise training feedback mechanism helps to accurately evaluate the performance of the model and provides strong support for further optimization of the model. Based on the error between the predicted response results and the actual response results, the language model is optimized by training the target training matrix, thereby improving the accuracy and generalization ability of the model in processing text data and reducing the complexity and computational cost of the model.

[0063] After introducing the basic principles of the present application, various non-limiting implementation methods of the present application are described in detail below.

[0064] Figure 1 A flow chart of a language model training method provided in an embodiment of the present application. Figure 1 As shown, the language model training method provided in this application specifically includes the following steps:

[0065] S101. Determine a first matrix, a second matrix, and a third matrix according to a pre-trained matrix of a language model.

[0066] Among them, the first matrix is ​​used to represent the high-rank features of the pre-training matrix; the second matrix and the third matrix are used to represent the low-rank features of the pre-training matrix.

[0067] In some embodiments, the pre-training matrix is ​​a weight matrix obtained by pre-training a large amount of text data during the language model training process, which contains the model's understanding and generation capabilities of language. The pre-training matrix can be decomposed into a part with the same shape as the pre-training matrix, namely, a high-rank feature matrix (first matrix), and a low-rank part that captures the high variance direction, namely, a low-rank feature matrix (second and third matrices) using singular value decomposition (SVD).

[0068] It should be noted that singular value decomposition is an algorithm widely used in data science and machine learning. It can decompose a matrix into three specific matrix products: an orthogonal matrix, a diagonal matrix (the elements on the diagonal are called singular values, arranged from largest to smallest), and the transpose of another orthogonal matrix. The decomposition process satisfies the following expression:

[0069] U·diag(S)·V T =SVD(W);

[0070]

[0071]

[0072]

[0073] W=W res +AB;

[0074] Where W∈R m×n , U∈R m×min(m,n) ,V∈R n×min(m,n) , s∈R min(m,n) ,A∈R m×r , B∈R r×n , W is the pre-training matrix, U is the orthogonal matrix containing the left singular vector, V T is the transpose of the orthogonal matrix containing the right singular vectors, W res is the first matrix, A is the second matrix, B is the third matrix, and r is the rank of the pre-training matrix.

[0075] It should be understood that the first matrix is ​​also a residual matrix, which retains important and informative features in the pre-trained matrix and will be frozen during the training process of this method.

[0076] In some embodiments, as Figure 2 As shown, S101 can be specifically implemented as follows S1011-S1015:

[0077] S1011. Obtain a pre-trained matrix of the language model.

[0078] In some embodiments, the language model is trained based on a large amount of text data, which is converted into a matrix form. The pre-trained matrix refers to the parameter matrix of the language model pre-trained on large-scale data. This matrix contains the patterns, structures, and relationships learned by the language model.

[0079] S1012. Determine a low-rank matrix set of the pre-training matrix according to a preset rank value set.

[0080] In some embodiments, based on the dimension of the pre-trained matrix, such as 4096, a low-rank set is selected. For example, the low-rank set is rank_set = {4, 8, 16, 32, 64, 128}. Then, matrix decomposition is performed on each preset rank value through methods such as singular value decomposition or truncated singular value decomposition to obtain the corresponding low-rank matrix. These low-rank matrices will be used in subsequent iterative optimization processes.

[0081] S1013. According to the iterative optimization algorithm, the sample data is input into a low-rank matrix set to determine the target low-rank matrix.

[0082] The sample data is a set of data randomly extracted from the training data set of the language model; the sample data is used to characterize the feature distribution of the training data set.

[0083] In some embodiments, the sample data is encoded into a vector form that the model can process. These vectors are then used as input to calculate the output of each low-rank matrix through forward propagation. Next, the loss function is calculated based on the output and the actual label (or target value), and the parameters of the low-rank matrix are updated using the backpropagation algorithm. After multiple iterations, the low-rank matrix with the smallest loss is selected as the target low-rank matrix. In this way, the rank value that can maximize the performance of the model is determined for the weight decomposition of the pre-trained model.

[0084] S1014. Determine a target rank value according to the target low-rank matrix.

[0085] In some embodiments, a corresponding rank value is determined from a target low-rank matrix, and this target rank value is the rank value with the best evaluation effect in a preset rank value set.

[0086] S1015. Based on the target rank value, perform singular value decomposition on the pre-training matrix to obtain a first matrix, a second matrix, and a third matrix.

[0087] In some embodiments, a singular value decomposition is performed on the pre-trained matrix using a determined target rank value. During the decomposition process, an orthogonal matrix U matrix, a diagonal matrix, and the transpose of another orthogonal matrix V matrix are obtained. Then, the first n singular values ​​in the diagonal matrix are selected based on the target rank value, where n is the target rank value, and the corresponding rows and columns of the U matrix and the V matrix, thereby obtaining a first matrix representing high-rank features and a second and third matrices representing low-rank features.

[0088] In some embodiments, as Figure 3 As shown, before S101, the language model training method provided in the embodiment of the present application further includes the following S100:

[0089] S100. Configure the hyperparameters of the language model.

[0090] In some embodiments, hyperparameters are set before model training based on the language model and training objectives. These hyperparameters directly affect the performance and generalization ability of the model, such as the learning rate, optimizer, number of iterations, number of hidden layer units, weight decay, batch size, rank, etc. For each hyperparameter, an initial value can be set, and then these values ​​can be optimized through subsequent adjustments.

[0091] In some embodiments, as Figure 4 As shown, S100 can be specifically implemented as follows S1001-S1003:

[0092] S1001. Determine a set of hyperparameters.

[0093] The hyperparameter set includes at least one set of hyperparameter combinations.

[0094] In some embodiments, based on professional knowledge, experiments, or literature research, multiple candidate values ​​are selected for each hyperparameter and combined into different hyperparameter combinations, and all hyperparameter combinations are aggregated to obtain a hyperparameter set.

[0095] In some embodiments, as Figure 5 As shown, S1001 can be specifically implemented as follows S1001a-S1001c:

[0096] S1001a. Input the hyperparameter combination into the initial model to obtain a language model.

[0097] In some embodiments, a suitable language model is selected as the initial model based on task requirements such as text classification or language translation, and a preset set of hyperparameter combinations, such as learning rate, batch size, number of model layers, etc., are applied to this model. The selection of these hyperparameters is based on prior knowledge or preliminary experiments on the impact of model performance. By adjusting these hyperparameters, language model instances with different performance characteristics can be obtained.

[0098] S1001b. Input the sample data into the language model to obtain a sample result.

[0099] The sample data is a set of data randomly extracted from the training data set of the language model.

[0100] In some embodiments, a set of sample data is randomly extracted from the training data set of the language model and input into the language model as a test. The language model predicts the sample data to obtain a sample result.

[0101] S1001c. In response to the sample results satisfying the preset evaluation indicators and the preset verification mechanism, determine a set of hyperparameters.

[0102] In some embodiments, evaluation metrics are set based on the task type. For example, for classification tasks, accuracy, recall, and F1 scores are used; for generative tasks, BLEU and ROUGE are used. Furthermore, the performance of each set of hyperparameters on the validation set is analyzed to identify the optimal parameter combination. A cross-validation mechanism is then used to divide the dataset into k parts, with k-1 parts used for training and the remaining part used for validation, repeated k times. This effectively utilizes the dataset, reduces the risk of overfitting, and provides a robust assessment of model performance.

[0103] S1002. Determine a target hyperparameter combination in a hyperparameter set according to a grid search method.

[0104] In some embodiments, grid search is a common hyperparameter optimization method that systematically traverses multiple hyperparameter combinations and determines the performance of each combination through cross-validation. A grid is defined in the hyperparameter space, and each grid point represents a specific combination of a set of hyperparameters. Each set of hyperparameter combinations is cross-validated to evaluate the performance of the model. Based on the performance evaluation results on the validation set, the hyperparameter combination with the best performance is selected as the target hyperparameter combination.

[0105] S1003. Configure the language model according to the target hyperparameter combination.

[0106] In some embodiments, the parameter values ​​in the target hyperparameter combination are set to the corresponding configuration of the language model, the language model is initialized using the set parameters, and the model is trained using the training dataset until a predetermined number of training rounds is reached or other stopping conditions are met. Then, the performance of the model is evaluated on the test dataset to ensure that it has good generalization ability. If the model performance does not meet expectations, return to S1001 to further adjust the hyperparameter set, and then repeat S1002 and S1003 to improve the generalization ability and prediction accuracy of the model.

[0107] S102: Determine a first scaling vector of the second matrix and a second scaling vector of the third matrix.

[0108] In some embodiments, in the second matrix and the third matrix, the scaling vector refers to a vector used to adjust the importance of the matrix. For example, when a feature is not important for the current task, the corresponding scaling vector can reduce the influence of this feature. The first scaling vector is used to fine-tune the second matrix. During the model training process, the first scaling vector can help the model better adapt to the needs of specific tasks or data by fine-tuning the second matrix. Similarly, the second scaling vector is used to fine-tune the third matrix. The first scaling vector works together with the second scaling vector to jointly optimize the low-rank part, so that the entire model can have better performance while maintaining high efficiency.

[0109] It should be understood that the value of the scaling vector can be determined based on the norms of the second and third matrices. Matrices with larger norms can have smaller scaling vectors to maintain stability during model training. Alternatively, the value of the scaling vector can be dynamically adjusted based on the performance of the model during training. For example, if the second matrix exhibits better performance during training, the value of its scaling vector can be gradually increased, and vice versa.

[0110] Exemplarily, the values ​​of the first scaling vector and the second scaling vector are initialized to 1.0.

[0111] S103 : In response to the norm of the second matrix being greater than or equal to the norm of the third matrix, determine the second matrix and the second scaling vector as target training matrices.

[0112] In some embodiments, the norm is a function with a "length" concept, assigning a non-zero positive length or size to all vectors in the vector space. By calculating the norms of the second and third matrices and comparing them, the matrix with the larger norm is selected as the target training matrix. The remaining parameter matrices are marked as frozen parameters and excluded from subsequent training optimization, thereby prioritizing matrices containing more important learning information.

[0113] It should be understood that in response to the norm of the second matrix being smaller than the norm of the third matrix, the third matrix and the first scaling vector are determined as target training parameters.

[0114] In some embodiments, each layer of the language model is traversed, and for each layer in the model, the bi-norm of the second matrix and the third matrix is ​​calculated, and the resulting values ​​are compared. The bi-norm, also known as the Euclidean norm, is a norm that measures the size of a vector in a vector space. In Euclidean space, the bi-norm is defined as the square root of the sum of the squares of the elements of the vector.

[0115] S104: Obtain the training text and the actual response result of the training text.

[0116] In some embodiments, training data is obtained based on historical data, public data sets or automated tools, and text types, such as sentiment analysis, text classification, machine translation, etc., or data types, such as text data, labels, user feedback, etc., are automatically labeled through natural language processing technology to obtain training text for the language model and corresponding response results.

[0117] As an exemplary scenario, in the field of sentiment analysis, the training text may be "This book is great!", and the corresponding response result may be a positive sentiment; the training text may be "The battery life of this camera is average.", and the corresponding response result may be a negative sentiment; the training text may be "Today is a sunny day.", and the corresponding response result may be a neutral sentiment, etc.

[0118] As another exemplary scenario, in the field of text classification, the training text may be "Computer system installation tutorial", and the corresponding response result may be computer category; the training text may be "Scientists discover new energy materials", and the corresponding response result may be science and technology news; the training text may be "Niche scenic spot travel guide", and the corresponding response result may be tourism category.

[0119] As another exemplary scenario, in the field of machine translation, the training text may be "Good morning", and the goal is to translate English into Chinese, and the corresponding answer result is good morning.

[0120] As another exemplary scenario, in the field of dialogue generation, the training text may be user input: "How's the weather today?" The corresponding response result may be the generated dialogue: "The weather is very good today, sunny and suitable for outdoor activities."

[0121] As another exemplary scenario, in the field of text summarization, the training text may be complete news content, and the corresponding reply result may be information such as the main events, key figures, and results of the news.

[0122] It should be understood that the above exemplary scenarios are only for illustrating how to obtain and use training text and its response results. This solution is not limited to the above exemplary scenarios. In actual applications, this solution is applicable to any natural language processing application scenario.

[0123] S105: Input the training text into the target training matrix to obtain the predicted answer result.

[0124] In some embodiments, after the training text is encoded into a vector form, it is input into the target training matrix for forward propagation calculation to obtain a vector representation of the predicted response result. Furthermore, through decoding algorithms such as greedy search and beam search, the vector representation of the predicted response result is decoded into a readable text form.

[0125] For example, the forward propagation process satisfies the following expression:

[0126] h=(W res +A·diag(a)·B·diag(b))x;

[0127] Among them, x is the input vector, h is the reduced dimensionality vector output by the hidden layer, a is the first scaling vector, and b is the second scaling vector.

[0128] S106. Determine the result error based on the predicted response result and the actual response result.

[0129] In some embodiments, the value of the loss function is calculated based on the predicted response result and the actual response result. The smaller the loss function, the closer the model prediction result is to the actual result. In addition, by using the backpropagation algorithm to calculate the gradient value of the loss function with respect to the parameters in the target training matrix, these gradient values ​​can be used in the subsequent model training process.

[0130] It should be noted that the loss function of the model training is the cross entropy loss function, and the resulting error satisfies the following expression:

[0131]

[0132] Among them, y i For the actual answer result, is the predicted response result, and n is the total number of categories.

[0133] S107. Based on the result error, train the target training matrix to obtain an optimized language model.

[0134] In some embodiments, the parameter values ​​in the target training matrix are updated using a gradient descent method based on the calculated gradient values. Through multiple iterations of the training process, the resulting error is gradually reduced and the model performance is optimized. At the same time, to prevent overfitting or underfitting of the model, regularization techniques such as L1 regularization and L2 regularization are introduced during the training process to limit the range or complexity of the model parameters. Training is stopped when performance on the validation set no longer improves to avoid overfitting.

[0135] It should be noted that the training process uses the AdamW optimizer. The AdamW optimizer can update model parameters more stably by explicitly handling weight decay. In the original Adam optimizer, weight decay is included in part of the gradient calculation, which will cause the actual effect to be inconsistent with expectations. AdamW processes weight decay as an independent step, effectively reducing this potential error and making the regularization effect of weight decay on the model more reliable and effective. In addition, because the AdamW optimizer can better control the update direction and size of the model parameters, it can accelerate the convergence process of the model. By combining the advantages of the Adam optimizer's fast convergence and the regularization effect of weight decay, the model can reach convergence faster during training, thereby reducing training time and resource consumption.

[0136] The language model training method, apparatus, device and storage medium provided in the present application decompose the pre-training matrix of the language model into a first matrix of high-rank features and a second matrix and a third matrix of low-rank features through a singular value decomposition algorithm, which helps to retain the key features in the model while significantly reducing the number of parameters that need to be optimized in the model. Furthermore, by determining the scaling vectors of the second matrix and the third matrix and selecting the target training matrix according to their norms, flexible adjustment of the low-rank feature representation is achieved. This dynamic adjustment mechanism can flexibly optimize the training parameters of the model for different situations, focusing on matrices that have a greater impact on the model, that is, matrices with larger norms, thereby improving optimization accuracy. At the same time, by reducing unnecessary training parameter updates, further, by obtaining the training text and its actual response results, and inputting the training text into the target training matrix to obtain the predicted response results, the difference between the predicted results and the actual results can be directly compared. This precise training feedback mechanism helps to accurately evaluate the performance of the model and provides strong support for further optimization of the model. Based on the error between the predicted response results and the actual response results, the language model is optimized by training the target training matrix, thereby improving the accuracy and generalization ability of the model in processing text data and reducing the complexity and computational cost of the model.

[0137] Furthermore, by determining the target hyperparameter combination through the grid search method, we can systematically traverse all possible hyperparameter combinations and find the optimal configuration. At the same time, the introduction of preset evaluation indicators and verification mechanisms further improves the accuracy and reliability of hyperparameter selection, significantly improving the performance and training efficiency of the model.

[0138] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied in a distributed scenario and performed by multiple devices working together. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the method.

[0139] It should be noted that the above description is limited to some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that described in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0140] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a language model training device.

[0141] refer to Figure 6 , the language model training device includes: a determination module 601, an acquisition module 602, a processing module 603 and a training module 604;

[0142] The determining module 601 is used to determine a first matrix, a second matrix, and a third matrix according to the pre-training matrix of the language model; the first matrix is ​​used to represent the high-rank features of the pre-training matrix; the second matrix and the third matrix are used to represent the low-rank features of the pre-training matrix;

[0143] The determining module 601 is further configured to determine a first scaling vector of the second matrix and a second scaling vector of the third matrix;

[0144] The determining module 601 is further configured to, in response to a norm of the second matrix being greater than or equal to a norm of the third matrix, determine that the second matrix and the second scaling vector are target training matrices;

[0145] The acquisition module 602 is used to acquire the training text and the actual response result of the training text;

[0146] The processing module 603 is used to input the training text into the target training matrix to obtain a predicted answer result;

[0147] The determining module 601 is further configured to determine a result error based on the predicted response result and the actual response result;

[0148] The training module 604 is configured to train the target training matrix based on the result error to obtain the optimized language model.

[0149] In some embodiments, the determination module 601 is further configured to, in response to a norm of the second matrix being smaller than a norm of the third matrix, determine the third matrix and the first scaling vector as the target training parameters.

[0150] In some embodiments, the norm is a two-norm.

[0151] In some embodiments, the determination module 601 is specifically used to obtain the pre-training matrix of the language model; determine the low-rank matrix set of the pre-training matrix based on a preset rank value set; input the sample data into the low-rank matrix set according to an iterative optimization algorithm to determine the target low-rank matrix; the sample data is a set of data randomly selected from the training data set of the language model; the sample data is used to characterize the feature distribution of the training data set; determine the target rank value based on the target low-rank matrix; based on the target rank value, perform singular value decomposition on the pre-training matrix to obtain the first matrix, the second matrix and the third matrix.

[0152] In some embodiments, the language model training device further includes a configuration module 605 .

[0153] The configuration module 605 is used to configure the hyperparameters of the language model.

[0154] In some embodiments, the configuration module 605 is specifically used to determine a hyperparameter set; the hyperparameter set includes at least one group of hyperparameter combinations; according to a grid search method, determine a target hyperparameter combination in the hyperparameter set; and configure the language model according to the target hyperparameter combination.

[0155] In some embodiments, the determination module 601 is further specifically used to input the hyperparameter combination into the initial model to obtain the language model; input sample data into the language model to obtain a sample result; the sample data is a set of data randomly selected from the training data set of the language model; and in response to the sample result satisfying the preset evaluation index and the preset verification mechanism, determine the hyperparameter set.

[0156] For the convenience of description, the above devices are described as being divided into various modules according to their functions. Of course, when implementing this application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0157] The apparatus of the above embodiment is used to implement the corresponding language model training method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0158] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the program, the language model training method described in any of the above embodiments is implemented.

[0159] Figure 7 10 is a schematic diagram showing a more specific hardware structure of an electronic device provided in this embodiment. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are communicatively connected to each other within the device via the bus 1050.

[0160] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0161] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 1020 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0162] The input / output interface 1030 is used to connect input / output modules to implement information input and output. The input / output modules can be configured as components within the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0163] The communication interface 1040 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WiFi, Bluetooth, etc.).

[0164] The bus 1050 comprises a path for transmitting information between the various components of the device (eg, the processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 ).

[0165] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may only include the components necessary to implement the embodiments of this specification, and does not necessarily include all the components shown in the figure.

[0166] The electronic device of the above embodiment is used to implement the corresponding language model training method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0167] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the language model training method described in any of the above embodiments.

[0168] The computer-readable media of this embodiment include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0169] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the language model training method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0170] Based on the same inventive concept, corresponding to the language model training method described in any of the above embodiments, the present disclosure also provides a computer program product, which includes a computer program. In some embodiments, the computer program is executable by one or more processors to cause the processors to perform the language model training method. Corresponding to the execution subject corresponding to each step in each embodiment of the language model training method, the processor that executes the corresponding step may belong to the corresponding execution subject.

[0171] The computer program product of the above embodiment is used to enable the processor to execute the language model training method described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0172] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. Within the scope of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0173] In addition, for simplicity of description and discussion, and in order not to make the embodiment of the application difficult to understand, the known power supply / ground connection with integrated circuit (IC) chip and other components may or may not be shown in the accompanying drawings provided. In addition, the device can be shown in the form of a block diagram to avoid making the embodiment of the application difficult to understand, and this also takes into account the following fact, that is, the details of the embodiment of these block diagram devices are highly dependent on the platform to be implemented in the embodiment of the application (that is, these details should be fully within the scope of understanding of those skilled in the art). When specific details (for example, circuit) are set forth to describe exemplary embodiments of the application, it will be apparent to those skilled in the art that the embodiment of the application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0174] Although the present invention has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may utilize the embodiments discussed.

[0175] The embodiments of the present application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of this application.

Claims

1. A method for training a language model, characterized in that: The method comprises: Determine a first matrix, a second matrix, and a third matrix according to the pre-training matrix of the language model; the first matrix is ​​used to represent the high-rank features of the pre-training matrix; the second matrix and the third matrix are used to represent the low-rank features of the pre-training matrix; determining a first scaling vector of the second matrix and a second scaling vector of the third matrix; In response to a norm of the second matrix being greater than or equal to a norm of the third matrix, determining the second matrix and the second scaling vector as a target training matrix; In response to a norm of the second matrix being smaller than a norm of the third matrix, determining the third matrix and the first scaling vector as the target training matrix; Obtaining training text and actual response results of the training text; The training text is input into the target training matrix to obtain a predicted answer result; wherein, after the training text is encoded into a vector form, it is input into the target training matrix for forward propagation calculation to obtain a vector representation of the predicted answer result; wherein, the forward propagation process satisfies the following expression: h=(W res +A·diag(a)·B·diag(b))x; Among them, x is the input vector, h is the dimension reduction vector of the hidden layer output, a is the first scaling vector, b is the second scaling vector, A∈R m×r , B∈R r×n , W res is a first matrix; determining a result error according to the predicted response result and the actual response result; Based on the result error, the target training matrix is ​​trained to obtain the optimized language model.

2. The method according to claim 1, characterized in that The norm is a two-norm.

3. The method according to claim 1, characterized in that The determining of the first matrix, the second matrix, and the third matrix according to the pre-trained matrix of the language model includes: Obtaining a pre-trained matrix of the language model; Determining a low-rank matrix set of the pre-trained matrix according to a preset rank value set; According to an iterative optimization algorithm, sample data is input into the low-rank matrix set to determine a target low-rank matrix; the sample data is a set of data randomly extracted from the training data set of the language model; the sample data is used to characterize the feature distribution of the training data set; Determining a target rank value according to the target low-rank matrix; Based on the target rank value, singular value decomposition is performed on the pre-training matrix to obtain the first matrix, the second matrix, and the third matrix.

4. The method according to claim 1, wherein Before determining the first matrix, the second matrix, and the third matrix according to the pre-trained matrix of the language model, the method further includes: Configure the hyperparameters of the language model.

5. The method according to claim 4, characterized in that Configuring the hyperparameters of the language model includes: Determining a hyperparameter set; the hyperparameter set includes at least one set of hyperparameter combinations; Determining a target hyperparameter combination from the hyperparameter set according to a grid search method; The language model is configured according to the target hyperparameter combination.

6. The method according to claim 5, characterized in that Determining the hyperparameter set includes: Inputting the hyperparameter combination into an initial model to obtain the language model; Inputting sample data into the language model to obtain a sample result; the sample data is a set of data randomly extracted from the training data set of the language model; In response to the sample results satisfying a preset evaluation indicator and a preset verification mechanism, the hyperparameter set is determined.

7. A language model training device, characterized in that: The device includes: a determination module, an acquisition module, a processing module and a training module; The determination module is used to determine a first matrix, a second matrix, and a third matrix according to the pre-training matrix of the language model; the first matrix is ​​used to represent the high-rank features of the pre-training matrix; the second matrix and the third matrix are used to represent the low-rank features of the pre-training matrix; The determining module is further configured to determine a first scaling vector of the second matrix and a second scaling vector of the third matrix; The determining module is further configured to, in response to a norm of the second matrix being greater than or equal to a norm of the third matrix, determine that the second matrix and the second scaling vector are target training matrices; The determining module is further configured to, in response to a norm of the second matrix being smaller than a norm of the third matrix, determine that the third matrix and the first scaling vector are the target training matrix; The acquisition module is used to obtain a training text and an actual response result of the training text; wherein, after encoding the training text into a vector form, the training text is input into the target training matrix for forward propagation calculation to obtain a vector representation of the predicted response result; wherein the forward propagation process satisfies the following expression: h=(W res +A·diag(a)·B·diag(b))x; Among them, x is the input vector, h is the dimension reduction vector of the hidden layer output, a is the first scaling vector, b is the second scaling vector, A∈R m×r , B∈R r×n , W res is the first matrix; The processing module is used to input the training text into the target training matrix to obtain a predicted answer result; The determining module is further configured to determine a result error based on the predicted response result and the actual response result; The training module is used to train the target training matrix based on the result error to obtain the optimized language model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 6 when executing the program. 9 . A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method according to claim 1 .

Images