Language model training method and device, equipment and storage medium

Abstract

The invention provides a language model training method and device, equipment and a storage medium, and the method comprises the steps: determining a first matrix, a second matrix and a third matrix according to a pre-training matrix of a language model; determining a first scaling vector of the second matrix and a second scaling vector of the third matrix; in response to the condition that the norm of the second matrix is greater than or equal to the norm of the third matrix, determining the second matrix and the second scaling vector as a target training matrix; obtaining a training text and an actual reply result of the training text; inputting the training text into the target training matrix to obtain a prediction reply result; determining a result error according to the predicted reply result and the actual reply result; and training the target training matrix based on the result error to obtain an optimized language model. Through the singular value decomposition algorithm, the training parameters of the language model are reduced, and the complexity and the calculation cost of the model are reduced.

Description

Technical Field

[0001] The present application relates to the field of natural language processing, and in particular to a language model training method, apparatus, device and storage medium. Background Art

[0002] With the rapid development of artificial intelligence technology, language models have become the core force in the field of natural language processing. For example, in machine translation, text summarization, sentiment analysis, and question-answering systems, these models have powerful text generation, understanding, and reasoning capabilities through deep learning and massive text data training.

[0003] However, as the size of language models continues to increase, the number of their parameters also grows exponentially, which not only places higher demands on storage space, but also makes the training process require expensive computing resources. Summary of the invention

[0004] In view of this, the purpose of this application is to propose a language model training method, device, equipment and storage medium.

[0005] As one aspect of the present application, a method for training a language model is provided, comprising:

[0006] According to the pre-trained matrix of the language model, a first matrix, a second matrix and a third matrix are determined; the first matrix is ​​used to characterize the high-rank features of the pre-trained matrix; the second matrix and the third matrix are used to characterize the low-rank features of the pre-trained matrix;

[0007] Determine a first scaling vector of the second matrix and a second scaling vector of the third matrix;

[0008] In response to the norm of the second matrix being greater than or equal to the norm of the third matrix, determining the second matrix and the second scaling vector as a target training matrix;

[0009] Obtaining training text and actual response results of the training text;

[0010] Inputting the training text into the target training matrix to obtain a predicted answer result;

[0011] Determining a result error according to the predicted response result and the actual response result;

[0012] Based on the result error, the target training matrix is ​​trained to obtain the optimized language model.

[0013] Optionally, in response to a norm of the second matrix being smaller than a norm of the third matrix, the third matrix and the first scaling vector are determined as the target training parameters.

[0014] Optionally, the norm is a binary norm.

[0015] Optionally, determining the first matrix, the second matrix, and the third matrix according to the pre-trained matrix of the language model includes:

[0016] Obtaining a pre-trained matrix of the language model;

[0017] Determining a low-rank matrix set of the pre-trained matrix according to a preset rank value set;

[0018] According to an iterative optimization algorithm, sample data is input into the low-rank matrix set to determine a target low-rank matrix; the sample data is a set of data randomly extracted from the training data set of the language model; the sample data is used to characterize the feature distribution of the training data set;

[0019] Determining a target rank value according to the target low rank matrix;

[0020] Based on the target rank value, singular value decomposition is performed on the pre-training matrix to obtain the first matrix, the second matrix and the third matrix.

[0021] Optionally, before determining the first matrix, the second matrix, and the third matrix according to the pre-trained matrix of the language model, the method further includes:

[0022] Configure the hyperparameters of the language model.

[0023] Optionally, configuring the hyper parameters of the language model includes:

[0024] Determine a hyperparameter set; the hyperparameter set includes at least one set of hyperparameter combinations;

[0025] Determine a target hyperparameter combination in the hyperparameter set according to a grid search method;

[0026] The language model is configured according to the target hyperparameter combination.

[0027] Optionally, determining a set of hyperparameters includes:

[0028] Inputting the hyperparameter combination into an initial model to obtain the language model;

[0029] Inputting sample data into the language model to obtain a sample result; the sample data is a set of data randomly selected from the training data set of the language model;

[0030] In response to the sample results satisfying a preset evaluation indicator and a preset verification mechanism, the hyperparameter set is determined.

[0031] As a second aspect of the present application, a language model training device is provided, comprising: a determination module, an acquisition module, a processing module and a training module;

[0032] The determination module is used to determine a first matrix, a second matrix and a third matrix according to the pre-training matrix of the language model; the first matrix is ​​used to characterize the high-rank features of the pre-training matrix; the second matrix and the third matrix are used to characterize the low-rank features of the pre-training matrix;

[0033] The determination module is further used to determine a first scaling vector of the second matrix and a second scaling vector of the third matrix;

[0034] The determination module is further configured to, in response to a norm of the second matrix being greater than or equal to a norm of the third matrix, determine that the second matrix and the second scaling vector are target training matrices;

[0035] The acquisition module is used to acquire the training text and the actual response result of the training text;

[0036] The processing module is used to input the training text into the target training matrix to obtain a predicted answer result;

[0037] The determination module is further used to determine a result error according to the predicted answer result and the actual answer result;

[0038] The training module is used to train the target training matrix based on the result error to obtain the optimized language model.

[0039] Optionally, the determination module is further used to, in response to a norm of the second matrix being smaller than a norm of the third matrix, determine the third matrix and the first scaling vector as the target training parameters.

[0040] Optionally, the norm is a binary norm.

[0041] Optionally, the determination module is specifically used to obtain a pre-trained matrix of the language model; determine a low-rank matrix set of the pre-trained matrix based on a preset rank value set; input sample data into the low-rank matrix set according to an iterative optimization algorithm to determine a target low-rank matrix; the sample data is a set of data randomly selected from the training data set of the language model; the sample data is used to characterize the feature distribution of the training data set; determine a target rank value based on the target low-rank matrix; and perform singular value decomposition on the pre-trained matrix based on the target rank value to obtain the first matrix, the second matrix and the third matrix.

[0042] Optionally, the language model training device also includes a configuration module.

[0043] The configuration module is used to configure the hyperparameters of the language model.

[0044] Optionally, the configuration module is specifically used to determine a hyperparameter set; the hyperparameter set includes at least one group of hyperparameter combinations; according to a grid search method, determine a target hyperparameter combination in the hyperparameter set; and configure the language model according to the target hyperparameter combination.

[0045] Optionally, the determination module is also specifically used to input the hyperparameter combination into the initial model to obtain the language model; input sample data into the language model to obtain a sample result; the sample data is a set of data randomly selected from the training data set of the language model; and determine the hyperparameter set in response to the sample result satisfying a preset evaluation indicator and a preset verification mechanism.

[0046] As a third aspect of the present application, an electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned language model training method when executing the program.

[0047] As a fourth aspect of the present application, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the above-mentioned language model training method provided in the present application.

[0048] From the above, it can be seen that the training method, apparatus, device and storage medium of the language model provided in the present application decompose the pre-training matrix of the language model into a first matrix of high-rank features and a second matrix and a third matrix of low-rank features through a singular value decomposition algorithm, which helps to retain the key features in the model while significantly reducing the number of parameters that need to be optimized in the model. Furthermore, by determining the scaling vectors of the second matrix and the third matrix and selecting the target training matrix according to their norm size, flexible adjustment of the low-rank feature representation is achieved. This dynamic adjustment mechanism can flexibly optimize the training parameters of the model for different situations, focusing on the matrix that has a greater impact on the model, that is, the matrix with a larger norm, thereby improving the optimization accuracy. At the same time, by reducing unnecessary training parameter updates, further, by obtaining the training text and its actual response results, and inputting the training text into the target training matrix to obtain the predicted response results, the difference between the predicted results and the actual results can be directly compared. This precise training feedback mechanism helps to accurately evaluate the performance of the model and provide strong support for further optimization of the model. Based on the error between the predicted response results and the actual response results, the language model is optimized by training the target training matrix, thereby improving the accuracy and generalization ability of the model in processing text data, and reducing the complexity and computational cost of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0050] Figure 1 A flow chart of a method for training a language model provided in an embodiment of the present application;

[0051] Figure 2 A flowchart of another language model training method provided in an embodiment of the present application;

[0052] Figure 3 A flowchart of another language model training method provided in an embodiment of the present application;

[0053] Figure 4 A flowchart of another language model training method provided in an embodiment of the present application;

[0054] Figure 5 A flowchart of another language model training method provided in an embodiment of the present application;

[0055] Figure 6A schematic diagram of the composition of a language model training device provided in an embodiment of the present application;

[0056] Figure 7 A schematic diagram of the composition of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0058] It should be noted that, in the embodiments of the present application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a concrete way. Unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should have the usual meaning understood by people with ordinary skills in the field to which the present application belongs. SUMMARY OF THE INVENTION

[0060] In related technologies, due to the popularity of language models, the rapid increase in model size and complexity has become a phenomenon that cannot be ignored. These models have demonstrated unprecedented performance advantages in many tasks due to their large number of parameters, but at the same time have also caused a surge in storage requirements and huge consumption of computing resources. Although the full fine-tuning strategy, that is, adjusting all parameters of the model to adapt to specific tasks, can maximize the model's potential, it faces severe challenges in practical applications due to its high cost. This situation is not only challenging in academic research, but has also become a major obstacle in industrial applications.

[0061] The inventors of the present application have found that in current technical practices, although methods such as adapters, prompt tuning, and low-rank adaptation (LoRA) can reduce the amount of parameter adjustment, during the fine-tuning process, the selection of optimal hyperparameters is usually significantly different from the hyperparameters used in full fine-tuning. The main reason for this difference is the different number of trainable parameters. For example, in efficient parameter fine-tuning, in order to adapt to fewer trainable parameters, the optimal learning rate often needs to be set higher than that in full fine-tuning.

[0062] In order to solve the above problems, the present application provides a training method for a language model. Through a singular value decomposition algorithm, a pre-training matrix of the language model is decomposed into a first matrix of high-rank features and a second matrix and a third matrix of low-rank features, which helps to retain the key features in the model while significantly reducing the number of parameters that need to be optimized in the model. Furthermore, by determining the scaling vectors of the second matrix and the third matrix and selecting the target training matrix according to their norm sizes, flexible adjustment of the low-rank feature representation is achieved. This dynamic adjustment mechanism can flexibly optimize the training parameters of the model for different situations, focusing on matrices that have a greater impact on the model, that is, matrices with larger norms, thereby improving the optimization accuracy. At the same time, by reducing unnecessary training parameter updates, further, by obtaining training text and its actual response results, and inputting the training text into the target training matrix to obtain the predicted response results, the difference between the predicted results and the actual results can be directly compared. This precise training feedback mechanism helps to accurately evaluate the performance of the model and provide strong support for further optimization of the model. Based on the error between the predicted response results and the actual response results, the language model is optimized by training the target training matrix, thereby improving the accuracy and generalization ability of the model in processing text data, and reducing the complexity and computational cost of the model.

[0063] After introducing the basic principles of the present application, various non-limiting implementation methods of the present application are described in detail below.

[0064] Figure 1 A flow chart of a method for training a language model provided in an embodiment of the present application. Figure 1 As shown, the language model training method provided in this application specifically includes the following steps:

[0065] S101. Determine a first matrix, a second matrix, and a third matrix according to a pre-trained matrix of a language model.

[0066] Among them, the first matrix is ​​used to represent the high-rank characteristics of the pre-training matrix; the second matrix and the third matrix are used to represent the low-rank characteristics of the pre-training matrix.

[0067] In some embodiments, the pre-trained matrix is ​​a weight matrix obtained by pre-training a large amount of text data during the language model training process, which contains the model's understanding and generation capabilities of language. The pre-trained matrix can be decomposed into a part with the same shape as the pre-trained matrix, namely, a high-rank feature matrix (first matrix), and a low-rank part that captures the high variance direction, namely, a low-rank feature matrix (second and third matrices) using singular value decomposition (SVD).

[0068] It should be noted that singular value decomposition is an algorithm widely used in the fields of data science and machine learning. It can decompose a matrix into three specific matrix products: an orthogonal matrix, a diagonal matrix (the elements on the diagonal are called singular values, arranged from large to small), and the transpose of another orthogonal matrix. The decomposition process satisfies the following expression:

[0069] U·diag(S)·V T = SVD(W);

[0070]

[0071]

[0072]

[0073] W=W res +AB;

[0074] Where W∈R m×n , U∈R m×min(m,n) ,V∈R n×min(m,n) , s∈R min(m,n) ,A∈R m×r , B∈R r×n , W is the pre-training matrix, U is the orthogonal matrix containing the left singular vector, V T is the transpose of the orthogonal matrix containing the right singular vectors, W res is the first matrix, A is the second matrix, B is the third matrix, and r is the rank of the pre-trained matrix.

[0075] It should be understood that the first matrix is ​​also a residual matrix, which retains important and informative features in the pre-trained matrix and will be frozen during the training process of this method.

[0076] In some embodiments, Figure 2 As shown, S101 can be specifically implemented as follows S1011-S1015:

[0077] S1011. Obtain a pre-trained matrix of a language model.

[0078] In some embodiments, the language model is trained based on a large amount of text data, which is converted into a matrix form. The pre-trained matrix refers to the parameter matrix of the language model pre-trained on large-scale data. This matrix contains the patterns, structures, and relationships learned by the language model.

[0079] S1012. Determine a low-rank matrix set of the pre-trained matrix according to a preset rank value set.

[0080] In some embodiments, based on the dimension of the pre-trained matrix, such as 4096, a low-rank set is selected. For example, the low-rank set is rank_set = {4, 8, 16, 32, 64, 128}. Then, matrix decomposition is performed on each preset rank value through singular value decomposition or truncated singular value decomposition to obtain the corresponding low-rank matrices. These low-rank matrices will be used in subsequent iterative optimization processes.

[0081] S1013. According to an iterative optimization algorithm, sample data is input into a low-rank matrix set to determine a target low-rank matrix.

[0082] The sample data is a set of data randomly extracted from the training data set of the language model; the sample data is used to characterize the feature distribution of the training data set.

[0083] In some embodiments, the sample data is encoded into a vector form that the model can process, and then these vectors are used as input to calculate the output of each low-rank matrix through forward propagation. Next, the loss function is calculated based on the output and the actual label (or target value), and the parameters of the low-rank matrix are updated using the back-propagation algorithm. After multiple iterations, the low-rank matrix with the smallest loss is selected as the target low-rank matrix. In this way, it is realized that for the weight decomposition of the pre-trained model, which rank value can maximize the performance of the model.

[0084] S1014. Determine a target rank value according to the target low-rank matrix.

[0085] In some embodiments, a corresponding rank value is determined from a target low-rank matrix, and this target rank value is the rank value with the best evaluation effect in a preset rank value set.

[0086] S1015. Based on the target rank value, perform singular value decomposition on the pre-training matrix to obtain a first matrix, a second matrix, and a third matrix.

[0087] In some embodiments, singular value decomposition is performed on the pre-trained matrix using a determined target rank value. During the decomposition process, an orthogonal matrix U matrix, a diagonal matrix, and the transpose of another orthogonal matrix V matrix are obtained. Then, the first n singular values ​​in the diagonal matrix are selected according to the target rank value, where n is the target rank value, and the corresponding rows and columns of the U matrix and the V matrix, thereby obtaining a first matrix representing high-rank features and a second and third matrix representing low-rank features.

[0088] In some embodiments, Figure 3 As shown, before S101, the language model training method provided in the embodiment of the present application further includes the following S100:

[0089] S100. Configure the hyperparameters of the language model.

[0090] In some embodiments, hyperparameters are set according to the language model and training objectives before model training, which directly affect the performance and generalization ability of the model, such as learning rate, optimizer, number of iterations, number of hidden layer units, weight decay, batch size, rank, etc. For each hyperparameter, an initial value can be set, and then these values ​​can be optimized through subsequent adjustments.

[0091] In some embodiments, Figure 4 As shown, S100 can be specifically implemented as follows S1001-S1003:

[0092] S1001. Determine a set of hyperparameters.

[0093] The hyperparameter set includes at least one set of hyperparameter combinations.

[0094] In some embodiments, based on professional knowledge, experiments or literature research, multiple candidate values ​​are selected for each hyperparameter and combined into different hyperparameter combinations, and all hyperparameter combinations are aggregated to obtain a hyperparameter set.

[0095] In some embodiments, Figure 5 As shown, S1001 can be specifically implemented as S1001a-S1001c as follows:

[0096] S1001a. Input the hyperparameter combination into the initial model to obtain a language model.

[0097] In some embodiments, a suitable language model is selected as the initial model according to task requirements such as text classification, language translation, etc., and a preset set of hyperparameter combinations, such as learning rate, batch size, number of model layers, etc., are applied to this model. The selection of these hyperparameters is based on prior knowledge or preliminary experiments on the impact of model performance. By adjusting these hyperparameters, language model instances with different performance characteristics can be obtained.

[0098] S1001b. Input the sample data into the language model to obtain a sample result.

[0099] The sample data is a set of data randomly selected from the training data set of the language model.

[0100] In some embodiments, a set of sample data is randomly extracted from a training data set of a language model and input into the language model as a test, and the language model predicts the sample data to obtain a sample result.

[0101] S1001c. In response to the sample results satisfying the preset evaluation index and the preset verification mechanism, determine a set of hyperparameters.

[0102] In some embodiments, evaluation indicators are set according to the task type. For example, in the case of classification tasks, accuracy, recall, F1 score, etc. are used as evaluation indicators; in the case of generative tasks, BLEU, ROUGE, etc. are used as evaluation indicators. Further, the performance of each set of hyperparameters on the validation set is analyzed to find the best parameter combination. The cross-validation mechanism is used to divide the data set into k parts, and k-1 parts are used for training in a loop. The remaining 1 part is used for validation, and this is repeated k times. This can effectively utilize the data set, reduce the risk of overfitting, and provide a robust evaluation of model performance.

[0103] S1002. Determine a target hyperparameter combination in a hyperparameter set according to a grid search method.

[0104] In some embodiments, grid search is a common hyperparameter optimization method, which systematically traverses multiple hyperparameter combinations, determines the performance of each combination through cross-validation, defines a grid in the hyperparameter space, and each grid point represents a specific combination of a set of hyperparameters. Each set of hyperparameter combinations is cross-validated to evaluate the performance of the model, and based on the performance evaluation results on the validation set, the hyperparameter combination with the best performance is selected as the target hyperparameter combination.

[0105] S1003. Configure the language model according to the target hyperparameter combination.

[0106] In some embodiments, each parameter value in the target hyperparameter combination is set to the corresponding configuration of the language model, the language model is initialized using the set parameters, and the model is trained using the training data set until a predetermined training round is reached or other stopping conditions are met. Then, the performance of the model is evaluated on the test data set to ensure that it has good generalization ability. If the model performance does not meet expectations, return to S1001 to further adjust the hyperparameter set, and then repeat S1002 and S1003 to improve the generalization ability and prediction accuracy of the model.

[0107] S102: Determine a first scaling vector of the second matrix and a second scaling vector of the third matrix.

[0108] In some embodiments, in the second matrix and the third matrix, the scaling vector refers to a vector used to adjust the importance of the matrix. For example, when a feature is not important for the current task, the corresponding scaling vector can reduce the influence of this feature. The first scaling vector is used to fine-tune the second matrix. During the model training process, the first scaling vector can help the model better adapt to the needs of specific tasks or data by fine-tuning the second matrix. Similarly, the second scaling vector is used to fine-tune the third matrix. The first scaling vector works with the second scaling vector to jointly optimize the low-rank part, so that the entire model can have better performance while maintaining high efficiency.

[0109] It should be understood that the value of the scaling vector can be determined according to the norms of the second matrix and the third matrix, and the scaling vector of the matrix with a larger norm can be set to a smaller value to maintain the stability of the model training. Alternatively, the value of the scaling vector can be dynamically adjusted according to the performance during the model training process. For example, if the second matrix shows better performance during the training process, the value of its scaling vector can be gradually increased, and vice versa.

[0110] Exemplarily, the values ​​of the first scaling vector and the second scaling vector are initialized to 1.0.

[0111] S103: In response to the norm of the second matrix being greater than or equal to the norm of the third matrix, determine the second matrix and the second scaling vector as a target training matrix.

[0112] In some embodiments, the norm is a function with the concept of "length", which gives non-zero positive length or size to all vectors in the vector space. By calculating the norms of the second matrix and the third matrix, the matrix with a larger norm is selected as the target training matrix according to the comparison result, and the other parameter matrices are marked as frozen parameters and do not participate in subsequent training optimization, so as to give priority to matrices containing more important learning information.

[0113] It should be understood that in response to the norm of the second matrix being smaller than the norm of the third matrix, the third matrix and the first scaling vector are determined as target training parameters.

[0114] In some embodiments, each layer of the language model is traversed, and for each layer in the model, the second norm of the second matrix and the third matrix is ​​calculated, and then the size of the result is compared. The second norm is also called the Euclidean norm, which is a norm that measures the size of a vector in a vector space. In the Euclidean space, the second norm is defined as the square root of the sum of the squares of the elements of the vector.

[0115] S104: Obtain training text and actual response results of the training text.

[0116] In some embodiments, training data is obtained based on historical data, public data sets or automated tools, and text types, such as sentiment analysis, text classification, machine translation, etc., or data types, such as text data, labels, user feedback, etc., are automatically annotated through natural language processing technology to obtain training text for the language model and corresponding response results.

[0117] As an exemplary scenario, in the field of sentiment analysis, the training text may be “This book is great!”, and the corresponding reply result may be a sentiment with a positive tendency; the training text may be “The battery life of this camera is average.”, and the corresponding reply result may be a sentiment with a negative tendency; the training text may be “Today is a sunny day.”, and the corresponding reply result may be a sentiment with a neutral tendency, etc.

[0118] As another exemplary scenario, in the field of text classification, the training text may be "Computer system installation tutorial", and the corresponding reply result may be computer category; the training text may be "Scientists discover new energy materials", and the corresponding reply result may be science and technology news; the training text may be "Niche scenic spot travel guide", and the corresponding reply result may be tourism category.

[0119] As another exemplary scenario, in the field of machine translation, the training text may be "Good morning", and the goal is to translate English into Chinese, and the corresponding answer result is good morning.

[0120] As another exemplary scenario, in the field of dialogue generation, the training text may be a user input: "How is the weather today?" The corresponding answer result may be a generated dialogue: "Today's weather is very good, the sun is shining, and it is suitable for outdoor activities."

[0121] As another exemplary scenario, in the field of text summarization, the training text may be complete news content, and the corresponding reply result may be information such as the main events, key figures, and results of the news.

[0122] It should be understood that the above exemplary scenarios are only for illustrating how to obtain and use training texts and their response results. The present solution is not limited to the above exemplary scenarios. In practical applications, the present solution is applicable to any application scenario of natural language processing.

[0123] S105: Input the training text into the target training matrix to obtain the predicted answer result.

[0124] In some embodiments, after the training text is encoded into a vector form, it is input into the target training matrix for forward propagation calculation to obtain a vector representation of the predicted reply result. Further, through decoding algorithms such as greedy search and beam search, the vector representation of the predicted reply result is decoded into a readable text form.

[0125] Exemplarily, the forward propagation process satisfies the following expression:

[0126] h=(W res +A·diag(a)·B·diag(b))x;

[0127] Among them, x is the input vector, h is the dimension-reduced vector of the hidden layer output, a is the first scaling vector, and b is the second scaling vector.

[0128] S106. Determine the result error based on the predicted response result and the actual response result.

[0129] In some embodiments, the value of the loss function is calculated based on the predicted response result and the actual response result. The smaller the loss function is, the closer the model prediction result is to the actual result. In addition, by using the back-propagation algorithm to calculate the gradient value of the loss function with respect to the parameters in the target training matrix, these gradient values ​​can be used in subsequent model training processes.

[0130] It should be noted that the loss function of the model training is the cross entropy loss function, and the result error satisfies the following expression:

[0131]

[0132] Among them, y i For the actual answer result, To predict the response results, n is the total number of categories.

[0133] S107. Based on the result error, train the target training matrix to obtain an optimized language model.

[0134] In some embodiments, the gradient descent method is used to update the parameter values ​​in the target training matrix according to the calculated gradient value. Through multiple iterations of the training process, the result error is gradually reduced and the model performance is optimized. At the same time, in order to prevent the occurrence of overfitting or underfitting of the model, regularization techniques such as L1 regularization and L2 regularization are introduced in the training process to limit the value range or complexity of the model parameters. When the performance on the validation set is no longer improved, the training is stopped to avoid overfitting.

[0135] It should be noted that the training process uses the AdamW optimizer. The AdamW optimizer can update the model parameters more stably by explicitly handling weight decay. In the original Adam optimizer, weight decay is included in part of the gradient calculation, which will cause the actual effect to be inconsistent with expectations. AdamW processes weight decay as an independent step, which effectively reduces this potential error and makes the regularization effect of weight decay on the model more reliable and effective. In addition, because the AdamW optimizer can better control the update direction and size of the model parameters, it can accelerate the convergence process of the model. By combining the advantages of the Adam optimizer's fast convergence and the regularization effect of weight decay, the model can reach a convergence state faster during training, thereby reducing training time and resource consumption.

[0136] The training method, apparatus, device and storage medium of the language model provided in the present application decompose the pre-training matrix of the language model into a first matrix of high-rank features and a second matrix and a third matrix of low-rank features through a singular value decomposition algorithm, which helps to retain the key features in the model and significantly reduces the number of parameters that need to be optimized in the model. Furthermore, by determining the scaling vectors of the second matrix and the third matrix and selecting the target training matrix according to their norm sizes, flexible adjustment of the low-rank feature representation is achieved. This dynamic adjustment mechanism can flexibly optimize the training parameters of the model according to different situations, focusing on the matrix that has a greater impact on the model, that is, the matrix with a larger norm, thereby improving the optimization accuracy. At the same time, by reducing unnecessary training parameter updates, further, by obtaining the training text and its actual response results, and inputting the training text into the target training matrix to obtain the predicted response results, the difference between the predicted results and the actual results can be directly compared. This precise training feedback mechanism helps to accurately evaluate the performance of the model and provide strong support for further optimization of the model, so that based on the error between the predicted response results and the actual response results, the language model is optimized by training the target training matrix, the accuracy and generalization ability of the model in processing text data are improved, and the complexity and computational cost of the model are reduced.

[0137] Furthermore, by determining the target hyperparameter combination through the grid search method, all possible hyperparameter combinations can be systematically traversed to find the optimal configuration. At the same time, the introduction of preset evaluation indicators and verification mechanisms further improves the accuracy and reliability of hyperparameter selection, significantly improving the performance and training efficiency of the model.

[0138] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0139] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0140] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a language model training device.

[0141] refer to Figure 6 , the language model training device comprises: a determination module 601, an acquisition module 602, a processing module 603 and a training module 604;

[0142] The determination module 601 is used to determine a first matrix, a second matrix and a third matrix according to the pre-training matrix of the language model; the first matrix is ​​used to characterize the high-rank features of the pre-training matrix; the second matrix and the third matrix are used to characterize the low-rank features of the pre-training matrix;

[0143] The determining module 601 is further used to determine a first scaling vector of the second matrix and a second scaling vector of the third matrix;

[0144] The determination module 601 is further configured to, in response to the norm of the second matrix being greater than or equal to the norm of the third matrix, determine that the second matrix and the second scaling vector are target training matrices;

[0145] The acquisition module 602 is used to acquire the training text and the actual response result of the training text;

[0146] The processing module 603 is used to input the training text into the target training matrix to obtain a predicted answer result;

[0147] The determination module 601 is further configured to determine a result error according to the predicted response result and the actual response result;

[0148] The training module 604 is used to train the target training matrix based on the result error to obtain the optimized language model.

[0149] In some embodiments, the determination module 601 is further configured to, in response to a norm of the second matrix being smaller than a norm of the third matrix, determine the third matrix and the first scaling vector as the target training parameters.

[0150] In some embodiments, the norm is a two-norm.

[0151] In some embodiments, the determination module 601 is specifically used to obtain a pre-trained matrix of the language model; determine a low-rank matrix set of the pre-trained matrix according to a preset rank value set; input sample data into the low-rank matrix set according to an iterative optimization algorithm to determine a target low-rank matrix; the sample data is a set of data randomly selected from the training data set of the language model; the sample data is used to characterize the feature distribution of the training data set; determine a target rank value according to the target low-rank matrix; based on the target rank value, perform singular value decomposition on the pre-trained matrix to obtain the first matrix, the second matrix and the third matrix.

[0152] In some embodiments, the language model training device further includes a configuration module 605 .

[0153] The configuration module 605 is used to configure the hyper parameters of the language model.

[0154] In some embodiments, the configuration module 605 is specifically used to determine a hyperparameter set; the hyperparameter set includes at least one group of hyperparameter combinations; according to a grid search method, determine a target hyperparameter combination in the hyperparameter set; and configure the language model according to the target hyperparameter combination.

[0155] In some embodiments, the determination module 601 is specifically used to input the hyperparameter combination into the initial model to obtain the language model; input sample data into the language model to obtain sample results; the sample data is a set of data randomly selected from the training data set of the language model; in response to the sample results satisfying preset evaluation indicators and a preset verification mechanism, determine the hyperparameter set.

[0156] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0157] The device of the above embodiment is used to implement the corresponding language model training method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0158] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the language model training method described in any of the above embodiments is implemented.

[0159] Figure 7 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0160] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0161] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0162] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0163] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0164] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0165] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0166] The electronic device of the above embodiment is used to implement the corresponding language model training method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0167] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the language model training method described in any of the above embodiments.

[0168] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0169] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the language model training method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0170] Based on the same inventive concept, corresponding to the training method of the language model described in any of the above embodiments, the present disclosure also provides a computer program product, which includes a computer program. In some embodiments, the computer program is executable by one or more processors so that the processor executes the training method of the language model. Corresponding to the execution subject corresponding to each step in each embodiment of the training method of the language model, the processor that executes the corresponding step may belong to the corresponding execution subject.

[0171] The computer program product of the above embodiment is used to enable the processor to execute the language model training method described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0172] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0173] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0174] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0175] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A method for training a language model, characterized in that: The method comprises: According to the pre-trained matrix of the language model, a first matrix, a second matrix and a third matrix are determined; the first matrix is ​​used to characterize the high-rank features of the pre-trained matrix; the second matrix and the third matrix are used to characterize the low-rank features of the pre-trained matrix; Determine a first scaling vector of the second matrix and a second scaling vector of the third matrix; In response to the norm of the second matrix being greater than or equal to the norm of the third matrix, determining the second matrix and the second scaling vector as a target training matrix; Obtaining training text and actual response results of the training text; Inputting the training text into the target training matrix to obtain a predicted answer result; Determining a result error according to the predicted response result and the actual response result; Based on the result error, the target training matrix is ​​trained to obtain the optimized language model.

2. The method according to claim 1, characterized in that The method further comprises: In response to a norm of the second matrix being smaller than a norm of the third matrix, determining the third matrix and the first scaling vector as the target training parameters.

3. The method according to any one of claims 1 to 2, characterized in that: The norm is a two-norm.

4. The method according to claim 1, characterized in that: The determining of the first matrix, the second matrix and the third matrix according to the pre-trained matrix of the language model includes: Obtaining a pre-trained matrix of the language model; Determining a low-rank matrix set of the pre-trained matrix according to a preset rank value set; According to an iterative optimization algorithm, sample data is input into the low-rank matrix set to determine a target low-rank matrix; the sample data is a set of data randomly extracted from the training data set of the language model; the sample data is used to characterize the feature distribution of the training data set; Determining a target rank value according to the target low rank matrix; Based on the target rank value, singular value decomposition is performed on the pre-training matrix to obtain the first matrix, the second matrix and the third matrix.

5. The method according to claim 1, characterized in that: Before determining the first matrix, the second matrix, and the third matrix according to the pre-trained matrix of the language model, the method further includes: Configure the hyperparameters of the language model.

6. The method according to claim 5, characterized in that The configuring of the hyper parameters of the language model includes: Determine a hyperparameter set; the hyperparameter set includes at least one set of hyperparameter combinations; Determine a target hyperparameter combination in the hyperparameter set according to a grid search method; The language model is configured according to the target hyperparameter combination.

7. The method according to claim 6, characterized in that Determining a set of hyperparameters includes: Inputting the hyperparameter combination into an initial model to obtain the language model; Inputting sample data into the language model to obtain a sample result; the sample data is a set of data randomly selected from the training data set of the language model; In response to the sample results satisfying a preset evaluation indicator and a preset verification mechanism, the hyperparameter set is determined.

8. A language model training device, characterized in that: The device comprises: a determination module, an acquisition module, a processing module and a training module; The determination module is used to determine a first matrix, a second matrix and a third matrix according to the pre-training matrix of the language model; the first matrix is ​​used to characterize the high-rank features of the pre-training matrix; the second matrix and the third matrix are used to characterize the low-rank features of the pre-training matrix; The determination module is further used to determine a first scaling vector of the second matrix and a second scaling vector of the third matrix; The determination module is further configured to, in response to a norm of the second matrix being greater than or equal to a norm of the third matrix, determine that the second matrix and the second scaling vector are target training matrices; The acquisition module is used to acquire the training text and the actual response result of the training text; The processing module is used to input the training text into the target training matrix to obtain a predicted answer result; The determination module is further used to determine a result error according to the predicted answer result and the actual answer result; The training module is used to train the target training matrix based on the result error to obtain the optimized language model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the program. 10 . A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method according to claim 1 .

Images