Language model training method and device, electronic equipment and storage medium

Abstract

The invention provides a language model training method and device, electronic equipment and a storage medium. The method comprises the following steps: training an initial language model to obtain a weight matrix, and decomposing the weight matrix to obtain a basic feature matrix and an initial task adaptation matrix; updating the initial task adaptation matrix to obtain an updated task adaptation matrix; determining the current sensitivity of each singular vector direction in the update task adaptation matrix of the current training period; determining a sensitive direction from a plurality of singular vector directions according to the current sensitivity, and performing regularization processing on a scaling coefficient associated with the sensitive direction to obtain a regularization loss function; in response to determining that the change sensitivity of the singular vector direction in the continuous preset number of training periods is in an attenuation state, performing parameter resetting on the zoom coefficient to obtain an updated zoom coefficient; and carrying out merging processing on the basic feature matrix and the update task adaptation matrix, and taking the update language model as a target language model.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a language model training method, device, electronic device, and storage medium. Background Art

[0002] With the rapid development of the artificial intelligence industry, language models have become a core technology in natural language processing. Large language models, trained through deep learning and extensive text data, have demonstrated strong capabilities for text generation, comprehension, and reasoning. However, the exponential growth in the number of parameters in language models leads to high training costs and slow training times.

[0003] In view of this, how to avoid the high training cost and slow training time caused by the large number of parameters in the language model has become a technical problem that needs to be solved urgently. Summary of the Invention

[0004] In view of this, the purpose of the present disclosure is to propose a language model training method, device, electronic device and storage medium to solve or partially solve the above technical problems.

[0005] Based on the above objectives, the first aspect of the present disclosure proposes a language model training method, the method comprising:

[0006] Acquire text sample data, use the text sample data to train an initial language model to obtain a weight matrix, and decompose the weight matrix to obtain a basic feature matrix and an initial task adaptation matrix;

[0007] Using the text sample data, the initial task adaptation matrix is updated to obtain an updated task adaptation matrix;

[0008] In response to determining that the number of training steps reaches a preset cycle boundary, determining a current sensitivity of each singular vector direction in the updated task adaptation matrix in the current training cycle;

[0009] Determining a sensitive direction from a plurality of singular vector directions according to the current sensitivity, and performing regularization processing on a scaling coefficient associated with the sensitive direction to obtain a regularized loss function;

[0010] In response to determining that the sensitivity of the singular vector direction to a change is in an attenuated state within a continuous preset number of training cycles, determining a reset probability according to a target attenuation period of continuous attenuation, and resetting the scaling factor according to the reset probability to obtain an updated scaling factor;

[0011] After determining that the training end condition is met, the basic feature matrix and the updated task adaptation matrix are merged, an updated language model is determined based on the updated scaling coefficient, and the updated language model is used as the target language model.

[0012] Based on the same inventive concept, the second aspect of the present disclosure proposes a language model training device, comprising:

[0013] a decomposition processing module configured to obtain text sample data, train an initial language model using the text sample data to obtain a weight matrix, and decompose the weight matrix to obtain a basic feature matrix and an initial task adaptation matrix;

[0014] An update processing module is configured to update the initial task adaptation matrix using the text sample data to obtain an updated task adaptation matrix;

[0015] a sensitivity determination module configured to determine a current sensitivity of each singular vector direction in the updated task adaptation matrix in a current training cycle in response to determining that the number of training steps reaches a preset cycle boundary;

[0016] a regularization processing module, configured to determine a sensitive direction from a plurality of singular vector directions according to the current sensitivity, and perform regularization processing on a scaling coefficient associated with the sensitive direction to obtain a regularized loss function;

[0017] a parameter resetting module configured to, in response to determining that the sensitivity of the singular vector direction to the change is in an attenuated state within a continuous preset number of training cycles, determine a reset probability according to a target attenuation period of continuous attenuation, and reset the scaling factor according to the reset probability to obtain an updated scaling factor;

[0018] The model determination module is configured to merge the basic feature matrix and the updated task adaptation matrix after determining that the training end condition is met, determine the updated language model based on the updated scaling coefficient, and use the updated language model as the target language model.

[0019] Based on the same inventive concept, the third aspect of the present disclosure proposes an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method described above when executing the computer program.

[0020] Based on the same inventive concept, a fourth aspect of the present disclosure proposes a non-transitory computer-readable storage medium, which stores computer instructions for causing a computer to execute the method described above.

[0021] As can be seen from the above, the language model training method, device, electronic device and storage medium provided by the present disclosure. Text sample data is obtained, and the initial language model is trained using the text sample data to obtain a weight matrix. The weight matrix is decomposed to obtain a basic feature matrix and an initial task adaptation matrix. The initial task adaptation matrix is updated using the text sample data to obtain an updated task adaptation matrix. In this way, the basic feature matrix remains unchanged, and the initial task adaptation matrix is updated, which can significantly reduce the scale of adjustable parameters while retaining the core pre-training knowledge. When the number of training steps reaches the preset cycle boundary, the current sensitivity of each singular vector direction in the updated task adaptation matrix of the current training cycle is determined. Based on the current sensitivity, a sensitive direction is determined from multiple singular vector directions, and the scaling coefficient associated with the sensitive direction is regularized to obtain a regularized loss function. When the sensitivity of the change of the singular vector direction is in an attenuated state within a continuous preset number of training cycles, the reset probability is determined according to the target attenuation cycle of continuous attenuation, and the scaling coefficient is reset according to the reset probability to obtain an updated scaling coefficient. This allows the singular vectors in sensitive directions to be dynamically maintained and only the scaling factors associated with those directions to be optimized. Furthermore, parameters in directions of continuously decreasing sensitivity are reset according to the reset probability. After determining that the training termination conditions have been met, the basic feature matrix and the updated task adaptation matrix are merged, and an updated language model is determined based on the updated scaling factors, which is then used as the target language model. This effectively avoids local optimality while reducing the complexity and computational cost of the language model, improving the target language model's focus on specific tasks, convergence speed, and generalization performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 This is a flow chart of the language model training method according to an embodiment of the present disclosure;

[0024] Figure 2 Schematic diagram of the structure of the language model training device according to an embodiment of the present disclosure;

[0025] Figure 3 Schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0027] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should have the usual meanings understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative position relationships. When the absolute position of the described object changes, the relative position relationship may also change accordingly.

[0028] Based on the background technology, with the rapid development of artificial intelligence, language models have become a core technology in the field of natural language processing (NLP). Large-scale language models, trained through deep learning and extensive text data, have demonstrated remarkable capabilities in text generation, comprehension, and reasoning. They are widely used in tasks such as machine translation, text summarization, sentiment analysis, and question-answering systems.

[0029] However, as language models continue to scale, the number of parameters in them is growing exponentially, often reaching billions or even more. This massive increase in parameters not only places unprecedented demands on storage space but also places enormous pressure on computing resources and memory, making current training methods more expensive and inefficient. Especially when fine-tuning for specific downstream tasks, fully training all parameters of a language model consumes enormous computing resources and time, making many practical applications impractical.

[0030] The rapid development of large-scale language models has revolutionized the field of natural language processing, bringing breakthroughs in a wide range of tasks, including commonsense reasoning and natural language understanding. However, these ultra-large language models, with billions of parameters, face significant challenges in fine-tuning downstream tasks. Fully fine-tuning all parameters consumes significant computational resources and memory, making it generally impractical in real-world applications.

[0031] To address this dilemma, the Parameter-Efficient Fine-Tuning (PEFT) method was developed. It minimizes computational overhead while maintaining or even exceeding the performance of full fine-tuning. Among various PEFT strategies, Low-Rank Adaptation (LoRA) has attracted considerable attention by introducing a low-rank matrix to capture task-specific updates. LoRA, based on the assumption that weight changes during fine-tuning exhibit low-rank properties, achieves efficient adaptation with a small number of trainable parameters.

[0032] The inventors of this application discovered that while LoRA and its variants have achieved remarkable results in many scenarios, they still have two core limitations in practical applications: first, the language model lacks an effective directional control mechanism for the update intensity of different semantic features during training; second, the perturbation direction of the parameter space is prone to convergence jitter during dynamic changes. These problems are particularly prominent in tasks such as style transfer and theme customization that require precise control of output characteristics. They may cause the generated content to deviate from the target semantics or the training process to be unstable, thereby limiting the practicality and generalization effect of the model.

[0033] As mentioned above, how to avoid the high training cost and slow training time caused by the large number of parameters in the language model has become an important research issue.

[0034] Based on the above description, if Figure 1 As shown, the language model training method proposed in this embodiment includes:

[0035] Step 101: obtain text sample data, use the text sample data to train an initial language model to obtain a weight matrix, and decompose the weight matrix to obtain a basic feature matrix and an initial task adaptation matrix.

[0036] In practice, the weight matrix is the pre-trained weight matrix, which is obtained by pre-training a large amount of text sample data during the language model training process. Specifically, the initial language model is trained based on a large amount of text sample data, which is converted into a matrix form. The pre-trained weight matrix refers to the parameter matrix of the language model pre-trained on large-scale text sample data. The pre-trained weight matrix contains the patterns, structures, and relationships learned by the language model.

[0037] The weight matrix contains the language model's understanding of language and the energy generated. According to the pre-trained weight matrix and energy distribution threshold of the language model, the frozen basic feature matrix and the task adaptation matrix for task update are determined.

[0038] Among them, the basic feature matrix is used to represent the high-energy core features of the weight matrix, and the task adaptation matrix is used to represent the low-energy adaptation features of the weight matrix. The task adaptation matrix obtained by decomposing the weight matrix is the initial task adaptation matrix, so as to update the initial task adaptation matrix subsequently.

[0039] Step 102: Update the initial task adaptation matrix by using the text sample data to obtain an updated task adaptation matrix.

[0040] Specifically, at the initial moment of each preset training cycle, a copy of the initial task adaptation matrix is saved, and only the initial task adaptation matrix is updated within the preset training cycle. Specifically, at the beginning of training, the parameters of the basic feature matrix are frozen, and at this time, only the parameters of the initial task adaptation matrix are allowed to be updated within the preset training cycle. When the input text sample data is received, the initial language model first performs forward calculation to obtain a prediction result, then calculates the loss function based on the difference between the prediction result and the actual target value (i.e., the actual result), and uses the backpropagation algorithm to update the parameters of the initial task adaptation matrix. The main function of this setting is to allow the initial language model to adjust the task adaptation matrix intensively in each training cycle, so as to capture the details and local features of the data and prepare for dynamically identifying the task-specific direction later.

[0041] It should be noted that the text sample data is a set of data randomly selected from the training dataset of the initial language model; the text sample data is used to represent the feature distribution of the training dataset.

[0042] Step 103: In response to determining that the training step reaches the preset cycle boundary, determine the current sensitivity in each singular vector direction of the updated task adaptation matrix in the current training cycle.

[0043] Specifically, the singular vector direction is a matrix component generated by the outer product operation of the left singular vector and the corresponding right singular vector, as follows: Consider an updated task adaptation matrix A ∈ R n×m , where n < m, which can be represented by singular value decomposition (Singular Value Decomposition, abbreviated as SVD) as A = U∑V T . Here, ∑ = diag(σ1,..., σ n ) is a diagonal matrix of singular values, and U and V are the left singular vector matrix and the right singular vector matrix respectively.

[0044] The above singular value decomposition can be further expressed as: Among them, u i represents the i-th left singular vector in the left singular vector matrix, v i represents the i-th right singular vector in the right singular vector matrix, and σi represents the i-th singular value in the singular value diagonal matrix. In addition, define u i v i is the direction of the singular vector, σ i is the scaling factor.

[0045] Step 104 : determining a sensitive direction from a plurality of singular vector directions according to the current sensitivity, and performing regularization processing on a scaling coefficient associated with the sensitive direction to obtain a regularized loss function.

[0046] In specific implementations, sensitive directions are change-sensitive directions. These refer to core directions sensitive to parameter updates identified during the initial language model training process based on a dynamic evaluation mechanism. These change-sensitive directions quantify the sensitivity to parameter updates by calculating the change sensitivity γ of the singular vector directions within the training cycle. By focusing on sensitive directions for parameter updates, the effectiveness of language model optimization can be improved, enabling efficient task adaptation while maintaining pre-trained knowledge. The scaling factor controls the contribution of each singular vector direction to the model update and determines the extent to which the associated singular vector direction adjusts the model parameters during training. By dynamically adjusting the scaling factor, the language model training process can be effectively optimized and unnecessary overfitting can be reduced.

[0047] Step 105: In response to determining that the sensitivity of the change in the singular vector direction within a continuous preset number of training cycles is in an attenuated state, a reset probability is determined according to a target attenuation period of continuous attenuation, and the scaling factor is parameter reset according to the reset probability to obtain an updated scaling factor.

[0048] In practice, the adaptive parameter reset mechanism effectively addresses the issue of parameter update stagnation by dynamically monitoring sensitivity decay patterns and applying targeted perturbations. When the sensitivity in a singular vector direction exhibits a sustained decay, it indicates that the corresponding singular vector direction may be trapped in a local optimum or a vanishing gradient state. In this case, a probabilistic reset strategy is used to restore parameter vitality. While maintaining the integrity of the matrix structure, the adaptive parameter reset mechanism introduces controllable randomness to enhance the language model's ability to escape suboptimal solutions.

[0049] Step 106: After determining that the training end condition is met, the basic feature matrix and the updated task adaptation matrix are merged, an updated language model is determined based on the updated scaling coefficient, and the updated language model is used as the target language model.

[0050] In specific implementation, the above steps 102 to 105 are looped until the training is completed, and the updated task adaptation matrix after the training is merged with the basic feature matrix to obtain an optimized target language model.

[0051] Specifically, based on the calculated gradient values, the gradient descent method is used to update the parameters of the low-rank matrix. Through repeated iterative training, the result error is continuously reduced and the performance of the language model is improved. At the same time, to prevent the model from overfitting or underfitting, regularization methods (such as L1 regularization and L2 regularization) are applied during training to constrain the numerical range or complexity of the language model parameters. When the performance of the validation set no longer improves, training is terminated to prevent overfitting.

[0052] It should be emphasized that the AdamW optimizer is used in the training phase. The AdamW optimizer can update the parameters of the language model more smoothly by clearly handling weight decay. In the original Adam optimizer, weight decay was included in the gradient calculation phase, which may cause the actual results to differ from expectations. The AdamW optimizer executes weight decay as an independent phase, which significantly reduces such potential deviations and makes the role of weight decay in model regularization more reliable and effective. In addition, because the AdamW optimizer can better control the update direction and strength of the language model parameters, it can speed up the convergence process of the language model. By integrating the efficient convergence characteristics of the Adam optimizer and the regularization effect of weight decay, the model can converge more quickly during training, thereby saving training time and resource overhead.

[0053] Through the above embodiment, text sample data is obtained, an initial language model is trained using the text sample data to obtain a weight matrix, and the weight matrix is decomposed to obtain a basic feature matrix and an initial task adaptation matrix. The initial task adaptation matrix is then updated using the text sample data to obtain an updated task adaptation matrix. This maintains the basic feature matrix while updating the initial task adaptation matrix, significantly reducing the number of adjustable parameters while preserving core pre-training knowledge. When the number of training steps reaches a preset cycle boundary, the current sensitivity of each singular vector direction in the updated task adaptation matrix for the current training cycle is determined. Based on the current sensitivity, a sensitive direction is determined from multiple singular vector directions, and the scaling coefficient associated with the sensitive direction is regularized to obtain a regularized loss function. When the sensitivity of a singular vector direction decays over a preset number of consecutive training cycles, a reset probability is determined based on a target decay period for the continuous decay, and the scaling coefficient is reset according to the reset probability to obtain an updated scaling coefficient. In this way, the singular vectors of the sensitive direction are dynamically maintained and only the scaling coefficient associated with the sensitive direction is optimized. Furthermore, the parameters of the direction with continuously decaying sensitivity are reset according to the reset probability. After determining that the training termination conditions have been met, the base feature matrix and the updated task adaptation matrix are merged, and an updated language model is determined based on the updated scaling factor. This updated language model is then used as the target language model. This reduces the complexity and computational cost of the language model while effectively avoiding local optimality and improving the target language model's focus on specific tasks, convergence speed, and generalization performance.

[0054] In some embodiments, step 101 includes:

[0055] Step 1011 : Decompose the weight matrix to obtain an orthogonal matrix and an upper triangular matrix, and sum the element energies of the diagonal elements of the upper triangular matrix to obtain a total energy.

[0056] Step 1012: Determine a preset number of diagonal elements from the diagonal elements of the upper triangular matrix in descending order.

[0057] Step 1013: In response to determining that the ratio between the first energy of the preset number of first diagonal elements and the total energy is a preset energy ratio, constructing a basic characteristic matrix based on the first diagonal elements.

[0058] Step 1014: Determine second diagonal elements other than the first diagonal elements from the diagonal elements of the upper triangular matrix, and construct an initial task adaptation matrix based on the second diagonal elements.

[0059] During implementation, the pre-trained weight matrix of the initial language model is obtained. Specifically, the language model is trained based on a large amount of text sample data, which is converted into a matrix form. The pre-trained weight matrix refers to the parameter matrix of the language model pre-trained on large-scale text sample data. The pre-trained weight matrix contains the patterns, structures, and relationships learned by the language model.

[0060] Perform rank-revealing QR factorization on the pre-trained weight matrix. Specifically, the pre-trained weight matrix is decomposed using the column-permuted rank-revealing QR factorization (RRQR). Rank-revealing QR factorization is a core algorithm in numerical linear algebra that can decompose any matrix into the product of three specific matrices: an orthogonal matrix Q, an upper triangular matrix R, and a column permutation matrix π. Its mathematical expression is: W pre · Π=Q·R, where W pre ∈R m×n , Q∈R m×m , R∈R m×n , Π∈R n×n .

[0061] It's important to note that rank-revealing QR decomposition identifies the core subspace of the pre-trained weight matrix through column permutation. This allows the weight matrix to be arranged in descending order according to its energy distribution, effectively distinguishing between key and minor parameters in the initial language model. This decomposition method is numerically more stable than standard QR decomposition and can reveal the inherent structure of the matrix with lower computational complexity, providing a theoretical basis for language model compression and parameter freezing strategies.

[0062] According to the energy distribution threshold, the task adaptation matrix and the basic feature matrix are determined. Specifically, the total energy of all diagonal elements is calculated Among them, σ i Represents the i-th diagonal element in the upper triangular matrix; determine the column order in the pre-training weight matrix according to the upper triangular matrix, and select the first k columns in descending order so that the cumulative energy E corresponding to the first diagonal element in the upper triangular matrix R is k Total energy E total The corresponding weight parameters constitute the frozen basic feature matrix; the remaining weight parameters constitute the updateable neglect task adaptation matrix for task update.

[0063] It should be understood that the basic feature matrix concentrates the core features with the highest energy ratio in the pre-training weight matrix. The features and information contained are the most important patterns and structures learned after training with a large amount of data, which are critical to the performance of the language model on most tasks. Therefore, freezing the basic feature matrix ensures that the language model can retain the knowledge learned during the pre-training process and avoids the loss of these basic features during the fine-tuning process. The task adaptation matrix contains detailed features with lower energy, representing the remaining part of the language model, that is, smaller features that are less involved in the pre-training process. This part is usually related to task-specific adaptability, which means that the task adaptation matrix can better reflect the specific needs of the task or field. By updating the initial task adaptation matrix, fine-tuning can be performed for specific tasks without affecting the core pre-training information of the language model.

[0064] It should be understood that the preset energy ratio can be dynamically calibrated based on the validation set performance. Specifically, when the target task data distribution is significantly different from the pre-training data, the preset energy ratio is reduced to expand the coverage of the task adaptation matrix; otherwise, the preset energy ratio is increased to enhance knowledge retention. For example, the preset energy ratio can be set to 95%, that is, the cumulative energy E of the first diagonal element in the upper triangular matrix R corresponding to the basic feature matrix is k Total energy E total 95% of.

[0065] Through the above scheme, rank-revealing QR decomposition identifies the core subspace of the pre-trained weight matrix through column permutation operations. The weight matrix can be arranged in order from high to low energy distribution, effectively distinguishing between key parameters and minor parameters in the initial language model. This decomposition method is numerically more stable than the standard QR decomposition and can reveal the intrinsic structure of the matrix with lower computational complexity, providing a theoretical basis for language model compression and parameter freezing strategies. By updating the initial task adaptation matrix, fine-tuning can be performed for specific tasks without affecting the core pre-trained information of the language model.

[0066] In some embodiments, step 102 includes:

[0067] Step 1021 : Perform forward operation processing on the text sample data using the initial language model to obtain a prediction result, and determine a loss function based on the prediction result and the actual result.

[0068] Step 1022: Using a back-propagation algorithm, the initial task adaptation matrix is updated according to the loss function to obtain an updated task adaptation matrix.

[0069] During specific implementation, a copy of the initial task adaptation matrix is saved at the initial moment of each preset training cycle, and only the initial task adaptation matrix is updated within the preset training cycle. Specifically, at the beginning of training, the parameters of the basic feature matrix are frozen, and at this time, only the parameters of the initial task adaptation matrix are allowed to be updated within the preset training cycle. When the text sample data is input, the initial language model first performs a forward calculation to obtain the prediction result, and then calculates the loss function based on the difference between the prediction result and the actual target value (ie, the actual result), and uses the backpropagation algorithm to update the parameters of the initial task adaptation matrix. The main purpose of this setting is to allow the initial language model to centrally adjust the task adaptation matrix in each training cycle in order to capture the details and local features of the data, in preparation for the subsequent dynamic recognition of the specific direction of the task.

[0070] When updating the initial task adaptation matrix, the singular value decomposition form must be maintained. The specific expression is:

[0071]

[0072] Among them, W res To update the task adaptation matrix, U res is the left singular vector matrix of the updated task adaptation matrix, ∑ res is the singular value diagonal matrix of the updated task adaptation matrix, The transpose matrix of the right singular vector matrix of the task adaptation matrix is used to update the task adaptation matrix. This strategy of maintaining the SVD form ensures the orthogonality of the weight update and improves the stability and efficiency of the training process.

[0073] It should be noted that singular value decomposition is an algorithm widely used in data science and machine learning. It can decompose a matrix into three specific matrix products: an orthogonal matrix, a diagonal matrix (the elements on the diagonal are called singular values, arranged from largest to smallest), and the transposed matrix of another orthogonal matrix. The decomposition process satisfies the following expression:

[0074] U·diag(S)·V T =SVD(W)

[0075] Where W∈R m×n , U∈R m×min(m,n) , V∈R n×min(m,n) , S∈R min(m,n) , W is the pre-training weight matrix, U is the orthogonal matrix containing the left singular vector, V T is the transposed matrix of the orthogonal matrix containing the right singular vectors, and S is the diagonal matrix containing the singular value vectors.

[0076] Through this approach, the initial language model performs forward operations on text sample data to obtain predictions, and a loss function is determined based on the predicted and actual results. Using a backpropagation algorithm, the initial task adaptation matrix is updated based on the loss function to produce an updated task adaptation matrix. This allows the initial language model to centrally adjust the task adaptation matrix during each training cycle to capture data details and local features, preparing for subsequent dynamic recognition of specific task directions. Maintaining the singular value decomposition form when updating the initial task adaptation matrix ensures orthogonality in weight updates, improving the stability and efficiency of the training process.

[0077] In some embodiments, step 103 includes:

[0078] Step 1031: Obtain the number of training steps in the current training cycle.

[0079] Step 1032 : In response to determining that the number of training steps reaches a preset cycle boundary, obtaining a previous sensitivity of each singular vector direction in a previous task adaptation matrix in a previous training cycle.

[0080] Step 1033: Determine the initial left singular matrix, initial singular value diagonal matrix, and initial right singular matrix of the initial task adaptation matrix, and perform calculations on the initial left singular matrix, the initial singular value diagonal matrix, the initial right singular matrix, and the update task adaptation matrix to obtain the current rate of change of each singular vector direction in the update task adaptation matrix of the current training cycle.

[0081]

[0082] Among them, δ i is the current rate of change of the direction of the i-th singular vector, is the transposed matrix of the i-th left singular vector in the left singular matrix, W res is the update task adaptation matrix, v i is the i-th right singular vector in the right singular matrix, σ i is the i-th diagonal element in the initial task adaptation matrix, and ε is the coefficient.

[0083] Step 1034: Determine the current sensitivity of each singular vector direction in the updated task adaptation matrix of the current training cycle based on the previous sensitivity and the current rate of change.

[0084] γ i (t) = λδ i (t)+(1-λ)γ i (t-1)

[0085] Among them, γ i(t) is the current sensitivity of the direction of the i-th singular vector in the updated task adaptation matrix of the t-th training cycle, λ is the smoothing factor, δ i (t) is the current rate of change of the direction of the i-th singular vector in the updated task adaptation matrix of the t-th training cycle, γ i (t-1) is the previous sensitivity of the direction of the i-th singular vector in the updated task adaptation matrix in the t-1-th training cycle.

[0086] In specific implementation, the previous task adaptation matrix saved at the beginning of the current training cycle, the updated task adaptation matrix at the end of the current training cycle, and the previous sensitivity of each singular vector direction in the previous training cycle are obtained. If the current training cycle is the first training cycle of the training process, the previous sensitivity of each singular vector direction in the previous training cycle is 0.

[0087] Calculate the rate of change of each singular vector direction of the updated task adaptation matrix. In some embodiments, the rate of change of each singular vector direction reflects the dynamic evolution intensity of the singular vector space during the training process, which is measured by calculating the projection of the task adaptation matrix change on the singular vector during the training cycle and performing singular value normalization. Specifically, through singular value decomposition, the left singular matrix, singular value diagonal matrix and right singular matrix of the previous task adaptation matrix saved at the initial moment of the current training cycle are obtained, and the current rate of change of each singular vector direction is calculated. The expression is:

[0088]

[0089] Among them, δ i is the current rate of change of the direction of the i-th singular vector, u i is the i-th left singular vector in the left singular matrix, v i is the i-th right singular vector in the right singular matrix, σ i The i-th diagonal element in the initial task adaptation matrix (i.e., the i-th singular value in the diagonal matrix), W res is the adaptation matrix of the current task at the end of the current training cycle, and v is a small constant set to prevent division by zero.

[0090] According to the current change rate of each singular vector direction of the updated task adaptation matrix and the previous sensitivity of each singular vector direction in the previous training cycle, the current sensitivity of each singular vector direction in the current training cycle is calculated.

[0091] In some embodiments, the current sensitivity of each singular vector direction is calculated by exponential weighted average based on the current change rate of each singular vector direction of the updated task adaptation matrix and the previous sensitivity of each singular vector direction in the previous cycle. The specific expression is:

[0092] γi (t) = λδ i (t)+(1-λ)γ i (t-1)

[0093] Among them, γ i (t) represents the sensitivity to changes in the direction of the i-th singular vector in the t-th training cycle, δ i (t) represents the current rate of change calculated in the direction of the i-th singular vector in the t-th preset period. λ is a smoothing factor, typically set between 0 and 1, to control the impact of historical information on the sensitivity of the current change. For example, the initial value of the smoothing factor is 0.9.

[0094] It's important to note that exponentially weighted averaging (EWA) is a data processing method widely used in time series analysis and machine learning optimization. It achieves smooth trend capture by dynamically decaying weights on historical data. The core of the EWA method is to assign exponentially decaying weights to new observations and historical data, enabling rapid response to recent changes while maintaining long-term stability. Here, EWA allows recent changes to have a greater impact on sensitivity calculations while preserving historical information, thereby more accurately capturing dynamic changes.

[0095] Through the above scheme, the current sensitivity of each singular vector direction in the updated task adaptation matrix of the current training cycle is determined based on the previous sensitivity and the current change rate, which can both quickly respond to recent changes and maintain long-term stability.

[0096] In some embodiments, step 104 includes:

[0097] Step 1041: Determine a preset number of sensitive directions with the highest current sensitivity from multiple singular vector directions based on the current sensitivity, retain the sensitive directions, and perform regularization processing on the scaling coefficients associated with the sensitive directions to obtain a regularized loss function.

[0098]

[0099] in, is the regularization loss function, σ i (t) is the i-th singular value in the updated task adaptation matrix of the t-th training cycle (i.e., the scaling coefficient in the updated task adaptation matrix of the t-th training cycle), is the L2 norm regularization term acting on σ i (t), η i (t) is the adaptive regularization coefficient of the t-th training cycle, γ i (t) is the current sensitivity of the direction of the i-th singular vector in the t-th training cycle, is the i-th singular value in the initial task adaptation matrix (i.e., the scaling coefficient in the initial task adaptation matrix), ||∑ (0) || F is the Frobenius norm of the second diagonal elements in the initial task adaptation matrix.

[0100] In specific implementation, the change-sensitive direction (i.e., sensitive direction) is a core direction subset of the pre-trained weight matrix W. The change-sensitive direction is related to each task, which means that the change-sensitive direction is different in different tasks, but for a specific task, the change-sensitive direction is fixed. It has the following characteristics: it is strongly related to the training process, and the directional sensitivity will be re-evaluated in each training cycle; the sensitivity will dynamically evolve with the training stage, and the highly sensitive direction may change in different training stages; the singular vector direction associated with the smaller singular value is more easily identified as the change-sensitive direction, because the scaling coefficient corresponding to the sensitive direction has a larger adjustment space in the parameter space. The task adaptation matrix constructed before training provides a structural basis for capturing these dynamically changing sensitive directions by retaining the directions corresponding to the smaller singular values.

[0101] In some embodiments, based on the change sensitivity of each singular vector direction in the current training cycle, multiple directions with the highest change sensitivity are selected as change-sensitive directions. The singular vectors of the selected change-sensitive directions are frozen during training to maintain the stability of these directions. At the same time, the scaling factors associated with these frozen singular vectors are updated to increase or decrease the contribution of specific directions.

[0102] It should be understood that the change-sensitive direction with the highest change sensitivity represents the task features that were not emphasized in pre-training. Paying attention to the change-sensitive direction helps to better capture the key features in the task and improve the performance of the language model on new tasks.

[0103] Change-sensitive directions represent core directions sensitive to parameter updates identified during language model training based on a dynamic evaluation mechanism. These directions are quantified by calculating the change sensitivity γ of the singular vector directions within a training cycle to quantify their sensitivity to parameter updates. By focusing on these highly sensitive directions for parameter updates, the effectiveness of language model optimization can be improved, enabling efficient task adaptation while preserving pretrained knowledge. The scaling factor controls the contribution of each singular vector direction to model updates and determines the extent to which the associated singular vector direction adjusts the language model parameters during training. By dynamically adjusting the scaling factor, the language model training process can be effectively optimized and unnecessary overfitting can be reduced.

[0104] In some embodiments, a regularization constraint is imposed on the scaling factor according to the change sensitivity and the regularization strength is dynamically adjusted to suppress excessive changes in the scaling factor. The specific expression is:

[0105]

[0106] in, is the regularization loss function, η i (t) is the adaptive regularization coefficient, which determines the regularization strength applied to the scaling coefficient in the direction of the i-th singular vector in the t-th training cycle. i (t) is the sensitivity to the change in the direction of the i-th singular vector in the t-th training cycle, is the i-th singular value of the singular value diagonal matrix obtained after the singular value decomposition of the initial task adaptation matrix, ||Σ (0) || F represents the Frobenius norm of the singular values in the singular value diagonal matrix obtained after the singular value decomposition of the initial task adaptation matrix, represents the singular value σ of the L2 regularization term acting on the t-th training cycle i (t).

[0107] This approach determines a preset number of sensitive directions with the highest current sensitivity from multiple singular vector directions based on the current sensitivity. These directions are then retained, and the scaling coefficients associated with these directions are regularized to produce a regularized loss function. This approach allows sensitive directions to represent task features that were not emphasized in pre-training. Focusing on these sensitive directions helps better capture key features in the task and improve the language model's performance on new tasks. By dynamically adjusting the scaling coefficients, the language model training process can be effectively optimized, reducing unnecessary overfitting.

[0108] In some embodiments, step 105 includes:

[0109] Step 1051 : Compare the current sensitivity with the previous sensitivity, wherein the previous sensitivity is the sensitivity of the change in the direction of each singular vector in the previous task adaptation matrix of the previous training cycle.

[0110] Step 1052: In response to determining that the current sensitivity is greater than the previous sensitivity, the recorded current attenuation period is set to zero.

[0111] Step 1053: In response to determining that the current sensitivity is less than the previous sensitivity, the recorded current attenuation period is updated to obtain a target attenuation period.

[0112] Step 1054: In response to determining that the target decay period reaches a preset period threshold, determining a reset probability according to the target decay period.

[0113]

[0114] Among them, p i is the reset probability, p base is the baseline probability, k is the attenuation coefficient, c i is the target attenuation period for the continuous attenuation of the sensitivity to changes in the direction of the i-th singular vector, is the i-th singular value in the initial task adaptation matrix, ||∑ (0) || F is the Frobenius norm of the second diagonal elements in the initial task adaptation matrix.

[0115] In specific implementation, when the current sensitivity of the singular vector direction in the current training cycle is less than the previous sensitivity in the previous training cycle, the recorded current attenuation period is increased by one. When the accumulated target attenuation period is greater than the preset period threshold, the adaptive parameter reset mechanism is triggered.

[0116] In some embodiments, when it is detected that the current sensitivity of the i-th singular vector direction in the current training cycle is lower than the previous sensitivity in the previous training cycle, the decay counter of the i-th singular vector direction performs an accumulation operation; otherwise, the decay counter is reset. When the target decay period is greater than a preset period threshold, the adaptive parameter reset evaluation process for the i-th singular vector direction is triggered.

[0117] Calculate the reset probability. Specifically, the reset probability is calculated based on the current decay state. The specific expression is: Among them, p i is the reset probability of the i-th singular value, p base is the baseline probability, k is the decay coefficient, is the i-th singular value of the singular value diagonal matrix obtained after the singular value decomposition of the initial task adaptation matrix, ||Σ (0) || F Represents the Frobenius norm of the singular values in the singular value diagonal matrix obtained after the singular value decomposition of the initial task adaptation matrix.

[0118] It should be understood that the above formula is based on the base probability p base As a benchmark, using the exponential decay term According to the stagnation time of the continuous decrease of the singular vector sensitivity, the probability of adjusting the direction of the corresponding singular vector is dynamically adjusted. Maintaining stability in the high-energy direction achieves a dynamic balance between directional sensitivity and task adaptation. This combination achieves a dynamic balance between stability and exploration, actively releasing the constraints of the local optimal direction while preventing random perturbations from destroying the core parameter distribution.

[0119] This approach accurately identifies local optima or vanishing gradient states in the corresponding singular vector direction when the target decay period reaches a preset threshold. By determining the reset probability based on the target decay period, the probability of adjusting the corresponding singular vector direction can be dynamically adjusted while maintaining stability in the high-energy direction. This balance between directional sensitivity and task adaptation is achieved, achieving a dynamic balance between stability and explorative potential. This allows for the proactive release of constraints on the local optimal direction without disrupting the core parameter distribution due to random perturbations.

[0120] In some embodiments, step 105 includes:

[0121] Step 1055, obtain a random number.

[0122] Step 1056, in response to determining that the reset probability is greater than or equal to the random number, restore the regularized loss function to the scaling factor, and determine the initial left singular vector and the initial right singular vector from the scaling factor.

[0123] Step 1057: adding uniform noise to the initial left singular vector to obtain an updated left singular vector, and adding uniform noise to the initial right singular vector to obtain an updated right singular vector.

[0124]

[0125] in, is the updated left singular vector, u i is the initial left singular vector, δ is the perturbation intensity, N(0,I) represents random noise with variance 0 and mean I, I is the orthogonalized matrix, is the updated right singular vector, v i is the initial right singular vector.

[0126] Step 1058: construct an updated scaling factor based on the updated left singular vector and the updated right singular vector.

[0127] In a specific implementation, when the random number is less than the reset probability, a random parameter reset operation is performed. In some embodiments, a random number ξ~U(0,1) is generated, and when the reset probability of the i-th singular value is greater than or equal to the random number, a parameter reset operation is performed, specifically including:

[0128] The scaling factor σ i Restore to initial value The noise is uniform and the noise range is limited to ±10% of the initial value. This is conducive to introducing randomness without destroying the original structure, helping to jump out of the local optimum and avoiding excessive disturbances that affect stability. i and the right singular vector vi Apply Gaussian noise perturbation and randomly adjust the direction of the singular vector on the unit hypersphere to force it out of the local optimal area. The specific expression is:

[0129]

[0130] in, is the left singular vector after applying Gaussian noise perturbation, is the right singular vector after applying Gaussian noise perturbation, δ is the perturbation intensity; for the perturbation U adapt and U adapt Perform Gram-Schmidt orthogonalization on the matrix, keeping and

[0131] It should be noted that Gram-Schmidt orthogonalization is a numerical method that converts a group of linearly independent vectors into a group of orthogonal unit vectors. It is used here to eliminate the non-orthogonality deviation between vectors caused by noise disturbances and ensure that the task adaptation matrix always satisfies the mathematical constraints of singular value decomposition.

[0132] Through the above scheme, when the reset probability is greater than or equal to the random number, the regularized loss function is restored to the scaling factor. This helps introduce randomness without destroying the original structure, helping to escape the local optimum while avoiding excessive perturbations that affect stability. The initial left and right singular vectors are determined from the scaling factor. Uniform noise is added to the initial left singular vector to obtain the updated left singular vector. Uniform noise is added to the initial right singular vector to obtain the updated right singular vector. The updated scaling factor is constructed based on the updated left and updated right singular vectors, thereby accurately obtaining the updated scaling factor.

[0133] Through the above embodiment, the pre-trained weight matrix is divided into a frozen basic feature matrix and an updateable task adaptation matrix through singular value energy distribution, which significantly reduces the scale of adjustable parameters while retaining the core pre-training knowledge. Furthermore, a copy of the task adaptation matrix is saved at the initial stage of each training cycle and is only updated within the cycle, providing a stable reference benchmark for subsequent dynamic evaluation, thereby achieving a balance between knowledge solidification and task adaptation. Furthermore, through the evaluation and attenuation detection of singular direction change sensitivity at the cycle boundary, the singular vectors in the highly sensitive direction are dynamically frozen and only the associated scaling coefficients are optimized. At the same time, a parameter reset operation is performed on the direction of continuous sensitivity attenuation according to a preset probability, the scaling coefficient is restored to the initial value and a controllable noise perturbation is applied. Subsequently, orthogonalization processing is performed to maintain the integrity of the matrix structure. Finally, the optimized language model is obtained through matrix reconstruction, thereby effectively avoiding the local optimal trap while reducing model complexity and computational cost, and improving the language model's focus on specific tasks, convergence speed, and generalization performance.

[0134] It should be noted that the method of the embodiments of the present disclosure can be performed by a single device, such as a computer or server. The method of the embodiments of the present disclosure can also be applied in a distributed scenario, where multiple devices cooperate to perform the method. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of the embodiments of the present disclosure, and the multiple devices will interact with each other to complete the method.

[0135] It should be noted that the above description is limited to some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that described in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0136] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides a language model training device.

[0137] refer to Figure 2 , the language model training device comprises:

[0138] A decomposition processing module 201 is configured to obtain text sample data, train an initial language model using the text sample data to obtain a weight matrix, and decompose the weight matrix to obtain a basic feature matrix and an initial task adaptation matrix;

[0139] An update processing module 202 is configured to update the initial task adaptation matrix using the text sample data to obtain an updated task adaptation matrix;

[0140] a sensitivity determination module 203 configured to determine a current sensitivity of each singular vector direction in the updated task adaptation matrix in a current training cycle in response to determining that the number of training steps reaches a preset cycle boundary;

[0141] A regularization processing module 204 is configured to determine a sensitive direction from a plurality of singular vector directions according to the current sensitivity, and perform regularization processing on a scaling coefficient associated with the sensitive direction to obtain a regularized loss function;

[0142] A parameter resetting module 205 is configured to, in response to determining that the sensitivity of the singular vector direction to the change is in an attenuated state within a continuous preset number of training cycles, determine a reset probability according to a target attenuation period of continuous attenuation, and reset the scaling factor according to the reset probability to obtain an updated scaling factor;

[0143] The model determination module 206 is configured to merge the basic feature matrix and the updated task adaptation matrix after determining that the training end condition is met, determine an updated language model based on the updated scaling coefficient, and use the updated language model as the target language model.

[0144] In some embodiments, the decomposition processing module 201 includes:

[0145] a decomposition unit configured to decompose the weight matrix to obtain an orthogonal matrix and an upper triangular matrix, and sum the element energies of the diagonal elements of the upper triangular matrix to obtain a total energy;

[0146] a sorting unit configured to determine a preset number of diagonal elements from the diagonal elements of the upper triangular matrix in descending order;

[0147] a basic characteristic matrix determining unit configured to construct a basic characteristic matrix based on the first diagonal elements in response to determining that a ratio between a first energy of the preset number of first diagonal elements and a total energy is a preset energy ratio;

[0148] The initial task adaptation matrix determining unit is configured to determine second diagonal elements other than the first diagonal elements from the diagonal elements of the upper triangular matrix, and construct an initial task adaptation matrix based on the second diagonal elements.

[0149] In some embodiments, the update processing module 202 includes:

[0150] a loss function determining unit configured to perform a forward operation on the text sample data using the initial language model to obtain a prediction result, and determine a loss function based on the prediction result and an actual result;

[0151] The updated task adaptation matrix determining unit is configured to update the initial task adaptation matrix according to the loss function through a back propagation algorithm to obtain an updated task adaptation matrix.

[0152] In some embodiments, the sensitivity determination module 203 includes:

[0153] A training step number obtaining unit is configured to obtain the number of training steps in a current training cycle;

[0154] a previous sensitivity determining unit, configured to, in response to determining that the number of training steps reaches a preset cycle boundary, obtain a previous sensitivity of each singular vector direction in a previous task adaptation matrix in a previous training cycle;

[0155] A current change rate determination unit is configured to determine an initial left singular matrix, an initial singular value diagonal matrix, and an initial right singular matrix of the initial task adaptation matrix, and perform calculations on the initial left singular matrix, the initial singular value diagonal matrix, the initial right singular matrix, and the update task adaptation matrix to obtain a current change rate of each singular vector direction in the update task adaptation matrix of the current training cycle.

[0156]

[0157] Among them, δ i is the current rate of change of the direction of the i-th singular vector, is the transposed matrix of the i-th left singular vector in the left singular matrix, W res is the update task adaptation matrix, v i is the i-th right singular vector in the right singular matrix, σ i is the i-th diagonal element in the initial task adaptation matrix, and ε is the coefficient;

[0158] a current sensitivity determination unit configured to determine the current sensitivity of each singular vector direction in the updated task adaptation matrix of the current training cycle according to the previous sensitivity and the current rate of change;

[0159] γ i (t) = λδ i (t)+(1-λ)γ i (t-1)

[0160] Among them, γ i (t) is the current sensitivity of the direction of the i-th singular vector in the updated task adaptation matrix of the t-th training cycle, λ is the smoothing factor, δ i (t) is the current rate of change of the direction of the i-th singular vector in the updated task adaptation matrix of the t-th training cycle, γ i (t-1) is the previous sensitivity of the direction of the i-th singular vector in the updated task adaptation matrix in the t-1-th training cycle.

[0161] In some embodiments, the regularization processing module 204 includes:

[0162] A regularized loss function determining unit is configured to determine a preset number of sensitive directions with the highest current sensitivity from a plurality of singular vector directions according to the current sensitivity, maintain the sensitive directions, and perform regularization processing on scaling coefficients associated with the sensitive directions to obtain a regularized loss function.

[0163]

[0164] in, is the regularization loss function, σ i (t) is the i-th singular value in the updated task adaptation matrix for the t-th training cycle, is the L2 norm regularization term acting on σ i (t), η i (t) is the adaptive regularization coefficient of the t-th training cycle, γ i (t) is the current sensitivity of the direction of the i-th singular vector in the t-th training cycle, is the i-th singular value in the initial task adaptation matrix, ||∑ (0) || F is the Frobenius norm of the second diagonal elements in the initial task adaptation matrix.

[0165] In some embodiments, the parameter resetting module 205 includes:

[0166] a comparison processing unit configured to compare the current sensitivity with a previous sensitivity, wherein the previous sensitivity is a change sensitivity of each singular vector direction in the previous task adaptation matrix of the previous training cycle;

[0167] a current attenuation period setting unit, configured to, in response to determining that the current sensitivity is greater than the previous sensitivity, set the recorded current attenuation period to a value of zero;

[0168] a current attenuation period updating unit, configured to, in response to determining that the current sensitivity is less than the previous sensitivity, update the recorded current attenuation period to obtain a target attenuation period;

[0169] a reset probability determining unit configured to, in response to determining that the target attenuation period reaches a preset period threshold, determine a reset probability according to the target attenuation period,

[0170]

[0171] Among them, p i is the reset probability, p base is the baseline probability, κ is the attenuation coefficient, c i is the target attenuation period for the continuous attenuation of the sensitivity to changes in the direction of the i-th singular vector, is the i-th singular value in the initial task adaptation matrix, ||∑ (0) || F is the Frobenius norm of the second diagonal elements in the initial task adaptation matrix.

[0172] In some embodiments, the parameter resetting module 205 includes:

[0173] A random number acquisition unit, configured to acquire a random number;

[0174] a restoration unit configured to, in response to determining that the reset probability is greater than or equal to a random number, restore the regularized loss function to the scaling factor, and determine an initial left singular vector and an initial right singular vector from the scaling factor;

[0175] an adding unit configured to add uniform noise to the initial left singular vector to obtain an updated left singular vector, and to add uniform noise to the initial right singular vector to obtain an updated right singular vector,

[0176]

[0177] in, is the updated left singular vector, u i is the initial left singular vector, δ is the perturbation intensity, N(0,I) represents random noise with variance 0 and mean I, I is the orthogonalized matrix, is the updated right singular vector, v i is the initial right singular vector;

[0178] A construction unit is configured to construct an updated scaling coefficient based on the updated left singular vector and the updated right singular vector.

[0179] For the convenience of description, the above devices are described as being functionally divided into various modules. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0180] The apparatus of the above embodiment is used to implement the corresponding language model training method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0181] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the program, the language model training method described in any of the above embodiments is implemented.

[0182] Figure 3 10 is a schematic diagram showing a more specific hardware structure of an electronic device provided in this embodiment. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are communicatively connected to each other within the device via the bus 1050.

[0183] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0184] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 1020 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0185] The input / output interface 1030 is used to connect input / output modules to implement information input and output. The input / output modules can be configured as components within the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0186] The communication interface 1040 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (e.g., USB (Universal Serial Bus), network cable, etc.) or a wireless method (e.g., mobile network, WIFI (Wireless Fidelity), Bluetooth, etc.).

[0187] The bus 1050 comprises a path for transmitting information between the various components of the device (eg, the processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 ).

[0188] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may only include the components necessary to implement the embodiments of this specification, and does not necessarily include all the components shown in the figure.

[0189] The electronic device of the above embodiment is used to implement the corresponding language model training method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0190] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the language model training method described in any of the above embodiments.

[0191] The computer-readable media of this embodiment include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0192] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the language model training method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0193] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a computer program product, including computer program instructions. When the computer program instructions are run on a computer, the computer executes the language model training method described in any of the above embodiments, which has the beneficial effects of the corresponding method embodiments and will not be repeated here.

[0194] It is understandable that before using the technical solutions of each embodiment of the present disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0195] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operation of the disclosed technical solution based on the prompt message.

[0196] As an optional but non-limiting implementation, in response to a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0197] It is understandable that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0198] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of simplicity.

[0199] In addition, to simplify the description and discussion, and so as not to obscure the embodiments of the present disclosure, known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, devices may be shown in the form of block diagrams to avoid obscuring the embodiments of the present disclosure, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present disclosure are to be implemented (i.e., these details should be fully within the purview of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present disclosure, it will be apparent to those skilled in the art that the embodiments of the present disclosure may be implemented without these specific details or with variations in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0200] Although the present disclosure has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0201] The embodiments of the present disclosure are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the present disclosure. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A language model training method, characterized in that: The method comprises: Acquire text sample data, use the text sample data to train an initial language model to obtain a weight matrix, and decompose the weight matrix to obtain a basic feature matrix and an initial task adaptation matrix; Using the text sample data, the initial task adaptation matrix is updated to obtain an updated task adaptation matrix; In response to determining that the number of training steps reaches a preset cycle boundary, determining a current sensitivity of each singular vector direction in the updated task adaptation matrix in the current training cycle; Determining a sensitive direction from a plurality of singular vector directions according to the current sensitivity, and performing regularization processing on a scaling coefficient associated with the sensitive direction to obtain a regularized loss function; In response to determining that the sensitivity of the singular vector direction to a change is in an attenuated state within a continuous preset number of training cycles, determining a reset probability according to a target attenuation period of continuous attenuation, and resetting the scaling factor according to the reset probability to obtain an updated scaling factor; After determining that the training end condition is met, the basic feature matrix and the updated task adaptation matrix are merged, an updated language model is determined based on the updated scaling coefficient, and the updated language model is used as the target language model.

2. The method according to claim 1, characterized in that The decomposing the weight matrix to obtain a basic feature matrix and an initial task adaptation matrix includes: Decomposing the weight matrix to obtain an orthogonal matrix and an upper triangular matrix, and summing the element energies of the diagonal elements of the upper triangular matrix to obtain a total energy; Determine a preset number of diagonal elements from the diagonal elements of the upper triangular matrix in descending order; In response to determining that a ratio between the first energy of the preset number of first diagonal elements and the total energy is a preset energy ratio, constructing a basic characteristic matrix based on the first diagonal elements; Second diagonal elements other than the first diagonal elements are determined from the diagonal elements of the upper triangular matrix, and an initial task adaptation matrix is constructed based on the second diagonal elements.

3. The method according to claim 1, characterized in that The updating process of the initial task adaptation matrix using the text sample data to obtain an updated task adaptation matrix includes: Performing forward operation processing on the text sample data using the initial language model to obtain a prediction result, and determining a loss function based on the prediction result and the actual result; The initial task adaptation matrix is updated according to the loss function through a back-propagation algorithm to obtain an updated task adaptation matrix.

4. The method according to claim 1, wherein In response to determining that the number of training steps reaches a preset cycle boundary, determining a current sensitivity of each singular vector direction in the updated task adaptation matrix in the current training cycle includes: Get the number of training steps in the current training cycle; In response to determining that the number of training steps reaches a preset cycle boundary, obtaining a previous sensitivity of each singular vector direction in a previous task adaptation matrix in a previous training cycle; Determine an initial left singular matrix, an initial singular value diagonal matrix, and an initial right singular matrix of the initial task adaptation matrix, and perform calculations on the initial left singular matrix, the initial singular value diagonal matrix, the initial right singular matrix, and the update task adaptation matrix to obtain a current rate of change of each singular vector direction in the update task adaptation matrix of the current training cycle. in,- i is the current rate of change of the direction of the i-th singular vector, is the transposed matrix of the i-th left singular vector in the left singular matrix, W res is the update task adaptation matrix, v i is the i-th right singular vector in the right singular matrix, σ i is the i-th diagonal element in the initial task adaptation matrix, and ε is the coefficient; Determine the current sensitivity of each singular vector direction in the updated task adaptation matrix of the current training cycle according to the previous sensitivity and the current rate of change, c i (t)=λδ i (t)+(1-λ)γ i (t-1) Among them, γ i (t) is the current sensitivity of the direction of the i-th singular vector in the updated task adaptation matrix of the t-th training cycle, λ is the smoothing factor, δ i (t) is the current rate of change of the direction of the i-th singular vector in the updated task adaptation matrix of the t-th training cycle, γ i (t-1) is the previous sensitivity of the direction of the i-th singular vector in the updated task adaptation matrix in the t-1-th training cycle.

5. The method according to claim 1, wherein The determining of a sensitive direction from a plurality of singular vector directions according to the current sensitivity, and performing regularization processing on a scaling coefficient associated with the sensitive direction to obtain a regularized loss function, includes: Determining a preset number of sensitive directions with the highest current sensitivity from a plurality of singular vector directions according to the current sensitivity, maintaining the sensitive directions, and performing regularization processing on scaling coefficients associated with the sensitive directions to obtain a regularized loss function, in, is the regularization loss function, σ i (t) is the i-th singular value in the updated task adaptation matrix of the t-th training cycle, is the L2 norm regularization term acting on σ i (t), η i (t) is the adaptive regularization coefficient of the t-th training cycle, Y i (t) is the current sensitivity of the direction of the i-th singular vector in the t-th training cycle, is the i-th singular value in the initial task adaptation matrix, ||∑ (0) || F is the Frobenius norm of the second diagonal elements in the initial task adaptation matrix.

6. The method according to claim 1, characterized in that In response to determining that the sensitivity to changes in the singular vector directions is in an attenuated state within a continuous preset number of training cycles, determining the reset probability according to a target attenuation cycle of the continuous attenuation includes: Comparing the current sensitivity with a previous sensitivity, wherein the previous sensitivity is the sensitivity of the change in the direction of each singular vector in the previous task adaptation matrix of the previous training cycle; In response to determining that the current sensitivity is greater than the previous sensitivity, setting the recorded current decay period to a value of zero; In response to determining that the current sensitivity is less than the previous sensitivity, updating the recorded current attenuation period to obtain a target attenuation period; In response to determining that the target decay period reaches a preset period threshold, determining a reset probability according to the target decay period, Among them, p i is the reset probability, p base is the baseline probability, κ is the attenuation coefficient, c i is the target attenuation period for the continuous attenuation of the sensitivity to changes in the direction of the i-th singular vector, is the i-th singular value in the initial task adaptation matrix, ||∑ (0) || F is the Frobenius norm of the second diagonal elements in the initial task adaptation matrix.

7. The method according to claim 1, characterized in that Resetting the scaling factor according to the reset probability to obtain an updated scaling factor includes: Get a random number; In response to determining that the reset probability is greater than or equal to a random number, restoring the regularized loss function to the scaling factor, and determining an initial left singular vector and an initial right singular vector from the scaling factor; Adding uniform noise to the initial left singular vector to obtain an updated left singular vector, adding uniform noise to the initial right singular vector to obtain an updated right singular vector, in, is the updated left singular vector, u i is the initial left singular vector, δ is the perturbation intensity, N(0,I) represents random noise with variance 0 and mean I, I is the orthogonalized matrix, is the updated right singular vector, v i is the initial right singular vector; An updated scaling coefficient is constructed based on the updated left singular vector and the updated right singular vector.

8. A language model training device, characterized in that: include: a decomposition processing module configured to obtain text sample data, train an initial language model using the text sample data to obtain a weight matrix, and decompose the weight matrix to obtain a basic feature matrix and an initial task adaptation matrix; An update processing module is configured to update the initial task adaptation matrix using the text sample data to obtain an updated task adaptation matrix; a sensitivity determination module configured to determine a current sensitivity of each singular vector direction in the updated task adaptation matrix in a current training cycle in response to determining that the number of training steps reaches a preset cycle boundary; a regularization processing module, configured to determine a sensitive direction from a plurality of singular vector directions according to the current sensitivity, and perform regularization processing on a scaling coefficient associated with the sensitive direction to obtain a regularized loss function; a parameter resetting module configured to, in response to determining that the sensitivity of the singular vector direction to the change is in an attenuated state within a continuous preset number of training cycles, determine a reset probability according to a target attenuation period of continuous attenuation, and reset the scaling factor according to the reset probability to obtain an updated scaling factor; The model determination module is configured to merge the basic feature matrix and the updated task adaptation matrix after determining that the training end condition is met, determine the updated language model based on the updated scaling coefficient, and use the updated language model as the target language model.

9. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to cause a computer to execute the method according to any one of claims 1 to 7.

Images