A model training method and device based on parameter adjustment, equipment and medium

Abstract

This invention discloses a model training method, apparatus, device, and medium based on parameter adjustment, relating to the field of model training. The method includes: during the training of a text generation model using a scaling factor and regularization parameters based on a low-rank matrix, triggering a model monitoring operation to obtain the training state of the text generation model at the current training time step; determining whether to adjust the scaling factor and regularization parameters based on the training state; if so, starting from the next training time step, continuing to train the text generation model using the adjusted scaling factor and regularization parameters based on the low-rank matrix, and then returning to the step of triggering the model monitoring operation until the text generation model meets a preset termination condition, resulting in a trained model, which is then used to generate corresponding text based on input data. This invention allows for the adjustment of scaling factors and regularization parameters without interrupting model training, enabling real-time response to changes in data distribution.

Description

Technical Field

[0001] This invention relates to the field of model training, and in particular to a model training method, apparatus, device, and medium based on parameter adjustment. Background Technology

[0002] With the development of large language models, efficient parameter fine-tuning techniques have become a core solution for reducing model training costs. LoRA (Low-Rank Adaptation) achieves fine-tuning by inserting low-rank matrices into key layers of the pre-trained model (such as the attention layer and FeedForward layer of the Transformer). It can achieve task adaptation by training only a small number of parameters, reducing training costs by more than 90% compared to full fine-tuning. Therefore, it is widely used in the field of natural language processing.

[0003] The core parameters of LoRA include the scaling factor and the regularization parameter. The scaling factor controls the contribution of the low-rank matrix to the model output, while the regularization parameter controls the risk of overfitting. Both directly affect the model fine-tuning effect. In LoRA fine-tuning, if the model is trained directly on new data or a subdivided task after the initial fine-tuning converges, using the original training parameters (such as the scaling factor and regularization parameter) may lead to overfitting (when the amount of new data is small) or underfitting (low convergence efficiency) due to changes in data distribution. Therefore, related techniques usually supplement new data and adjust the training parameters after the initial training, and then start a second training (incremental fine-tuning) based on the converged model to adapt to the new scenario and avoid performance issues.

[0004] However, related technologies have the following problems when adjusting scaling factors and regularization parameters: 1. The parameters are statically configured and cannot be adjusted during training. If modifications are needed, training must be interrupted and the configuration file manually modified before training can take effect; 2. Parameter adjustment relies on human experience and is not related to model training, making it difficult to respond to changes in data distribution in real time. Therefore, how to adjust scaling factors and regularization parameters without interrupting model training to respond to changes in data distribution in real time is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a model training method, apparatus, device, and medium based on parameter adjustment, which can adjust scaling factors and regularization parameters without interrupting model training to respond in real time to changes in data distribution. The specific solution is as follows:

[0006] In a first aspect, the present invention provides a model training method based on parameter adjustment, comprising:

[0007] During the iterative training of the text generation model using scaling factors and regularization parameters and based on the iteratively updated low-rank matrix, a monitoring operation on the text generation model is triggered to obtain the training status of the text generation model at the current training time step; the low-rank matrix is ​​pre-inserted into the preset network layer of the text generation model.

[0008] Whether to adjust the scaling factor and regularization parameter is determined based on the training state;

[0009] If so, starting from the next training time step, the text generation model is iteratively trained using the adjusted scaling factor and regularization parameter and based on the iteratively updated low-rank matrix. The process then jumps back to the step that triggers the monitoring operation of the text generation model until the text generation model meets the preset termination condition, resulting in a trained model. The trained model is then used to generate corresponding text based on the input data.

[0010] Optionally, trigger monitoring operations on the text generation model to obtain the training status of the text generation model at the current training time step, including:

[0011] The monitoring operation of the text generation model is triggered periodically based on a preset time step to obtain the average updated gradient of the text generation model at the current training time step and the evaluation result of the generated text.

[0012] Among them, the updated gradient mean is used to reflect the training stability of the text generation model at this training time step; the generated text evaluation result is used to reflect the adaptability of the text generation model to the text generation task at this training time step.

[0013] Optionally, obtain the mean of the updated gradients of the text generation model at this training time step and the evaluation result of the generated text, including:

[0014] The mean of the update gradient of the low-rank matrix at the current training time step is calculated to obtain the mean of the update gradient.

[0015] Several test cases for the text generation task are input into the text generation model to obtain text generation results corresponding to the test cases respectively.

[0016] Based on the text generation results and the expected generation results of the corresponding test cases, the evaluation results of the generated text are determined.

[0017] Optionally, the generated text evaluation results may include the text output formatting error rate;

[0018] Based on the generated text results and the expected generated results of the corresponding test cases, the evaluation results of the generated text are determined, including:

[0019] The consistency of the output format of each text generation result with the expected generation result of the corresponding test case is judged to determine the target generation result from each text generation result; if the output format of the target generation result is inconsistent with the expected generation result of the corresponding test case;

[0020] The text output format error rate is determined based on the ratio between the number of target generated results and the total number of text generated results.

[0021] Optionally, determine whether to adjust the scaling factor and regularization parameters based on the training state, including:

[0022] Determine whether the updated gradient mean meets the preset mean condition; the preset mean condition includes that the updated gradient mean is greater than the preset gradient mean.

[0023] If the updated gradient mean satisfies the preset mean condition, then the scaling factor is reduced and the regularization parameter is increased using the first preset adjustment rule.

[0024] If the updated gradient mean does not meet the preset mean condition, then determine whether the text output format error rate meets the preset error rate condition; the preset error rate condition includes the text output format error rate being greater than the preset format error rate.

[0025] If the text output format error rate does not meet the preset error rate condition, it is determined that the scaling factor and regularization parameter will not be adjusted.

[0026] If the text output format error rate meets the preset error rate condition, the scaling factor is increased and the regularization parameter is decreased using the second preset adjustment rule.

[0027] Optionally, the scaling factor is reduced and the regularization parameter is increased using the first preset adjustment rule, including:

[0028] Determine the first product of the scaling factor and the first preset value, and determine the adjusted scaling factor based on the maximum value between the first product and the lower limit of the preset scaling factor; wherein the first preset value is a value less than one.

[0029] The addition result of the regularization parameter and the second preset value is determined, and the adjusted regularization parameter is determined based on the minimum value between the addition result and the upper limit of the preset regularization parameter; wherein, the second preset value is a value greater than zero and less than one.

[0030] Optionally, the scaling factor can be increased and the regularization parameter can be decreased using the second preset adjustment rule, including:

[0031] Determine the second product of the scaling factor and the third preset value, and determine the adjusted scaling factor based on the minimum value between the second product and the upper limit of the preset scaling factor; wherein the third preset value is a value greater than one.

[0032] The result of subtracting the regularization parameter from the second preset value is determined, and the adjusted regularization parameter is determined based on the maximum value between the subtraction result and the lower limit of the preset regularization parameter.

[0033] Secondly, the present invention provides a model training apparatus based on parameter adjustment, comprising:

[0034] The training state acquisition module is used to trigger monitoring operations on the text generation model during iterative training of the text generation model using scaling factors and regularization parameters and based on the iteratively updated low-rank matrix, so as to obtain the training state of the text generation model at the current training time step; the low-rank matrix is ​​pre-inserted into the preset network layer of the text generation model.

[0035] The parameter tuning module is used to determine whether to adjust the scaling factor and regularization parameter based on the training state;

[0036] The model training module is used to, if so, start from the next training time step, use the adjusted scaling factor and regularization parameters and based on the iteratively updated low-rank matrix to continue iteratively training the text generation model, and then jump back to the step that triggers the monitoring operation of the text generation model until the text generation model meets the preset termination condition, and obtain the trained model, so as to generate the corresponding text based on the input data using the trained model.

[0037] Thirdly, the present invention provides an electronic device, comprising:

[0038] Memory, used to store computer programs;

[0039] A processor is used to execute computer programs to implement the steps of the aforementioned parameter-tuning-based model training method.

[0040] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the aforementioned model training method based on parameter adjustment.

[0041] In this invention, during the iterative training of the text generation model using scaling factors and regularization parameters based on an iteratively updated low-rank matrix, a monitoring operation is triggered to obtain the training state of the text generation model at the current training time step. The low-rank matrix is ​​pre-inserted into a preset network layer of the text generation model. Based on the training state, it is determined whether to adjust the scaling factors and regularization parameters. If so, starting from the next training time step, the text generation model is iteratively trained again using the adjusted scaling factors and regularization parameters based on the iteratively updated low-rank matrix, and the process jumps back to the step that triggers the monitoring operation of the text generation model until the text generation model meets the preset termination condition, resulting in a trained model. The trained model is then used to generate corresponding text based on the input data.

[0042] Beneficial effects: This invention monitors the model during training to obtain the model's training status at each training time step. Based on the model's training status at each training time step, it automatically adjusts the scaling factor and regularization parameters to solve the problem of manual parameter adjustment and to correlate parameter adjustments with the model's training status, achieving real-time response to changes in data distribution. Furthermore, after adjusting the scaling factor and regularization parameters, this invention immediately applies the adjustments to subsequent model training without interrupting or restarting the model training. By starting model training only once, the model can be efficiently adapted to text generation tasks. Attached Figure Description

[0043] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 A flowchart of a model training method based on parameter adjustment provided in an embodiment of the present invention;

[0045] Figure 2 A flowchart of a model training process based on parameter adjustment is provided for an embodiment of the present invention;

[0046] Figure 3 A schematic diagram of a model training device based on parameter adjustment provided in an embodiment of the present invention;

[0047] Figure 4 This is a structural diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0049] The terms "comprising" and "having," and any variations thereof, in the specification and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may include steps or units not listed.

[0050] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0051] Related technologies suffer from the following problems when adjusting scaling factors and regularization parameters: 1. Parameters are statically configured and cannot be adjusted during training. If modifications are needed, training must be interrupted, the configuration file manually modified, and training restarted for the changes to take effect; 2. Parameter adjustment relies on manual experience and is not linked to model training, making it difficult to respond to changes in data distribution in real time. Therefore, this invention provides a model training method based on parameter adjustment, which allows for the adjustment of scaling factors and regularization parameters without interrupting model training, thus responding to changes in data distribution in real time.

[0052] See Figure 1 As shown, this embodiment of the invention discloses a model training method based on parameter adjustment, including:

[0053] Step S11: During the iterative training of the text generation model using scaling factors and regularization parameters and based on the iteratively updated low-rank matrix, a monitoring operation on the text generation model is triggered to obtain the training status of the text generation model at the current training time step; the low-rank matrix is ​​pre-inserted into the preset network layer of the text generation model.

[0054] In the initialization phase of this invention, a text generation model and an initial LoRA configuration are preloaded. The text generation model can be a model using the Transformer architecture. The initial LoRA configuration includes, but is not limited to, the initial values ​​of the scaling factor, the initial values ​​of the regularization parameter, and the dimension of the low-rank matrix (or the rank of the low-rank matrix). Then, the LoRA adapter is initialized so that the low-rank matrix is ​​inserted into the preset network layers (e.g., attention layers, feedforward layers, etc.) of the text generation model based on the dimension of the low-rank matrix. The scaling factor and the regularization parameter are set to their initial values, and the trainer (e.g., optimizer, learning rate scheduler, etc.) is started.

[0055] It should be noted that the forward propagation of the text generation model without LoRA can be represented as h = Wx, where h represents the model output, x represents the model input, and W represents the weight matrix with a dimension of d × k, where d and k are relatively large values, such as 4096. In this case, the number of parameters that need to be updated in each training iteration is d × k. The forward propagation of the text generation model using LoRA can be represented as h = Wx + ΔWx, where ΔW = AB (low-rank decomposition), where h represents the model output, x represents the model input, and W represents the weight matrix with a dimension of d × k. In this case, the weight matrix W is not updated during iterative training; instead, the low-rank matrices A and B are updated. The low-rank matrix A has a dimension of d × r, and the low-rank matrix B has a dimension of r × k, where r represents the dimension of the low-rank matrix, which is a relatively small value, such as 8. In this case, the number of parameters that need to be updated in each training iteration is d × r + r × k. It can be seen that by using LoRA, the number of parameters that need to be updated can be greatly reduced.

[0056] After the initialization phase, the text generation model is iteratively trained using scaling factors and regularization parameters and based on the iteratively updated low-rank matrix. During the iterative training process, monitoring operations on the text generation model are triggered to obtain the training status of the text generation model at the current training time step.

[0057] Specifically, during the iterative training of the text generation model using scaling factors and regularization parameters and based on an iteratively updated low-rank matrix, monitoring operations on the text generation model are periodically triggered based on a preset time step to obtain the mean of the updated gradients and the evaluation results of the generated text at the current training time step. The mean of the updated gradients reflects the training stability of the text generation model at the current training time step, while the evaluation results of the generated text reflect the adaptability of the text generation model to the text generation task at the current training time step.

[0058] It should be noted that the preset time step can be set according to the model training requirements and training resources. The model training requirements include model training accuracy, which is negatively correlated with the preset time step; that is, the higher the model training accuracy, the shorter the preset time step, and vice versa. Similarly, training resources are also negatively correlated with the preset time step; the more training resources available, the shorter the preset time step, and vice versa. Therefore, embodiments of the present invention can set the preset time step based on the positive and negative correlations between model training accuracy, training resources, and the preset time step, respectively, so that the setting of the preset time step is simultaneously adapted to both model training accuracy and training resources.

[0059] To obtain the mean of the update gradient of the text generation model at the current training time step and the evaluation result of the generated text, the specific steps may include: calculating the mean of the update gradient of the low-rank matrix at the current training time step to obtain the mean of the update gradient; inputting several test cases for the text generation task into the text generation model to obtain the text generation results corresponding to the test cases respectively; and determining the evaluation result of the generated text based on each text generation result and the expected generation result of the corresponding test cases.

[0060] Specifically, the mean of the updated gradient can be calculated by averaging the updated gradients of each low-rank matrix at the current training time step. It should be noted that a mean updated gradient that is too high indicates large fluctuations in parameter updates, suggesting a risk of overfitting; conversely, a mean updated gradient that is too low indicates slow parameter updates, suggesting a risk of underfitting.

[0061] In addition to calculating the mean of the updated gradient, embodiments of this invention can also calculate the variance and / or standard deviation based on the updated gradient of the low-rank matrix at the current training time step and the mean of the updated gradient. The variance and standard deviation reflect the differences in the updated gradients of different low-rank matrices. That is, when the variance and standard deviation are large, it indicates a significant difference in the updated gradients of different low-rank matrices, which may lead to the following problems: 1. Overfitting in some layers and underfitting in others; 2. Early signs of gradient explosion / vanishing; 3. The impact of uneven data distribution. When the variance and standard deviation are small, it indicates a smaller difference in the updated gradients of different low-rank matrices, with a concentrated updated gradient distribution and relatively balanced training.

[0062] Furthermore, a training state view for the text generation model can be generated based on variance and standard deviation, and the training state view can be visualized for easy viewing by users. This helps users analyze and solve problems during the development and debugging phase, and the resource overhead of calculating variance and standard deviation is also relatively small.

[0063] In one example, the evaluation results for generated text include a text output format error rate. Specifically, several test cases for the text generation task are input into the text generation model to obtain text generation results corresponding to each test case. The consistency of the output format of each text generation result with the expected generation result of the corresponding test case is judged to determine the target generation result from the text generation results. Where the output format of the target generation result is inconsistent with the expected generation result of the corresponding test case, the text output format error rate is determined based on the ratio between the number of target generation results and the total number of text generation results.

[0064] Taking the expected output format of the test case as "generate JSON (JavaScript Object Notation, a lightweight data interchange format) data containing id and name" as an example, we judge the consistency between the output format of each text generation result and the expected output format of the corresponding test case, so as to identify the target generation result with inconsistent output format from each text generation result, that is, the JSON structure of the target generation result is incomplete or incorrect; assuming that the number of target generation results is 2 and the total number of text generation results is 10, then the text output format error rate is 2 / 10=0.2.

[0065] In addition, the generated text evaluation results may also include text output format accuracy, average text matching degree, etc.

[0066] The process of determining the text output format accuracy rate includes: judging the consistency of the output format of each text generation result with the expected generation result of the corresponding test case, so as to determine the target generation result with consistent output format from each text generation result, and determining the text output format error rate based on the ratio between the number of target generation results and the total number of each text generation result.

[0067] The process of determining the average text matching degree includes: calculating the text matching degree between each text generation result and the expected generation result of the corresponding test case to obtain the text matching degree corresponding to each text generation result; and calculating the average value of each text matching degree to obtain the average text matching degree.

[0068] Step S12: Determine whether to adjust the scaling factor and regularization parameter based on the training state.

[0069] In this embodiment of the invention, after triggering the monitoring operation of the text generation model to obtain the training status of the text generation model at the current training time step, it is necessary to determine whether to adjust the scaling factor and regularization parameter based on the training status of the text generation model at the current training time step.

[0070] According to one example, embodiments of the present invention can determine whether to adjust the scaling factor and regularization parameters based on the mean of the updated gradients of the text generation model at the current training time step and the evaluation result of the generated text.

[0071] For example, embodiments of the present invention can determine whether to adjust the scaling factor and regularization parameters based on the mean of the updated gradients of the text generation model at the current training time step and the text output format error rate.

[0072] Specifically, in this embodiment of the invention, it is first determined whether the updated gradient mean meets a preset mean condition; wherein, the preset mean condition includes the updated gradient mean being greater than a preset gradient mean; if the updated gradient mean meets the preset mean condition, then the scaling factor is reduced and the regularization parameter is increased using a first preset adjustment rule; if the updated gradient mean does not meet the preset mean condition, it is determined whether the text output format error rate meets a preset error rate condition; wherein, the preset error rate condition includes the text output format error rate being greater than a preset format error rate; if the text output format error rate does not meet the preset error rate condition, then it is determined that the scaling factor and the regularization parameter will not be adjusted; if the text output format error rate meets the preset error rate condition, then the scaling factor is increased and the regularization parameter is reduced using a second preset adjustment rule.

[0073] Since the updated gradient mean reflects the training stability of the text generation model at the current training time step, and the text output format error rate reflects the adaptability of the text generation model to the text generation task at the current training time step, and training stability is more important than task adaptability, it is necessary to first determine whether to adjust the scaling factor and regularization parameters based on the updated gradient mean. If no adjustment is needed, then determine whether to adjust the scaling factor and regularization parameters based on the text output format error rate.

[0074] Furthermore, if the mean of the updated gradient is greater than the preset mean of the gradient, it indicates that the model training is unstable. It is necessary to reduce the scaling factor to weaken the adaptation signal and increase the regularization parameter to enhance regularization. If the mean of the updated gradient is not greater than the preset mean of the gradient and the text output format error rate is greater than the preset format error rate, it indicates that the task adaptability is insufficient. It is necessary to increase the scaling factor to enhance the adaptation signal and reduce the regularization parameter to reduce feature loss.

[0075] In the process of reducing the scaling factor and increasing the regularization parameter using the first preset adjustment rule, the first product of the scaling factor and the first preset value is determined, and the adjusted scaling factor is determined based on the maximum value between the first product and the preset lower limit of the scaling factor; wherein the first preset value is a value less than one; the sum of the regularization parameter and the second preset value is determined, and the adjusted regularization parameter is determined based on the minimum value between the sum and the preset upper limit of the regularization parameter; wherein the second preset value is a value greater than zero and less than one.

[0076] In the process of increasing the scaling factor and decreasing the regularization parameter using the second preset adjustment rule, the second product of the scaling factor and the third preset value is determined, and the adjusted scaling factor is determined based on the minimum value between the second product and the upper limit of the preset scaling factor; wherein, the third preset value is a value greater than one; the result of subtracting the regularization parameter from the second preset value is determined, and the adjusted regularization parameter is determined based on the maximum value between the subtraction result and the lower limit of the preset regularization parameter.

[0077] Taking a first preset value of 0.8, a second preset value of 0.05, a third preset value of 1.2, a preset scaling factor lower limit of 8, a preset scaling factor upper limit of 128, a preset regularization parameter upper limit of 0.5, and a preset regularization parameter lower limit of 0.05 as an example, if the updated gradient mean is greater than the preset gradient mean, then the adjusted scaling factor is determined based on the maximum value between the product of the scaling factor and 0.8 and 8, that is, the adjusted scaling factor = max(scaling factor × 0.8, 8). And the adjusted regularization parameter is determined based on the sum of the regularization parameter and 0.05 and the minimum value between 0.5 and 8. That is, the adjusted regularization parameter = min(regularization parameter + 0.05, 0.5); if the mean of the updated gradient is not greater than the preset mean of the gradient, and the text output format error rate is greater than the preset format error rate, then the adjusted scaling factor is determined based on the minimum value between the second product of the scaling factor and 1.2 and 128, that is, the adjusted scaling factor = min(scaling factor × 1.2, 128), and the adjusted regularization parameter is determined based on the result of subtracting the regularization parameter from 0.05 and the maximum value between 0.05 and 0.05, that is, the adjusted regularization parameter = max(regularization parameter - 0.05, 0.05).

[0078] Step S13: If yes, then starting from the next training time step, use the adjusted scaling factor and regularization parameter and based on the iteratively updated low-rank matrix to continue iteratively training the text generation model, and jump back to the step that triggers the monitoring operation of the text generation model until the text generation model meets the preset termination condition, and obtain the trained model, so as to use the trained model to generate the corresponding text based on the input data.

[0079] In this embodiment of the invention, if the scaling factor and regularization parameter are adjusted based on the training state of the text generation model at the current training time step, they can be applied immediately to the next training time step without interrupting the model training. Starting from the next training time step, the adjusted scaling factor and regularization parameter, based on the iteratively updated low-rank matrix, are used to continue iteratively training the text generation model. Then, the process jumps back to step S11, which triggers the monitoring operation of the text generation model, until the text generation model meets the preset termination condition to obtain a trained model. The trained model is then used to generate corresponding text based on the input data.

[0080] The preset termination conditions include any one or a combination of several of the following: the text generation model's accuracy reaches a preset accuracy, the text generation model's training loss is less than a preset loss, and the text generation model's cumulative training iterations reach a preset number of iterations.

[0081] It should be noted that if the scaling factor and regularization parameters are not adjusted based on the training state of the text generation model at the current training time step, the text generation model will continue to be trained iteratively using the unadjusted scaling factor and regularization parameters and based on the iteratively updated low-rank matrix. During the training process, the monitoring operation of the text generation model will continue to be triggered periodically based on the preset time step, so as to determine whether the scaling factor and regularization parameters should be adjusted based on the training state of the text generation model at the current training time step obtained from the monitoring.

[0082] In this way, the embodiments of the present invention form a closed loop of "model monitoring - parameter adjustment - parameter application - re-monitoring" through steps S11 to S13. By starting a model training, and dynamically adjusting the scaling factor and regularization parameters based on the model training state during the model training process, the model can respond to changes in data distribution in real time without interrupting the model training and can be efficiently adapted to the text generation task.

[0083] It should be noted that the method of this invention can be used not only to train text generation models, but also to train speech recognition models, image recognition models, image generation models, etc., and the core idea of ​​model training is similar.

[0084] Beneficial effects: This invention monitors the model during training to obtain the model's training status at each training time step. Based on the model's training status at each training time step, it automatically adjusts the scaling factor and regularization parameters to solve the problem of manual parameter adjustment and to correlate parameter adjustments with the model's training status, achieving real-time response to changes in data distribution. Furthermore, after adjusting the scaling factor and regularization parameters, this invention immediately applies the adjustments to subsequent model training without interrupting or restarting the model training. By starting model training only once, the model can be efficiently adapted to text generation tasks.

[0085] See Figure 2 As shown, this embodiment of the invention discloses a model training method based on parameter adjustment, including:

[0086] The method of this invention is specifically applied to a preset model training system, which includes a model adaptation layer, a dual-state monitoring layer, a parameter control layer, and a training execution layer.

[0087] The model adaptation layer includes a low-rank matrix insertion unit and a parameter adjustment interface. The low-rank matrix insertion unit is used to insert a low-rank matrix into the text generation model, and the parameter adjustment interface is used to adjust the scaling factor and regularization parameters.

[0088] The dual-state monitoring layer includes a gradient monitor and a generated text monitor. The gradient monitor is used to monitor the average gradient update of the text generation model at the current training time step, while the generated text monitor is used to monitor the generated text evaluation results of the text generation model at the current training time step, such as the text output format error rate.

[0089] The parameter tuning layer is used to determine whether to adjust the scaling factor and regularization parameters based on the mean gradient and text output format error rate of the text generation model at the current training time step. If adjustments are needed, the scaling factor and regularization parameters are adjusted using the parameter tuning interface of the model adaptation layer.

[0090] The training execution layer is used to perform iterative training of the text generation model.

[0091] Specifically, the model adaptation layer preloads the text generation model and the initial LoRA configuration; the initial LoRA configuration includes, but is not limited to, the initial value of the scaling factor, the initial value of the regularization parameter, and the dimension of the low-rank matrix; then, the LoRA adapter is initialized through the low-rank matrix insertion unit to insert a low-rank matrix into the preset network layer of the text generation model based on the low-rank matrix dimension, and the scaling factor and the regularization parameter are set to their initial values ​​through the parameter adjustment interface.

[0092] During the iterative training of the text generation model using scaling factors and regularization parameters and based on an iteratively updated low-rank matrix in the training execution layer, the dual-state monitoring layer periodically triggers monitoring operations on the text generation model based on a preset time step. This allows the gradient monitor to obtain the mean of the updated gradient of the text generation model at the current training time step, and the text generation monitor to obtain the evaluation results of the generated text at the current training time step, such as the text output format error rate. Then, the mean of the updated gradient and the text output format error rate are transmitted to the parameter control layer.

[0093] The parameter tuning layer determines whether to adjust the scaling factor and regularization parameters based on the mean of the updated gradient and the text output format error rate. If the mean of the updated gradient is greater than a preset mean, it indicates model training instability, requiring a reduction in the scaling factor to weaken the adaptation signal, while simultaneously increasing the regularization parameter to enhance regularization. If the mean of the updated gradient is not greater than a preset mean and the text output format error rate is greater than a preset format error rate, it indicates insufficient task adaptability, requiring an increase in the scaling factor to enhance the adaptation signal, while simultaneously decreasing the regularization parameter to reduce feature loss. Furthermore, the parameter tuning layer adjusts the scaling factor and regularization parameters using the parameter adjustment interface of the model adaptation layer without interrupting model training.

[0094] The training execution layer starts from the next training time step, and continues to iteratively train the text generation model using the adjusted scaling factor and regularization parameters and based on the iteratively updated low-rank matrix. At the same time, the dual-state monitoring layer continues to periodically trigger monitoring operations on the text generation model based on the preset time step until the text generation model meets the preset termination condition, so as to obtain the trained model.

[0095] Beneficial effects: This invention monitors the model during training to obtain the model's training status at each training time step. Based on the model's training status at each training time step, it automatically adjusts the scaling factor and regularization parameters to solve the problem of manual parameter adjustment and to correlate parameter adjustments with the model's training status, achieving real-time response to changes in data distribution. Furthermore, after adjusting the scaling factor and regularization parameters, this invention immediately applies the adjustments to subsequent model training without interrupting or restarting the model training. By starting model training only once, the model can be efficiently adapted to text generation tasks.

[0096] See Figure 3 As shown, this embodiment of the invention discloses a model training device based on parameter adjustment, comprising:

[0097] The training state acquisition module 11 is used to trigger the monitoring operation of the text generation model during the iterative training of the text generation model using scaling factors and regularization parameters and based on the iteratively updated low-rank matrix, so as to obtain the training state of the text generation model at the current training time step; the low-rank matrix is ​​pre-inserted into the preset network layer of the text generation model.

[0098] The parameter adjustment module 12 is used to determine whether to adjust the scaling factor and regularization parameter based on the training state;

[0099] The model training module 13 is used to, if so, start from the next training time step, use the adjusted scaling factor and regularization parameter and based on the iteratively updated low-rank matrix to continue iteratively training the text generation model, and jump back to the step that triggers the monitoring operation of the text generation model until the text generation model meets the preset termination condition, and obtain the trained model, so as to use the trained model to generate the corresponding text according to the input data.

[0100] Since the embodiments of the device part correspond to the embodiments described above, please refer to the embodiments described in the method part for the embodiments of the device part, and will not be repeated here.

[0101] Beneficial effects: This invention monitors the model during training to obtain the model's training status at each training time step. Based on the model's training status at each training time step, it automatically adjusts the scaling factor and regularization parameters to solve the problem of manual parameter adjustment and to correlate parameter adjustments with the model's training status, achieving real-time response to changes in data distribution. Furthermore, after adjusting the scaling factor and regularization parameters, this invention immediately applies the adjustments to subsequent model training without interrupting or restarting the model training. By starting model training only once, the model can be efficiently adapted to text generation tasks.

[0102] Furthermore, embodiments of this application also disclose an electronic device, Figure 4 This is a structural diagram of an electronic device according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. Specifically, the electronic device may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the parameter-adjustment-based model training method disclosed in any of the foregoing embodiments. Furthermore, the electronic device in this embodiment may specifically be an electronic computer.

[0103] In this embodiment, the power supply 23 is used to provide operating voltage for various hardware devices on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0104] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0105] The operating system 221 is used to manage and control the various hardware devices on the electronic device and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the parameter-adjustment-based model training method executed by the electronic device as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.

[0106] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned parameter-adjustment-based model training method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0107] Furthermore, this application also discloses a computer program product, including a computer program / instructions, wherein the computer program / instructions, when executed by a processor, implement the aforementioned disclosed parameter-adjustment-based model training method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0108] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0109] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0110] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0111] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0112] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only intended to help understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A model training method based on parameter tuning, characterized in that, include: During the iterative training of the text generation model using scaling factors and regularization parameters and based on the iteratively updated low-rank matrix, a monitoring operation on the text generation model is triggered to obtain the training status of the text generation model at the current training time step. The low-rank matrix is ​​pre-inserted into a preset network layer of the text generation model; Based on the training state, determine whether to adjust the scaling factor and the regularization parameter; If so, starting from the next training time step, the text generation model is iteratively trained using the adjusted scaling factor and regularization parameter and based on the iteratively updated low-rank matrix. The process then jumps back to the step that triggers the monitoring operation of the text generation model until the text generation model meets the preset termination condition, and a trained model is obtained. The trained model is then used to generate corresponding text based on the input data. The step of triggering a monitoring operation on the text generation model to obtain the training status of the text generation model at the current training time step includes: The text generation model is periodically monitored based on a preset time step to obtain the mean of the updated gradient and the evaluation result of the generated text at the current training time step. The mean of the updated gradient is used to reflect the training stability of the text generation model at the current training time step. The evaluation result of the generated text is used to reflect the adaptability of the text generation model to the text generation task at the current training time step.

2. The model training method based on parameter adjustment according to claim 1, characterized in that, The step of obtaining the mean of the updated gradients of the text generation model at the current training time step and the evaluation result of the generated text includes: The mean of the update gradient of the low-rank matrix at the current training time step is calculated to obtain the mean of the update gradient. Several test cases for the text generation task are input into the text generation model to obtain text generation results corresponding to the several test cases respectively; The evaluation result of the generated text is determined based on the text generation results and the expected generation results of the corresponding test cases.

3. The model training method based on parameter adjustment according to claim 2, characterized in that, The generated text evaluation results include the text output format error rate; The determination of the generated text evaluation result based on each of the text generation results and the expected generation result of the corresponding test cases includes: The consistency of the output format of each text generation result with the expected generation result of the corresponding test case is judged to determine the target generation result from each text generation result; the output format of the target generation result is inconsistent with the expected generation result of the corresponding test case. The text output format error rate is determined based on the ratio between the number of target generated results and the total number of all text generated results.

4. The model training method based on parameter adjustment according to claim 3, characterized in that, The step of determining whether to adjust the scaling factor and the regularization parameter based on the training state includes: Determine whether the updated gradient mean satisfies a preset mean condition; the preset mean condition includes that the updated gradient mean is greater than a preset gradient mean. If the mean of the updated gradient satisfies the preset mean condition, then the scaling factor is reduced and the regularization parameter is increased using the first preset adjustment rule. If the mean of the updated gradient does not meet the preset mean condition, then it is determined whether the text output format error rate meets the preset error rate condition; the preset error rate condition includes the text output format error rate being greater than the preset format error rate. If the text output format error rate does not meet the preset error rate condition, it is determined that the scaling factor and the regularization parameter will not be adjusted. If the text output format error rate meets the preset error rate condition, then the scaling factor is increased and the regularization parameter is decreased using the second preset adjustment rule.

5. The model training method based on parameter adjustment according to claim 4, characterized in that, The step of reducing the scaling factor and increasing the regularization parameter using a first preset adjustment rule includes: Determine the first product of the scaling factor and the first preset value, and determine the adjusted scaling factor based on the maximum value between the first product and the lower limit of the preset scaling factor; wherein the first preset value is a value less than one. The addition result of the regularization parameter and the second preset value is determined, and the adjusted regularization parameter is determined based on the minimum value between the addition result and the upper limit of the preset regularization parameter; wherein the second preset value is a value greater than zero and less than one.

6. The model training method based on parameter adjustment according to claim 5, characterized in that, The step of increasing the scaling factor and decreasing the regularization parameter using the second preset adjustment rule includes: The scaling factor is determined by multiplying the scaling factor by a second product of the scaling factor and a third preset value, and the adjusted scaling factor is determined based on the minimum value between the second product and the upper limit of the preset scaling factor; wherein the third preset value is a value greater than one. The result of subtracting the regularization parameter from the second preset value is determined, and the adjusted regularization parameter is determined based on the maximum value between the subtraction result and the preset lower limit of the regularization parameter.

7. A model training device based on parameter adjustment, characterized in that, include: The training state acquisition module is used to trigger a monitoring operation on the text generation model during the iterative training of the text generation model using scaling factors and regularization parameters and based on an iteratively updated low-rank matrix, so as to obtain the training state of the text generation model at the current training time step. The low-rank matrix is ​​pre-inserted into a preset network layer of the text generation model; The parameter adjustment module is used to determine whether to adjust the scaling factor and the regularization parameter based on the training state. The model training module is used to, if so, start from the next training time step, use the adjusted scaling factor and regularization parameter and based on the iteratively updated low-rank matrix to continue iteratively training the text generation model, and then jump back to the step of triggering the monitoring operation of the text generation model until the text generation model meets the preset termination condition, and obtain the trained model, so as to use the trained model to generate corresponding text according to the input data. Specifically, the training state acquisition module is used to periodically trigger monitoring operations on the text generation model based on a preset time step, so as to obtain the average updated gradient and the generated text evaluation result of the text generation model at the current training time step; the average updated gradient is used to reflect the training stability of the text generation model at the current training time step; the generated text evaluation result is used to reflect the adaptability of the text generation model to the text generation task at the current training time step.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the parameter-adjustment-based model training method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the model training method based on parameter adjustment as described in any one of claims 1 to 6.

Images