Dynamic loss and data construction collaborative large model fine tuning method and system

Abstract

The invention relates to the technical field of artificial intelligence large models, and discloses a dynamic loss and data construction collaborative large model fine tuning method and system, and the method comprises the steps: constructing an initial fine tuning data set, and initializing a dynamic loss function; performing uncertainty evaluation and dynamic data construction based on the model state, and generating training batch data; a dynamic loss function structure is constructed, loss coefficients are dynamically adjusted, and model fine adjustment is completed through a closed-loop optimization process. According to the method, the problems of data construction staticization, loss function immobilization and optimization disjunction can be solved, and the learning efficiency, the stability and the task generalization ability of the large language model in the fine tuning process are improved.

Description

Technical Field

[0001] This invention relates to the field of large-scale artificial intelligence models, and in particular to a method and system for fine-tuning large-scale models that combines dynamic loss and data construction. Background Technology

[0002] With the widespread application of Large Language Models (LLMs) in fields such as natural language processing, healthcare, and finance, achieving efficient, stable, and high-performance fine-tuning for specific downstream tasks has become a key challenge.

[0003] Current mainstream fine-tuning methods generally rely on static data construction strategies and fixed-form loss function designs. However, both have significant limitations in practical applications, specifically in the following aspects: 1) Static data construction methods lack model-aware capabilities. In existing technologies, fine-tuning datasets are typically constructed through manual annotation, rule-based filtering, or keyword matching, such as expert-annotated electronic medical record question-and-answer pairs in clinical decision support systems. While these methods can provide some quality of supervisory signals, they suffer from insufficient data diversity, high annotation costs, and long update cycles. Especially in long-tail tasks such as rare disease diagnosis and uncommon symptom identification, available samples are extremely scarce, making it difficult for models to fully learn key semantic patterns.

[0004] 2) Fixed loss function design, lacking dynamic adaptability. Current fine-tuning commonly uses static loss functions such as cross-entropy loss or mean squared error, whose weights remain unchanged throughout the training process. These methods assume that all samples contribute equally to the model's learning, ignoring differences in sample difficulty, class frequency, and semantic importance. For example, in medical text classification tasks, the model might overlook crucial but rare critical illness categories because common disease samples dominate.

[0005] 3) Data construction and loss function optimization are disconnected, lacking a collaborative mechanism. In traditional fine-tuning processes, the data preparation stage and the model training stage are independent: the dataset is constructed all at once before training, while the loss function remains fixed throughout the training process. Even when data augmentation or resampling techniques are introduced, they are mostly pre-defined strategies and cannot be linked with the loss function.

[0006] 4) Weak adaptability to tasks with small sample sizes and long-tail distributions. In professional fields such as medicine and law, the cost and time required to acquire labeled data are high, and task categories often exhibit significant long-tail distribution characteristics (e.g., most are common diseases, and a few are rare diseases). Existing fine-tuning methods perform poorly in low-resource scenarios, making it difficult to effectively capture the semantic features of the tail categories, resulting in low accuracy of the model on key sub-tasks and affecting the reliability of actual deployment.

[0007] In summary, existing large model fine-tuning techniques have significant shortcomings in terms of the intelligence of data construction, the dynamic adaptability of loss functions, and system-level collaborative optimization capabilities, making it difficult to meet the needs of complex application scenarios such as high precision, low resource consumption, and strong generalization.

[0008] Therefore, there is an urgent need to provide a technical solution to address the above problems. Summary of the Invention

[0009] To address the aforementioned technical problems, this invention provides a method and system for fine-tuning large models that coordinates dynamic loss and data construction.

[0010] Firstly, this invention provides a method for fine-tuning large models that combines dynamic loss and data construction. The technical solution of this method is as follows: S1. Construct the initial fine-tuning dataset for the pre-trained large language model, configure the training hyperparameters of the pre-trained large language model, and initialize the adjustable coefficients and corresponding adjustment function parameters in the dynamic loss function of the pre-trained large language model. S2. Based on the current state of the pre-trained large language model, perform uncertainty assessment on the samples in the unlabeled data pool, generate a new training sample set according to the assessment results, combine the new training sample set with the training sample set of the current round to obtain an updated training sample set, and dynamically reweight each training sample in the updated training sample set to construct the training batch data of the current round. S3. Based on the training batch data of the current round, construct a dynamic loss function structure for the current round that includes task loss, consistency regularization loss and semantic alignment loss, and dynamically adjust the adjustable coefficients corresponding to the consistency regularization loss and the semantic alignment loss according to the current round and the adjustment function parameters to obtain the dynamic loss function for the current round. S4. Calculate the gradient based on the dynamic loss function of the current round, and update the parameters of the pre-trained large language model according to the gradient and the training hyperparameters to obtain the updated model; S5. Use the updated model as the pre-trained large language model, and return to execute steps S2-S4 until the model convergence condition is met, then determine the updated model as the final fine-tuned model.

[0011] The beneficial effects of the large model fine-tuning method that combines dynamic loss and data construction according to the present invention are as follows: The method of this invention solves the problems of static data construction, fixed loss function and disconnect between optimization by constructing a collaborative optimization framework of dynamic data construction and dynamic loss function. It evaluates uncertainty and generates samples based on the current state of the model, and adaptively adjusts the loss function coefficient as the training process progresses. This improves the learning efficiency, stability and task generalization ability of large language models in the fine-tuning process.

[0012] Based on the above scheme, the large model fine-tuning method of dynamic loss and data construction coordination of the present invention can be further improved as follows.

[0013] In one alternative approach, S1 specifically includes: Construct and load the pre-trained large language model, wherein the parameters of the pre-trained large language model are θ; Construct the initial fine-tuning dataset ,in, This indicates the number of samples in the initial fine-tuning dataset. This represents the input text of the i-th sample in the initial fine-tuning dataset. This represents the label corresponding to the input text of the i-th sample in the initial fine-tuning dataset; Configure the training hyperparameters of the pre-trained large language model, including optimizer, learning rate scheduler, batch size, and maximum training epochs; Initialize the adjustable coefficients in the dynamic loss function , and and the adjustment function parameters, among which Indicates the training round.

[0014] The advantages of adopting the above optional approach are: it further clarifies the complete process of initial fine-tuning dataset construction, hyperparameter configuration, and dynamic loss function adjustable coefficient initialization, provides a standardized basis for subsequent uncertainty assessment and dynamic adjustment, and ensures that dynamic data construction and loss function adjustment operate stably under a unified hyperparameter framework.

[0015] In one alternative approach, the step of performing uncertainty assessment on samples in the unlabeled data pool based on the current state of the pre-trained large language model, and generating a new training sample set based on the assessment result, includes: Based on the current state of the pre-trained large language model, uncertainty is assessed for samples in the unlabeled data pool, and the prediction uncertainty for each sample in the unlabeled data pool is calculated; wherein, the prediction uncertainty is calculated using prediction entropy, and the calculation formula is: ; Indicates sample The uncertainty of prediction This indicates that the pre-trained large language model is in the current parameters The following sample Category The predicted probability, Represents the set of all possible categories. Represents the Shannon entropy function; Samples with prediction uncertainty higher than a preset threshold τ are selected to form a high uncertainty sample set; Based on the high-uncertainty sample set, a new training sample set is generated through semantically preserved data augmentation, knowledge-guided sample generation, or adversarial sample construction.

[0016] The beneficial effects of adopting the above-mentioned optional methods are as follows: further uncertainty assessment of unlabeled samples is carried out through prediction entropy, high-value difficult cases with prediction uncertainty higher than the threshold are actively screened, and training samples are expanded through semantic preservation enhancement, knowledge-guided generation or adversarial construction methods, so that the data construction process is linked with the current learning state of the model in real time.

[0017] In one alternative approach, the step of dynamically reweighting each training sample in the updated training sample set to construct the training batch data for the current round includes: Calculate the dynamic weight of each training sample in the updated training sample set. The formula for calculating the dynamic weight is as follows: ;in, Indicates the first The dynamic weights of each training sample. and Indicates hyperparameters, Indicates the first The prediction uncertainty of each training sample Indicates the indicator function, when the first Labels of each training sample Belongs to the predefined rare category set The value is 1 if the condition is met, and 0 otherwise. Using the dynamic weights of each training sample in the updated training sample set, weighted random sampling is performed on the training samples in the updated training sample set to construct the training batch data for the current round.

[0018] The beneficial effects of adopting the above optional approach are as follows: further calculate dynamic weights based on the uncertainty of sample prediction and the rare category attribute of the label, and perform weighted random sampling on the updated training sample set based on the weights, so that the training batch data is automatically tilted towards difficult examples and long-tail categories, thereby improving the learning strength of the model for key semantic patterns.

[0019] In one alternative approach, the step of constructing a dynamic loss function structure for the current round, comprising task loss, consistency regularization loss, and semantic alignment loss, based on the training batch data of the current round, includes: Based on the training batch data of the current round, the task loss is calculated, and the formula for calculating the task loss is: ;in, Indicates the first Training batch data in each round, Indicates the first The number of samples in the training batch data of each round. Indicates the first The first training batch of data in each round The input text for each sample, Indicates the first The first training batch of data in each round The label of each sample, Indicates the first The first training batch of data in each round Dynamic weights of each sample This indicates that the pre-trained large language model is in the first... Round parameters Next to the The first training batch of data in each round Input text for each sample Predict as label The probability of; The consistency regularization loss is calculated using the following formula: ;in, Indicates the first The first training batch of data in each round Input text for each sample The version with random enhancements applied, This indicates that the pre-trained large language model is in the first... Round parameters Next to the The first training batch of data in each round Input text for each sample The predicted probability distribution This indicates that the pre-trained large language model is in the first... Round parameters Below is an enhanced version The predicted probability distribution Indicates the Kullback-Leibler divergence; Calculate the semantic alignment loss, the formula for which is: ;in, This indicates that the pre-trained large language model encoder is in the first... Round parameters The next output The first training batch of data in each round Input text for each sample semantic vectors, Indicates the first The first training batch of data in each round Labels of each sample Predefined semantic embedding; Based on the task loss, the consistency regularization loss, and the semantic alignment loss, the dynamic loss function structure is constructed.

[0020] The beneficial effects of adopting the above optional approach are as follows: further constructing a composite structure that includes weighted task loss, consistency regularization loss and semantic alignment loss, which can collaboratively optimize the model representation capability from three dimensions: prediction accuracy, anti-disturbance stability and input-output semantic alignment, thereby enhancing the robustness and generalization of the fine-tuning process.

[0021] In one optional approach, the step of dynamically adjusting the adjustable coefficients corresponding to the consistency regularization loss and the semantic alignment loss based on the current round and the adjustment function parameters to obtain the dynamic loss function for the current round includes: Calculate the adjustable coefficient corresponding to the consistency regularization loss based on the current round and the adjustment function parameters. The adjustable coefficients corresponding to the semantic alignment loss ;in, The calculation formula is: , The calculation formula is: ; Indicates the first Rounds, This represents the Sigmoid function. , , , , , Indicates the parameters of the adjustment function; Based on the task loss, the consistency regularization loss, and the semantic alignment loss, as well as the adjustable coefficient... , and Construct the dynamic loss function for the current round; wherein the formula for calculating the dynamic loss function is: ; This represents the fixed coefficient corresponding to the task loss.

[0022] The advantages of adopting the above optional approach are as follows: the adjustable coefficients of consistency regularization loss and semantic alignment loss are further dynamically adjusted according to the training rounds, and the weights are smoothly transitioned through the Sigmoid function and the logistic function, so that the stability constraints are emphasized in the early stage of training and the semantic alignment is strengthened in the later stage, thus realizing a phased adaptive optimization strategy.

[0023] In one alternative approach, S4 specifically includes: Based on the dynamic loss function of the current round, calculate the gradient of the dynamic loss function of the current round with respect to the parameters of the pre-trained large language model; Based on the gradient and the training hyperparameters, the parameters of the pre-trained large language model are updated to obtain the updated model; wherein, the update formula is: ; This represents the parameters of the pre-trained large language model in round t. This represents the updated parameters of the pre-trained large language model. Let represent the gradient of the dynamic loss function at round t with respect to the parameters of the pre-trained large language model. Let represent the training hyperparameters in round t.

[0024] The advantages of adopting the above optional approach are: further calculating the gradient based on the dynamic loss function and clarifying the parameter update formula, ensuring that the gradients of each component are effectively propagated during the backpropagation of the composite loss, so that the parameters of the pre-trained large language model can stably converge to the optimal state under the drive of dynamic weights and adjustment coefficients.

[0025] Secondly, this invention provides a large model fine-tuning system that coordinates dynamic loss and data construction. The technical solution of this system is as follows: The initialization module is used to construct the initial fine-tuning dataset of the pre-trained large language model, configure the training hyperparameters of the pre-trained large language model, and initialize the adjustable coefficients and corresponding adjustment function parameters in the dynamic loss function of the pre-trained large language model. The construction module is used to perform uncertainty assessment on samples in the unlabeled data pool based on the current state of the pre-trained large language model, generate a new training sample set according to the assessment result, combine the new training sample set with the training sample set of the current round to obtain an updated training sample set, and dynamically reweight each training sample in the updated training sample set to construct the training batch data of the current round. The adjustment module is used to construct a dynamic loss function structure for the current round, which includes task loss, consistency regularization loss and semantic alignment loss, based on the training batch data of the current round, and dynamically adjust the adjustable coefficients corresponding to the consistency regularization loss and the semantic alignment loss according to the current round and the adjustment function parameters to obtain the dynamic loss function for the current round. The update module is used to calculate the gradient based on the dynamic loss function of the current round, and update the parameters of the pre-trained large language model according to the gradient and the training hyperparameters to obtain the updated model. The iteration module is used to take the updated model as the pre-trained large language model and return to the update module by calling the construction module until the model convergence condition is met, and then the updated model is determined as the final fine-tuned model.

[0026] The beneficial effects of the large model fine-tuning system that combines dynamic loss and data construction according to the present invention are as follows: The system of this invention solves the problems of static data construction, fixed loss function and disconnect between optimization by constructing a collaborative optimization framework of dynamic data construction and dynamic loss function. It performs uncertainty assessment and sample generation based on the current state of the model, and adaptively adjusts the loss function coefficient as the training process progresses. This improves the learning efficiency, stability and task generalization ability of large language models in the fine-tuning process.

[0027] Thirdly, the technical solution of an electronic device according to the present invention is as follows: It includes a memory, a processor, and a program stored in the memory and running on the processor, wherein the processor executes the program to implement the steps of the large model fine-tuning method of dynamic loss and data construction coordination as described in this invention.

[0028] Fourthly, the technical solution of a computer-readable storage medium provided by the present invention is as follows: The computer-readable storage medium stores instructions that, when read, cause the computer-readable storage medium to perform the steps of the large model fine-tuning method of the dynamic loss and data construction synergy of the present invention.

[0029] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0030] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a flowchart illustrating an embodiment of a large model fine-tuning method that combines dynamic loss and data construction according to the present invention. Figure 2 This is a schematic diagram of an embodiment of a large model fine-tuning system that combines dynamic loss and data construction according to the present invention. Figure 3 This is a schematic diagram of an embodiment of an electronic device according to the present invention. Detailed Implementation

[0031] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0032] Figure 1 This diagram illustrates a flowchart of an embodiment of a large model fine-tuning method that combines dynamic loss and data construction, provided by the present invention. This method can be executed by electronic devices such as terminal devices or servers. The terminal device can be any fixed or mobile terminal, such as user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, or wearable device. The server can be a single server or a server cluster consisting of multiple servers. Any electronic device can implement the large model fine-tuning method that combines dynamic loss and data construction by having its processor call computer-readable instructions stored in memory. Figure 1 As shown, it includes the following steps: S1. Construct the initial fine-tuning dataset for the pre-trained large language model, configure the training hyperparameters of the pre-trained large language model, and initialize the adjustable coefficients and corresponding adjustment function parameters in the dynamic loss function of the pre-trained large language model.

[0033] Among them, the pre-trained large language model refers to a basic model with strong language understanding and generation capabilities, pre-trained on massive text data; for example, the LLaMA-2 model, after pre-training on general corpora, possesses basic language capabilities and can be used as an initial model for medical question-answering tasks. The initial fine-tuning dataset refers to a set of labeled samples prepared for a specific downstream task; for example, a set containing 5000 electronic medical record question-answer pairs, where the questions are the patient's complaints and the answers are the doctor's diagnoses. Training hyperparameters refer to configuration parameters that control the model training process; for example, the AdamW optimizer, cosine annealing learning rate scheduler, batch size of 32, and maximum training epochs of 100. The dynamic loss function refers to a composite loss function that adaptively adjusts the weight coefficients as the training progresses; for example, a loss function consisting of a weighted sum of task loss, consistency regularization loss, and semantic alignment loss, whose weights change with the training epochs. Adjustable coefficients refer to the weight parameters of each loss term in the dynamic loss function. Adjustment function parameters refer to parameters that control the variation of the adjustable coefficients.

[0034] S2. Based on the current state of the pre-trained large language model, perform uncertainty assessment on the samples in the unlabeled data pool, generate a new training sample set according to the assessment results, combine the new training sample set with the training sample set of the current round to obtain an updated training sample set, and dynamically reweight each training sample in the updated training sample set to construct the training batch data of the current round.

[0035] Here, "current state" refers to the parameter values ​​of the pre-trained large language model in a specific training epoch. "Unlabeled data pool" refers to a large-scale text dataset without labeled information; for example, 100,000 unlabeled emergency room records accumulated by a hospital. "Uncertainty assessment" refers to a method for quantifying the model's confidence in predicting samples; for example, calculating the model's prediction entropy for diagnosing medical records to obtain a numerical uncertainty score. "Evaluation results" refers to a set of quantitative indicators obtained based on uncertainty assessment; for example, a list of prediction entropy values ​​calculated for samples in the unlabeled pool. "New training sample set" refers to an augmented sample set generated through data construction strategies; for example, 3000 new question-answer pairs generated after synonym replacement and sentence transformation of high-uncertainty samples. "Current epoch" refers to the iteration number in the model training process; for example, the 15th training phase where parameter updates are performed. "Training sample set" refers to the dataset used for model training in the current epoch; for example, the dataset containing 8000 samples used in the 15th training epoch. The updated training sample set refers to the training data set after merging new samples; for example, a set of 8000 samples formed by merging the original 5000 samples with 3000 newly added samples. Dynamic reweighting refers to the process of adjusting sampling weights based on sample importance; for example, calculating weights based on sample uncertainty and class rarity to make difficult samples easier to sample.

[0036] S3. Based on the training batch data of the current round, construct a dynamic loss function structure for the current round that includes task loss, consistency regularization loss, and semantic alignment loss, and dynamically adjust the adjustable coefficients corresponding to the consistency regularization loss and the semantic alignment loss according to the current round and the adjustment function parameters to obtain the dynamic loss function for the current round.

[0037] Among these, task loss refers to the loss function that measures the performance of the model on its main task; for example, weighted cross-entropy loss, where hard examples have higher loss weights. Consistency regularization loss refers to the loss term that constrains the model to improve the consistency of predictions for enhanced samples; for example, the KL divergence loss between the original medical record text and the version with added synonyms. Semantic alignment loss refers to the loss term that strengthens the semantic association between input and output; for example, the Euclidean distance loss between the medical record text encoding vector and the diagnostic label embedding vector. Dynamic loss function structure refers to a function framework that includes a weighted sum of multiple loss terms.

[0038] S4. Calculate the gradient based on the dynamic loss function of the current round, and update the parameters of the pre-trained large language model according to the gradient and the training hyperparameters to obtain the updated model.

[0039] Here, gradient refers to the partial derivative of the loss function with respect to the model parameters; for example, the vector of partial derivatives of the dynamic loss function with respect to the weight matrix of the Transformer layer. Updated model refers to the new version of the model after parameter optimization.

[0040] S5. Use the updated model as the pre-trained large language model, and return to execute steps S2-S4 until the model convergence condition is met, then determine the updated model as the final fine-tuned model.

[0041] The model convergence condition refers to the criterion for terminating the training process; for example, the validation set accuracy does not improve for 5 consecutive rounds or reaches the maximum training rounds of 100. The final fine-tuned model refers to the usable model output after satisfying the convergence condition; for example, the model parameters θ* that achieve the highest accuracy on the validation set.

[0042] The technical solution of this embodiment solves the problems of static data construction, fixed loss function and disconnect between optimization by constructing a collaborative optimization framework of dynamic data construction and dynamic loss function. It evaluates uncertainty and generates samples based on the current state of the model, and adaptively adjusts the loss function coefficient as the training process progresses. This improves the learning efficiency, stability and task generalization ability of large language models in the fine-tuning process.

[0043] In one alternative approach, S1 specifically includes: Construct and load the pre-trained large language model, the parameters of which are θ.

[0044] Construct the initial fine-tuning dataset ,in, This indicates the number of samples in the initial fine-tuning dataset. This represents the input text of the i-th sample in the initial fine-tuning dataset. This represents the label corresponding to the input text of the i-th sample in the initial fine-tuning dataset.

[0045] Configure the training hyperparameters of the pre-trained large language model, including optimizer, learning rate scheduler, batch size, and maximum training epochs.

[0046] Initialize the adjustable coefficients in the dynamic loss function , and and the adjustment function parameters, among which Indicates the training round.

[0047] Among the above optional methods, the complete process of initial fine-tuning dataset construction, hyperparameter configuration, and dynamic loss function adjustable coefficient initialization is further clarified, providing a standardized basis for subsequent uncertainty assessment and dynamic adjustment, and ensuring that dynamic data construction and loss function adjustment run stably under a unified hyperparameter framework.

[0048] In one alternative approach, the step of performing uncertainty assessment on samples in the unlabeled data pool based on the current state of the pre-trained large language model, and generating a new training sample set based on the assessment result, includes: Based on the current state of the pre-trained large language model, uncertainty is assessed for samples in the unlabeled data pool, and the prediction uncertainty for each sample in the unlabeled data pool is calculated; wherein, the prediction uncertainty is calculated using prediction entropy, and the calculation formula is: ; Indicates sample The uncertainty of prediction This indicates that the pre-trained large language model is in the current parameters The following sample Category The predicted probability, Represents the set of all possible categories. This represents the Shannon entropy function.

[0049] Prediction uncertainty refers to the quantified confidence level of the model's prediction results for the samples; for example, the uncertainty score obtained by calculating the entropy of the diagnostic probability distribution. .

[0050] Samples with prediction uncertainty higher than a preset threshold τ are selected to form a high-uncertainty sample set.

[0051] The preset threshold refers to the critical value for filtering samples with high uncertainty; for example, setting the entropy threshold τ=0.8 to filter... Samples >0.8.

[0052] Based on the high-uncertainty sample set, a new training sample set is generated through semantically preserved data augmentation, knowledge-guided sample generation, or adversarial sample construction.

[0053] High-uncertainty sample sets refer to the set of samples where the prediction uncertainty exceeds a threshold; for example, 2000 high-entropy medical records selected from an unlabeled pool. Semantic-preserving data augmentation refers to text transformation methods that preserve the original semantics; for example, rewriting "the patient has a severe headache" as "the patient's head pain is severe." Knowledge-guided sample generation refers to methods that create new samples based on domain knowledge; for example, combining "fever and cough" with "respiratory infection" into a new question-answer pair based on a medical knowledge base. Adversarial sample construction refers to methods that add perturbations to generate challenging samples; for example, inserting the irrelevant word "occasionally" into medical record text to generate adversarial samples.

[0054] Among the above-mentioned optional methods, the uncertainty of unlabeled samples is further evaluated by predictive entropy, high-value difficult cases with predictive uncertainty higher than the threshold are actively screened, and training samples are expanded by semantic preservation enhancement, knowledge-guided generation or adversarial construction methods, so that the data construction process is linked with the current learning state of the model in real time.

[0055] In one alternative approach, the step of dynamically reweighting each training sample in the updated training sample set to construct the training batch data for the current round includes: Calculate the dynamic weight of each training sample in the updated training sample set. The formula for calculating the dynamic weight is as follows: ;in, Indicates the first The dynamic weights of each training sample. and Indicates hyperparameters, Indicates the first The prediction uncertainty of each training sample Indicates the indicator function, when the first Labels of each training sample Belongs to the predefined rare category set The value is 1 if it is true, and 0 otherwise.

[0056] Using the dynamic weights of each training sample in the updated training sample set, weighted random sampling is performed on the training samples in the updated training sample set to construct the training batch data for the current round.

[0057] Dynamic weights refer to the importance coefficients of samples during training; for example, weights calculated by combining uncertainty entropy and rare disease labels. Weighted random sampling refers to a method of sampling according to a weighted distribution; for example, based on dynamic weights. A batch of 32 samples was drawn from the updated training set.

[0058] In the above-mentioned optional methods, dynamic weights are further calculated based on the sample prediction uncertainty and the rare category attribute of the label, and the updated training sample set is weighted and randomly sampled based on the weights, so that the training batch data is automatically tilted towards difficult examples and long-tail categories, thereby improving the model's learning strength of key semantic patterns.

[0059] In one alternative approach, the step of constructing a dynamic loss function structure for the current round, comprising task loss, consistency regularization loss, and semantic alignment loss, based on the training batch data of the current round, includes: Based on the training batch data of the current round, the task loss is calculated, and the formula for calculating the task loss is: ;in, Indicates the first Training batch data in each round, Indicates the first The number of samples in the training batch data of each round. Indicates the first The first training batch of data in each round The input text for each sample, Indicates the first The first training batch of data in each round The label of each sample, Indicates the first The first training batch of data in each round Dynamic weights of each sample This indicates that the pre-trained large language model is in the first... Round parameters Next to the The first training batch of data in each round Input text for each sample Predict as label The probability of.

[0060] The consistency regularization loss is calculated using the following formula: ;in, Indicates the first The first training batch of data in each round Input text for each sample The version with random enhancements applied, This indicates that the pre-trained large language model is in the first... Round parameters Next to the The first training batch of data in each round Input text for each sample The predicted probability distribution This indicates that the pre-trained large language model is in the first... Round parameters Below is an enhanced version The predicted probability distribution This represents the Kullback-Leibler divergence.

[0061] Calculate the semantic alignment loss, the formula for which is: ;in, This indicates that the pre-trained large language model encoder is in the first... Round parameters The next output The first training batch of data in each round Input text for each sample semantic vectors, Indicates the first The first training batch of data in each round Labels of each sample Predefined semantic embeddings.

[0062] Based on the task loss, the consistency regularization loss, and the semantic alignment loss, the dynamic loss function structure is constructed.

[0063] Among the above-mentioned optional approaches, a composite structure including weighted task loss, consistency regularization loss, and semantic alignment loss is further constructed to collaboratively optimize the model representation capability from three dimensions: prediction accuracy, perturbation resistance stability, and input-output semantic alignment, thereby enhancing the robustness and generalization of the fine-tuning process.

[0064] In one optional approach, the step of dynamically adjusting the adjustable coefficients corresponding to the consistency regularization loss and the semantic alignment loss based on the current round and the adjustment function parameters to obtain the dynamic loss function for the current round includes: Calculate the adjustable coefficient corresponding to the consistency regularization loss based on the current round and the adjustment function parameters. The adjustable coefficients corresponding to the semantic alignment loss ;in, The calculation formula is: , The calculation formula is: ; Indicates the first Rounds, This represents the Sigmoid function. , , , , , This represents the parameters of the adjustment function.

[0065] Based on the task loss, the consistency regularization loss, and the semantic alignment loss, as well as the adjustable coefficient... , and Construct the dynamic loss function for the current round; wherein the formula for calculating the dynamic loss function is: ; This represents the fixed coefficient corresponding to the task loss.

[0066] In the above-mentioned optional methods, the adjustable coefficients of consistency regularization loss and semantic alignment loss are further dynamically adjusted according to the training rounds. The weights are smoothly transitioned through the Sigmoid function and the logistic function, so that the early training focuses on stability constraints and the later training strengthens semantic alignment, thus realizing a phased adaptive optimization strategy.

[0067] In one alternative approach, S4 specifically includes: Based on the dynamic loss function of the current round, calculate the gradient of the dynamic loss function of the current round with respect to the parameters of the pre-trained large language model.

[0068] Based on the gradient and the training hyperparameters, the parameters of the pre-trained large language model are updated to obtain the updated model; wherein, the update formula is: ; This represents the parameters of the pre-trained large language model in round t. This represents the updated parameters of the pre-trained large language model. Let represent the gradient of the dynamic loss function at round t with respect to the parameters of the pre-trained large language model. Let represent the training hyperparameters in round t.

[0069] In the above-mentioned optional methods, the gradient is further calculated based on the dynamic loss function and the parameter update formula is clarified to ensure that the gradient of each component is effectively propagated during the backpropagation of the composite loss, so that the parameters of the pre-trained large language model can be stably converged to the optimal state under the drive of dynamic weights and adjustment coefficients.

[0070] To better illustrate the technical solution of this embodiment, the following complete example is used for explanation, specifically: S10: Construct the initial fine-tuning dataset for the pre-trained large language model, configure the training hyperparameters of the pre-trained large language model, and initialize the adjustable coefficients and corresponding adjustment function parameters in the dynamic loss function of the pre-trained large language model; load the pre-trained large language model, the parameters of which are... Construct an initial fine-tuning dataset containing 1000 electronic medical record question-answer pairs, where the input text for each sample is a description of the patient's symptoms, and the label is the corresponding diagnostic conclusion; configure training hyperparameters, including an optimizer of AdamW, a learning rate scheduler of cosine annealing, a batch size of 32, and a maximum training epoch of 50; initialize the adjustable coefficients in the dynamic loss function. , and and adjustment function parameters , , , , , ,in Indicates the training round; S20: Based on the current state of the pre-trained large language model, perform uncertainty assessment on the samples in the unlabeled data pool, generate a new training sample set based on the assessment results, merge the new training sample set with the training sample set of the current round to obtain an updated training sample set, and dynamically reweight each training sample in the updated training sample set to construct the training batch data for the current round; based on the parameters of the pre-trained large language model in the current round... Under the given conditions, uncertainty assessment is performed on 10,000 electronic medical records in the unlabeled data pool. The prediction uncertainty for each sample is calculated using prediction entropy, as shown in the formula below. ,in For the diagnostic category; select those with predictive uncertainty exceeding a preset threshold. The samples constitute a high-uncertainty sample set. Based on this high-uncertainty sample set, new samples are generated through semantically preserved data augmentation, such as rewriting the symptom description "severe headache" as "severe head pain." New question-answer pairs are generated from a medical knowledge base through knowledge-guided sample generation, such as associating fever and cough with respiratory infections. New samples are generated by adding slight perturbations through adversarial sample construction. The newly generated training sample set is merged with the current round's training sample set to form an updated training sample set. Dynamic weights are calculated for each training sample in the updated training sample set, using the following formula: ,in , For hyperparameters, For the sample The uncertainty of prediction For indicator functions, when the label Belongs to the pre-defined rare category set The value is 1; the updated training sample set is weighted and randomly sampled using dynamic weights to construct the training batch data for the current round; S30: Based on the training batch data of the current round, construct a dynamic loss function structure for the current round that includes task loss, consistency regularization loss, and semantic alignment loss, and dynamically adjust the adjustable coefficients corresponding to the consistency regularization loss and the semantic alignment loss according to the current round and the adjustment function parameters to obtain the dynamic loss function for the current round; calculate the task loss based on the training batch data of the current round, using the following formula: ; Calculate the consistency regularization loss. The formula for calculating the consistency regularization loss is as follows: Calculate the semantic alignment loss, the formula for which is: A dynamic loss function structure is constructed based on task loss, consistency regularization loss, and semantic alignment loss; the structure is adjusted according to the current round. And adjust the adjustable coefficients corresponding to the consistency regularization loss by adjusting the function parameters. Adjustable coefficients corresponding to semantic alignment loss , The calculation formula is , The calculation formula is ,in The Sigmoid function is used to obtain the dynamic loss function for the current round. ; S40: Calculate the gradient based on the dynamic loss function of the current round, and update the parameters of the pre-trained large language model according to the gradient and the training hyperparameters to obtain the updated model; based on the dynamic loss function of the current round... Calculate gradient Based on the gradient and the learning rate in the training hyperparameters. Update the parameters of the pre-trained large language model using the following formula: ; obtain the updated model; S50: Use the updated model as the pre-trained large language model, and return to execute steps S20-S40 until the model convergence condition is met, then determine the updated model as the final fine-tuned model; set the updated model parameters... As a pre-trained large language model, the uncertainty assessment, dynamic data construction, dynamic loss function construction, and parameter update steps are repeatedly executed. When the diagnostic accuracy of the validation set does not improve for 5 consecutive rounds or reaches the maximum training round of 50 rounds, the model convergence condition is met. The updated model at this time is determined as the final fine-tuned model.

[0071] Figure 2 This diagram illustrates a structural schematic of an embodiment of a large model fine-tuning system 200 that combines dynamic loss and data construction according to the present invention. Figure 2 As shown, the large model fine-tuning system 200, which coordinates dynamic loss and data construction, includes: Initialization module 201 is used to construct the initial fine-tuning dataset of the pre-trained large language model, configure the training hyperparameters of the pre-trained large language model, and initialize the adjustable coefficients and corresponding adjustment function parameters in the dynamic loss function of the pre-trained large language model. The construction module 202 is used to perform uncertainty assessment on samples in the unlabeled data pool based on the current state of the pre-trained large language model, generate a new training sample set according to the assessment result, combine the new training sample set with the training sample set of the current round to obtain an updated training sample set, and dynamically reweight each training sample in the updated training sample set to construct the training batch data of the current round. The adjustment module 203 is used to construct a dynamic loss function structure for the current round, which includes task loss, consistency regularization loss and semantic alignment loss, based on the training batch data of the current round, and dynamically adjust the adjustable coefficients corresponding to the consistency regularization loss and the semantic alignment loss according to the current round and the adjustment function parameters to obtain the dynamic loss function for the current round. The update module 204 is used to calculate the gradient based on the dynamic loss function of the current round, and update the parameters of the pre-trained large language model according to the gradient and the training hyperparameters to obtain the updated model. The iteration module 205 is used to use the updated model as the pre-trained large language model and return to call the construction module 202 to the update module 204 until the model convergence condition is met, and then the updated model is determined as the final fine-tuned model.

[0072] In an alternative embodiment, the initialization module 201 is specifically used for: Construct and load the pre-trained large language model, wherein the parameters of the pre-trained large language model are θ; Construct the initial fine-tuning dataset ,in, This indicates the number of samples in the initial fine-tuning dataset. This represents the input text of the i-th sample in the initial fine-tuning dataset. This represents the label corresponding to the input text of the i-th sample in the initial fine-tuning dataset; Configure the training hyperparameters of the pre-trained large language model, including optimizer, learning rate scheduler, batch size, and maximum training epochs; Initialize the adjustable coefficients in the dynamic loss function , and and the adjustment function parameters, among which Indicates the training round.

[0073] In an alternative embodiment, the building module 202 is specifically used for: Based on the current state of the pre-trained large language model, uncertainty is assessed for samples in the unlabeled data pool, and the prediction uncertainty for each sample in the unlabeled data pool is calculated; wherein, the prediction uncertainty is calculated using prediction entropy, and the calculation formula is: ; Indicates sample The uncertainty of prediction This indicates that the pre-trained large language model is in the current parameters The following sample Category The predicted probability, Represents the set of all possible categories. Represents the Shannon entropy function; Samples with prediction uncertainty higher than a preset threshold τ are selected to form a high uncertainty sample set; Based on the high-uncertainty sample set, a new training sample set is generated through semantically preserved data augmentation, knowledge-guided sample generation, or adversarial sample construction.

[0074] In an alternative embodiment, the building module 202 is specifically used for: Calculate the dynamic weight of each training sample in the updated training sample set. The formula for calculating the dynamic weight is as follows: ;in, Indicates the first The dynamic weights of each training sample. and Indicates hyperparameters, Indicates the first The prediction uncertainty of each training sample Indicates the indicator function, when the first Labels of each training sample Belongs to the predefined rare category set The value is 1 if the condition is met, and 0 otherwise. Using the dynamic weights of each training sample in the updated training sample set, weighted random sampling is performed on the training samples in the updated training sample set to construct the training batch data for the current round.

[0075] In an alternative embodiment, the adjustment module 203 is specifically used for: Based on the training batch data of the current round, the task loss is calculated, and the formula for calculating the task loss is: ;in, Indicates the first Training batch data in each round, Indicates the first The number of samples in the training batch data of each round. Indicates the first The first training batch of data in each round The input text for each sample, Indicates the first The first training batch of data in each round The label of each sample, Indicates the first The first training batch of data in each round Dynamic weights of each sample This indicates that the pre-trained large language model is in the first... Round parameters Next to the The first training batch of data in each round Input text for each sample Predict as label The probability of; The consistency regularization loss is calculated using the following formula: ;in, Indicates the first The first training batch of data in each round Input text for each sample The version with random enhancements applied, This indicates that the pre-trained large language model is in the first... Round parameters Next to the The first training batch of data in each round Input text for each sample The predicted probability distribution This indicates that the pre-trained large language model is in the first... Round parameters Below is an enhanced version The predicted probability distribution Indicates the Kullback-Leibler divergence; Calculate the semantic alignment loss, the formula for which is: ;in, This indicates that the pre-trained large language model encoder is in the first... Round parameters The next output The first training batch of data in each round Input text for each sample semantic vectors, Indicates the first The first training batch of data in each round Labels of each sample Predefined semantic embedding; Based on the task loss, the consistency regularization loss, and the semantic alignment loss, the dynamic loss function structure is constructed.

[0076] In an alternative embodiment, the adjustment module 203 is specifically used for: Calculate the adjustable coefficient corresponding to the consistency regularization loss based on the current round and the adjustment function parameters. The adjustable coefficients corresponding to the semantic alignment loss ;in, The calculation formula is: , The calculation formula is: ; Indicates the first Rounds, This represents the Sigmoid function. , , , , , Indicates the parameters of the adjustment function; Based on the task loss, the consistency regularization loss, and the semantic alignment loss, as well as the adjustable coefficient... , and Construct the dynamic loss function for the current round; wherein the formula for calculating the dynamic loss function is: ; This represents the fixed coefficient corresponding to the task loss.

[0077] In an alternative embodiment, the update module 204 is specifically used for: Based on the dynamic loss function of the current round, calculate the gradient of the dynamic loss function of the current round with respect to the parameters of the pre-trained large language model; Based on the gradient and the training hyperparameters, the parameters of the pre-trained large language model are updated to obtain the updated model; wherein, the update formula is: ; This represents the parameters of the pre-trained large language model in round t. This represents the updated parameters of the pre-trained large language model. Let represent the gradient of the dynamic loss function at round t with respect to the parameters of the pre-trained large language model. Let represent the training hyperparameters in round t.

[0078] It should be noted that the beneficial effects of the large model fine-tuning system 200 with coordinated dynamic loss and data construction provided in the above embodiments are the same as those of the large model fine-tuning method with coordinated dynamic loss and data construction, and will not be repeated here. Furthermore, the system provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to the actual situation to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, and will not be repeated here.

[0079] The large model fine-tuning system 200 of the present invention, which combines dynamic loss and data construction, can be a computer program (including program code) running on a computer device. For example, the large model fine-tuning system 200 of the present invention is an application software that can be used to execute the corresponding steps in the large model fine-tuning method of the present invention.

[0080] In some embodiments, the large model fine-tuning system 200 of the present invention, which coordinates dynamic loss and data construction, can be implemented in a combination of hardware and software. As an example, the large model fine-tuning system 200 of the present invention, which coordinates dynamic loss and data construction, can be a processor in the form of a hardware decoding processor, which is programmed to execute the large model fine-tuning method of the present invention. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0081] The modules described in the embodiments of this invention can be implemented in software or hardware. The names of the modules are not, in some cases, limiting the scope of the module itself.

[0082] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-mentioned methods for fine-tuning a large model that combines dynamic loss and data construction. That is, an electronic device according to an embodiment of the present invention may include, but is not limited to: a processor and a memory; the memory is used to store the computer program; the processor is used to execute the method for fine-tuning a large model that combines dynamic loss and data construction as shown in any embodiment of the present invention by calling the computer program.

[0083] In one alternative embodiment, an electronic device is provided, such as Figure 3 As shown, Figure 3 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.

[0084] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0085] Bus 4002 may include a path for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The bus 4002 is represented by only one thick line, but this does not mean that there is only one bus or one type of bus.

[0086] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0087] The memory 4003 stores application code (computer program) for executing the present invention, and its execution is controlled by the processor 4001. The processor 4001 executes the application code stored in the memory 4003 to implement the content shown in the foregoing method embodiments.

[0088] Among them, electronic devices can also be terminal devices. A terminal device can be any terminal device that can install applications and access web pages through applications, including at least one of smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, smart TVs, and smart in-vehicle devices.

[0089] It should be noted that, Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0090] An embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-mentioned methods for fine-tuning large models in conjunction with dynamic loss and data construction.

[0091] Alternatively, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, a floppy disk, and an optical data storage device, etc.

[0092] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the aforementioned large model fine-tuning method that combines dynamic loss and data construction.

[0093] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0094] It should be understood that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0095] The computer-readable storage medium provided in this invention can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0096] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method shown in the above embodiments.

[0097] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

[0098] It should be noted that the terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and represent a limitation on a specific order or sequence. Where appropriate, the order of use for similar objects can be interchanged so that the embodiments of this application described herein can be implemented in an order other than that shown or described.

[0099] Those skilled in the art will recognize that this invention can be implemented as a system, method, or computer program product. Therefore, this invention can be specifically implemented in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, this invention can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.

[0100] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for fine-tuning large models that combines dynamic loss and data construction, characterized in that, include: S1. Construct the initial fine-tuning dataset for the pre-trained large language model, configure the training hyperparameters of the pre-trained large language model, and initialize the adjustable coefficients and corresponding adjustment function parameters in the dynamic loss function of the pre-trained large language model. S2. Based on the current state of the pre-trained large language model, perform uncertainty assessment on the samples in the unlabeled data pool, generate a new training sample set according to the assessment results, combine the new training sample set with the training sample set of the current round to obtain an updated training sample set, and dynamically reweight each training sample in the updated training sample set to construct the training batch data of the current round. S3. Based on the training batch data of the current round, construct a dynamic loss function structure for the current round that includes task loss, consistency regularization loss and semantic alignment loss, and dynamically adjust the adjustable coefficients corresponding to the consistency regularization loss and the semantic alignment loss according to the current round and the adjustment function parameters to obtain the dynamic loss function for the current round. S4. Calculate the gradient based on the dynamic loss function of the current round, and update the parameters of the pre-trained large language model according to the gradient and the training hyperparameters to obtain the updated model; S5. Use the updated model as the pre-trained large language model, and return to execute steps S2-S4 until the model convergence condition is met, then determine the updated model as the final fine-tuned model.

2. The large model fine-tuning method that combines dynamic loss and data construction according to claim 1, characterized in that, S1 specifically includes: Construct and load the pre-trained large language model, wherein the parameters of the pre-trained large language model are θ; Construct the initial fine-tuning dataset ,in, This indicates the number of samples in the initial fine-tuning dataset. This represents the input text of the i-th sample in the initial fine-tuning dataset. This represents the label corresponding to the input text of the i-th sample in the initial fine-tuning dataset; Configure the training hyperparameters of the pre-trained large language model, including optimizer, learning rate scheduler, batch size, and maximum training epochs; Initialize the adjustable coefficients in the dynamic loss function , and and the adjustment function parameters, among which Indicates the training round.

3. The large model fine-tuning method that combines dynamic loss and data construction according to claim 2, characterized in that, The step of performing uncertainty assessment on samples in the unlabeled data pool based on the current state of the pre-trained large language model, and generating a new training sample set based on the assessment results, includes: Based on the current state of the pre-trained large language model, uncertainty is assessed for samples in the unlabeled data pool, and the prediction uncertainty for each sample in the unlabeled data pool is calculated; wherein, the prediction uncertainty is calculated using prediction entropy, and the calculation formula is: ; Indicates sample The uncertainty of prediction This indicates that the pre-trained large language model is in the current parameters The following sample Category The predicted probability, Represents the set of all possible categories. Represents the Shannon entropy function; Samples with prediction uncertainty higher than a preset threshold τ are selected to form a high uncertainty sample set; Based on the high-uncertainty sample set, a new training sample set is generated through semantically preserved data augmentation, knowledge-guided sample generation, or adversarial sample construction.

4. The large model fine-tuning method that combines dynamic loss and data construction according to claim 3, characterized in that, The step of dynamically reweighting each training sample in the updated training sample set to construct the training batch data for the current round includes: Calculate the dynamic weight of each training sample in the updated training sample set. The formula for calculating the dynamic weight is as follows: ;in, Indicates the first The dynamic weights of each training sample. and Indicates hyperparameters, Indicates the first The prediction uncertainty of each training sample Indicates the indicator function, when the first Labels of each training sample Belongs to the predefined rare category set The value is 1 if it is true, and 0 otherwise. Using the dynamic weights of each training sample in the updated training sample set, weighted random sampling is performed on the training samples in the updated training sample set to construct the training batch data for the current round.

5. The large model fine-tuning method that combines dynamic loss and data construction according to claim 4, characterized in that, The step of constructing a dynamic loss function structure for the current round, including task loss, consistency regularization loss, and semantic alignment loss, based on the training batch data of the current round, includes: Based on the training batch data of the current round, the task loss is calculated, and the formula for calculating the task loss is: ;in, Indicates the first Training batch data in each round, Indicates the first The number of samples in the training batch data of each round. Indicates the first The first training batch of data in each round The input text for each sample, Indicates the first The first training batch of data in each round The label of each sample Indicates the first The first training batch of data in each round Dynamic weights of each sample This indicates that the pre-trained large language model is in the first... Round parameters Next to the The first training batch of data in each round Input text for each sample Predict as label The probability of; The consistency regularization loss is calculated using the following formula: ;in, Indicates the first The first training batch of data in each round Input text for each sample The version with random enhancements applied, This indicates that the pre-trained large language model is in the first... Round parameters Next to the The first training batch of data in each round Input text for each sample The predicted probability distribution This indicates that the pre-trained large language model is in the first... Round parameters Below is an enhanced version The predicted probability distribution Indicates the Kullback-Leibler divergence; Calculate the semantic alignment loss, the formula for which is: ;in, This indicates that the pre-trained large language model encoder is in the first... Round parameters The next output The first training batch of data in each round Input text for each sample semantic vectors, Indicates the first The first training batch of data in each round Labels of each sample Predefined semantic embedding; Based on the task loss, the consistency regularization loss, and the semantic alignment loss, the dynamic loss function structure is constructed.

6. The large model fine-tuning method that combines dynamic loss and data construction according to claim 5, characterized in that, The step of dynamically adjusting the adjustable coefficients corresponding to the consistency regularization loss and the semantic alignment loss based on the current round and the adjustment function parameters to obtain the dynamic loss function for the current round includes: Calculate the adjustable coefficient corresponding to the consistency regularization loss based on the current round and the adjustment function parameters. The adjustable coefficients corresponding to the semantic alignment loss ;in, The calculation formula is: , The calculation formula is: ; Indicates the first Rounds, This represents the Sigmoid function. , , , , , Indicates the parameters of the adjustment function; Based on the task loss, the consistency regularization loss, and the semantic alignment loss, as well as the adjustable coefficient... , and Construct the dynamic loss function for the current round; wherein the formula for calculating the dynamic loss function is: ; This represents the fixed coefficient corresponding to the task loss.

7. The large model fine-tuning method that combines dynamic loss and data construction according to claim 6, characterized in that, S4 specifically includes: Based on the dynamic loss function of the current round, calculate the gradient of the dynamic loss function of the current round with respect to the parameters of the pre-trained large language model; Based on the gradient and the training hyperparameters, the parameters of the pre-trained large language model are updated to obtain the updated model; wherein, the update formula is: ; This represents the parameters of the pre-trained large language model in round t. This represents the updated parameters of the pre-trained large language model. Let represent the gradient of the dynamic loss function at round t with respect to the parameters of the pre-trained large language model. Let represent the training hyperparameters in round t.

8. A large model fine-tuning system that coordinates dynamic loss and data construction, characterized in that, include: The initialization module is used to construct the initial fine-tuning dataset of the pre-trained large language model, configure the training hyperparameters of the pre-trained large language model, and initialize the adjustable coefficients and corresponding adjustment function parameters in the dynamic loss function of the pre-trained large language model. The construction module is used to perform uncertainty assessment on samples in the unlabeled data pool based on the current state of the pre-trained large language model, generate a new training sample set according to the assessment result, combine the new training sample set with the training sample set of the current round to obtain an updated training sample set, and dynamically reweight each training sample in the updated training sample set to construct the training batch data of the current round. The adjustment module is used to construct a dynamic loss function structure for the current round, which includes task loss, consistency regularization loss and semantic alignment loss, based on the training batch data of the current round, and dynamically adjust the adjustable coefficients corresponding to the consistency regularization loss and the semantic alignment loss according to the current round and the adjustment function parameters to obtain the dynamic loss function for the current round. The update module is used to calculate the gradient based on the dynamic loss function of the current round, and update the parameters of the pre-trained large language model according to the gradient and the training hyperparameters to obtain the updated model. The iteration module is used to take the updated model as the pre-trained large language model and return to the update module by calling the construction module until the model convergence condition is met, and then the updated model is determined as the final fine-tuned model.

9. An electronic device, characterized in that, The electronic device includes a processor coupled to a memory, the memory storing at least one computer program, which is loaded and executed by the processor to enable the electronic device to implement the large model fine-tuning method for dynamic loss and data construction as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which, when executed by a processor, implements the large model fine-tuning method for dynamic loss and data construction as described in any one of claims 1 to 7.

Images