A training method and system for multi-model collaborative distillation and dynamic fine-tuning models

Abstract

The present invention relates to the technical field of computer data processing, and in particular to a training method and system for a multi-model collaborative distillation and dynamic fine-tuning model; the method comprises: collecting data, inputting unlabeled data into a model to make labeled data, integrating the data with labeled data into a data set, and then dividing the data into a training set, a validation set and a test set; inputting the training set into the model, using prompt words to guide the model to generate extended data, and using the extended training set to train a student model; evaluating the student model on the validation set, and if the evaluation result shows that the evaluation index of the student model does not reach the expected value, then inputting error information into the model for analysis and data generation, and then adjusting the student model until the evaluation index of the student model reaches the expected value; the method has the advantages of using multiple large language models for collaborative distillation, making the training data richer and more diverse, and the student model more accurate, and then dynamically fine-tuning the student model through a cycle, so that the model has higher accuracy and stronger adaptability.

Description

Technical Field

[0001] The present invention relates to the field of computer data processing technology, and in particular to a training method and system for a multi-model collaborative distillation and dynamic fine-tuning model. Background Art

[0002] In the field of artificial intelligence and natural language processing, knowledge distillation has become a widely used technology. By transferring the knowledge from a large-scale and high-performance teacher model to a smaller student model, the size of the model can be greatly reduced without significantly losing model performance. The main methods of knowledge distillation include directly distilling the results of the output layer and distilling intermediate features layer by layer. In addition, prompt engineering has gradually become a technology that effectively improves model performance. By designing appropriate prompt words, it can effectively guide large language models to generate more expected results.

[0003] In the above context, researchers have proposed a variety of methods based on knowledge distillation and hint engineering to improve the performance and efficiency of natural language processing models; these methods are mainly used to improve the reasoning efficiency and task adaptability of student models in terms of model compression, model fine-tuning, and model optimization for specific tasks, but these methods cannot solve the lack of model generalization ability and dynamic adaptability.

[0004] For example, a Chinese patent, whose application number is 202410652249.4 and application date is 2024.05.24, and the patent name is "A text summary generation model training method, system and storage medium based on knowledge distillation". Its technical solution is: a text summary generation model training method, system and storage medium based on knowledge distillation. The present invention collects text data from different fields as training data, fine-tunes the collected teacher model and student model, and uses the reasoning process of the teacher model and the labels generated in the reasoning process to perform distillation training on the student model, which helps to achieve efficient and high-performance text summary content generation, and can solve the problem of text summary content generation under low-resource conditions with high quality.

[0005] The above patent uses the reasoning process of the teacher model and the labels generated in the reasoning process to perform distillation training on the student model, which helps to achieve efficient and high-performance text summary content generation. However, the patent does not provide a specific testing and evaluation method for the student model, nor can it dynamically adjust and optimize the model. Summary of the invention

[0006] In order to solve the problems existing in the prior art, the present invention provides a training method and system for multi-model collaborative distillation and dynamic fine-tuning models that can perform multi-model collaborative distillation and dynamically fine-tune the optimization model.

[0007] In order to achieve the above technical effects, the technical solution of this application is as follows:

[0008] In the first aspect, a training method for a multi-model collaborative distillation and dynamic fine-tuning model includes the following specific steps:

[0009] Step 1: Collect and integrate the data set, input the unlabeled data in the data set into multiple large language models to make labeled data, and integrate it with the labeled data in the data set to form a complete data set, and then divide it into training set, validation set and test set;

[0010] Step 2: Input the training set into multiple large language models, guide the multiple large language models to generate more extended data through prompt words, add the extended data to the extended training set, and use the extended training set to train a student model;

[0011] Step 3: Evaluate the trained student model on the validation set and make a judgment based on the evaluation results. If the evaluation results show that the evaluation indicators of the student model do not reach the expected values, the error information is input into multiple large language models for analysis and data generation. The student model is then adjusted based on the analyzed and generated data. The student model is dynamically adjusted in this cycle until the evaluation indicators of the student model reach the expected values ​​and the adjustment is ended.

[0012] Furthermore, in step one, the specific method of converting unlabeled data into labeled data is: inputting the unlabeled data into multiple large language models to generate labeled data; performing statistics on the generated labeled data, taking the mode of the statistical results as the final labeled data, and integrating it with the labeled data in the data set into one data set; in step two, different prompt words are selected including "generate reasons for choosing the correct answer" and "generate more questions"; the prompt word "generate reasons for choosing the correct answer" guides multiple large language models to generate explanations and reasons for choosing the answers; the prompt word "generate more questions" guides multiple large language models to generate more questions; the expanded training set includes the original questions, the newly generated questions, the answers to the newly generated questions, and the reasons for generating the answers.

[0013] Furthermore, in step three, the student model is evaluated on the validation set, and a judgment is made based on the evaluation results. When the evaluation index reaches or exceeds the expected value, the evaluation is ended after the last verification on the test set.

[0014] Furthermore, the specific steps for evaluating the student model on the validation set are as follows:

[0015] Step 1: Predict and solve the validation set. The validation set is , where N is the number of samples in the validation set, is the input text sequence, is the corresponding target output sequence; the student model generates a prediction by generating a probability distribution of words or characters at each position , for each input , the model predicts a probability distribution ;

[0016] Step 2: Calculate the loss function and use the cross entropy loss to calculate the difference between the prediction of the next word generated by the model and the real word. The specific formula of the cross entropy loss is:

[0017] ;

[0018] In the formula, N is the number of samples in the validation set, T is the length of the word sequence of each sample, is the true word label of the i-th sample at time step t, is the probability distribution predicted by the i-th sample at time step t;

[0019] Step 3: Calculate the evaluation indicators, refer to the calculated evaluation indicators, and evaluate and analyze the performance of the student model on the validation set; the evaluation indicators include accuracy, perplexity and BLEU score; the specific solution formula for accuracy is:

[0020] ;

[0021] Where N is the number of samples in the validation set, express The value is 1 when it is, otherwise it is 0;

[0022] The perplexity is used to reflect the model's ability to predict the next word on the validation set. The lower the perplexity value, the more realistic the selection distribution of the model during prediction and the better the performance. The specific solution formula for perplexity is:

[0023] ;

[0024] Where N is the number of samples in the validation set, It is the probability of the model predicting the target output sequence corresponding to the target word under a given context input text sequence. exp is an exponential function with the natural constant e as the base in higher mathematics.

[0025] The BLEU score is used to evaluate the overlap between the generated text and the reference text. The specific solution formula for the BLEU score is:

[0026] ;

[0027] Where N is the number of samples in the validation set, BP is the penalty factor used to prevent the model from generating short sentences. is the overlap ratio of n-grams, is the weight of the corresponding n-gram.

[0028] Furthermore, the specific method of analyzing and generating data in step three is: analyzing the wrong answers of the student model on the validation set, generating more questions and options based on the error information, inputting these newly generated questions and options into multiple large language models for training, and the large language model regenerates new wrong questions and their accurate answers based on the error information, adding them to the training set used to adjust the student model, and then inputting the training set into the student model for cyclic adjustment.

[0029] Furthermore, when the student model is adjusted dynamically in a loop, if the number of loop adjustments is greater than the maximum number of loop adjustments set in the loop, it is considered that the model has reached the adjustment limit and the adjustment is terminated.

[0030] Second, a multi-model collaborative distillation and dynamic fine-tuning model training system, including:

[0031] The data processing module is used to collect and integrate the data set, make the unlabeled data in the data set into labeled data, and integrate it with the labeled data in the data set into a complete data set, and then divide the complete data set into a training set, a validation set, and a test set;

[0032] The pre-training module inputs the training set into multiple large language models, guides the multiple large language models to generate more extended data through prompt words, adds the extended data to the extended training set, and uses the extended training set to train a student model;

[0033] In the subsequent adjustment module, the trained student model is evaluated on the validation set and a judgment is made based on the evaluation results. If the evaluation results show that the evaluation indicators of the student model do not reach the expected values, the error information is input into multiple large language models for analysis and data generation, and then the student model is adjusted. The student model is dynamically adjusted in this cycle until the evaluation indicators of the student model reach the expected values ​​and the adjustment is ended.

[0034] Furthermore, the data processing module includes a data collection and preliminary processing module, a multi-model voting module and a data integration module; the data collection and preliminary processing module collects unlabeled data related to the target professional field, and then inputs the unlabeled data into multiple different large language models, performs statistics on the generated labeled data, and then takes the mode of the statistical results as the final labeled data, and integrates it with the labeled data in the data set to become a data set, and divides it into three parts: a training set, a validation set and a test set; the multiple large language models include gpt, glm, qwen and llama.

[0035] Furthermore, the pre-training module includes a large language model generation module, a data supplementation module and a student model training module; the training set generated by the data processing module is input into the large language model module, and is input into multiple large language models in the large language module through different prompt words, so as to guide the multiple large language models to generate a large amount of extended data; the supplemented extended data is added to the training set as an extended training set; the extended training set is input into the large language model again to train a student model.

[0036] Furthermore, the subsequent adjustment module includes a verification set evaluation module, an evaluation result judgment module, a data analysis generation module and a dynamic adjustment module; the trained student model is evaluated on the verification set evaluation module, and the evaluation result judgment module determines whether the evaluation index meets the expectations; if the expectations are met, the process ends; if the expectations are not met, the process enters the data analysis generation module, extracts and analyzes the error information on the learning model, and the data generated based on the error information is input into the large language model again to generate new data, and then the accurate data is added to the training set. Finally, the dynamic adjustment module adjusts the student model based on the supplemented training set to meet the model expectations.

[0037] According to the above technical solution, the present application has the following beneficial effects:

[0038] 1. The present invention adopts a multi-model collaborative distillation and dynamic fine-tuning model training method. This method uses multiple large language models for collaborative distillation, making the training data richer and more diverse, and the obtained student model more accurate. The student model is then adjusted dynamically through a cycle to make the model more accurate and adaptable.

[0039] 2. The multi-model collaborative distillation and dynamic fine-tuning model training method of the present invention generates more accurate data by combining prompt words to guide the training of multiple large language models, which significantly improves the decision-making quality of the trained student model and the transparency in application.

[0040] 3. The multi-model collaborative distillation and dynamic fine-tuning model training method of the present invention adopts a method of training dynamically adjusting the student model, so that the student model can be adaptively and dynamically adjusted according to the actual model performance, ensuring that the student model can meet the expected performance standards and improving the accuracy and stability of the student model.

[0041] 4. The present invention adopts a training system of multi-model collaborative distillation and dynamic fine-tuning model. The system obtains a student model with high precision and stronger generalization ability through training and adjustment of data processing module, pre-training module and subsequent fine-tuning module.

[0042] 5. The multi-model collaborative distillation and dynamic fine-tuning model training system of the present invention, the large language model generation module of the system simultaneously uses multiple large language models to collaboratively generate training data, thereby improving the diversity and coverage of the data, thereby avoiding the problem of possible deviation in the results obtained from the training of a single large language model, and enhancing the generalization ability of the student model.

[0043] 6. The multi-model collaborative distillation and dynamic fine-tuning model training method and system of the present invention adopts a smaller-scale student model and a dynamic adjustment strategy, achieves a balance between resource consumption and performance improvement, optimizes resource utilization efficiency, and improves training efficiency. Compared with the existing technology, larger-scale models or more training resources are usually required to achieve similar effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A flowchart of data processing;

[0045] Figure 2 This is a flowchart of pre-training;

[0046] Figure 3 A flowchart for subsequent adjustments. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0048] Example 1

[0049] A training method for a multi-model collaborative distillation and dynamic fine-tuning model includes the following specific steps:

[0050] Step 1: Collect and integrate the data set, input the unlabeled data in the data set into multiple large language models to make labeled data, and integrate it with the labeled data in the data set to form a complete data set, and then divide it into training set, validation set and test set; Figure 1 As shown, the unlabeled data can be professional field questions and the corresponding four options are option a, option b, option c and option d, and the labeled data can be professional field questions, the four options corresponding to the professional field questions are option a, option b, option c and option d, and the actual options included in the answer; the collection of data sets can include text, technical documents, question-answer pairs and other contents;

[0051] Step 2: Input the training set into multiple large language models, guide the multiple large language models to generate more extended data through prompt words, add the extended data to the extended training set, and use the extended training set to train a student model;

[0052] Step 3: Evaluate the trained student model on the validation set and make a judgment based on the evaluation results. If the evaluation results show that the evaluation indicators of the student model do not reach the expected values, the error information is input into multiple large language models for analysis and data generation. The student model is then adjusted based on the analyzed and generated data. The student model is dynamically adjusted in this cycle until the evaluation indicators of the student model reach the expected values ​​and the adjustment is ended.

[0053] In step 1, the specific method of converting unlabeled data into labeled data is as follows: input the unlabeled data into multiple large language models, and use the diversity of multiple large language models to generate labeled data, such as the answer label of the question, whether the text is an accurate label, etc.; the multiple different large language models can be gpt, glm, qwen and llama, etc.; the generated labeled data are counted, and then the mode of the statistical results is taken as the final labeled data, and it is integrated with the labeled data in the data set into one data set; the method of taking the mode is to apply a multi-model voting mechanism to count the multiple answers to each question, and select the answer with the highest frequency as the final answer to the question; in step 2, different prompt words are selected, including "generate reasons for choosing the correct answer" (prompt word 1) and "generate more questions" (prompt word 2); the prompt word "generate reasons for choosing the correct answer" (prompt word 1) guides Multiple large language models generate explanations and reasons for selecting answers, and generate explanations and reasons for selecting specific answers, explaining why a specific answer is selected. This prompt word can not only enhance the diversity of data, but also obtain knowledge information that is more critical to model training; the prompt word "generate more questions" (prompt word 2) guides multiple large language models to generate more questions, which are similar to the original questions; the newly generated questions have different focus patterns and are combined into the training set. As an extended training set, they are used together with the corresponding answers, which can greatly increase the samples and diversity of training and are suitable for scenarios and fields with insufficient data; the generated new questions include more similar questions, answers to the newly generated questions, and reasons for the generated answers. Because they are to be input into the training set as a supplement, the extended training set also includes the original questions, the generated new questions, the answers to the newly generated questions, and the reasons for the generated answers;

[0054] like Figure 3As shown in the figure, in step 3, the student model is evaluated on the validation set, and a judgment is made based on the evaluation results. When the evaluation index reaches or exceeds the expected value, the evaluation is ended after the last verification on the test set. The verification set contains some new data that has not participated in the previous training. Using the verification set for evaluation can test the performance of the student model on unseen data and measure its generalization ability. Because the test set is completely independent of the training set and validation set data, the purpose of the last verification on the test set is to comprehensively evaluate the generalization ability and final performance of the student model.

[0055] The specific steps for evaluating the student model on the validation set are as follows:

[0056] Step 1: Predict and solve the validation set. The validation set is , where N is the number of samples in the validation set, is the input text sequence, is the corresponding target output sequence; the student model (small LLM) generates a prediction by generating a word or character probability distribution at each position , for each input , the model predicts a probability distribution ;

[0057] Step 2: Calculate the loss function and use the cross entropy loss to calculate the difference between the prediction of the next word generated by the model and the real word. The specific formula of the cross entropy loss is:

[0058] ;

[0059] In the formula, N is the number of samples in the validation set, T is the length of the word sequence of each sample, is the true word label of the i-th sample at time step t, is the probability distribution predicted by the i-th sample at time step t;

[0060] Step 3: Calculate the evaluation indicators. Refer to the calculated evaluation indicators to evaluate and analyze the performance of the student model on the validation set. The evaluation indicators include accuracy, perplexity and BLEU score. The evaluation method can rely on a single indicator or combine multiple evaluation indicators by weighted merging. The specific evaluation criteria are determined by different tasks and data formats. If the evaluation is based on Figure 1 , Figure 2 and Figure 3 The sample data shown is in the form of multiple-choice questions, so the accuracy rate can be used as a single evaluation indicator; the specific solution formula for the accuracy rate is:

[0061] ;

[0062] Where N is the number of samples in the validation set, express The value is 1 when it is, otherwise it is 0;

[0063] The perplexity is used to reflect the model's ability to predict the next word on the validation set. The lower the perplexity value, the more realistic the model's selection distribution is during prediction and the better the performance is. The specific solution formula for perplexity is:

[0064] ;

[0065] Where N is the number of samples in the validation set, It is the probability of the model predicting the target output sequence corresponding to the target word under a given context input text sequence. exp is an exponential function with the natural constant e as the base in higher mathematics.

[0066] The BLEU score is used to evaluate the overlap between the generated text and the reference text. The specific solution formula for the BLEU score is:

[0067] ;

[0068] Where N is the number of samples in the validation set, BP is the penalty factor used to prevent the model from generating sentences that are too short. is the overlap ratio of n-grams, is the weight of the corresponding n-gram.

[0069] like Figure 3As shown in FIG, the specific method of analyzing and generating data in step three is: first extract samples of wrong answers on the student model, extract wrong answer data with low scores on the validation set through the obtained evaluation indicators, and analyze the wrong answers of the student model on the validation set. The specific content of the wrong answer analysis includes summarizing the questions on which the student model made mistakes and the knowledge blind spots that these mistakes may reflect; to summarize the questions on which the student model made mistakes, it is necessary to design different information summary prompt words for multiple large language models. The prompt words can be written according to the official prompt word templates recommended by the large language models in the prior art, to ensure that multiple large language models can accurately summarize and output relatively uniform patterns in the error samples. formula, and the reasons that lead to the student model errors, etc.; according to the sample information of the wrong answers, it is input into multiple large language models to generate more questions and options, and these newly generated questions and options are input into multiple large language models for training again. The large language model regenerates according to the error information, including generating accurate answers according to the questions, generating question types and options according to the summary, analyzing error information, and summarizing related knowledge, which are added to the training set used to adjust the student model, and then the supplemented training set is input into the student model for cyclic adjustment; the purpose of regeneration including generating accurate answers according to the questions, generating similar question types and options according to the summary, analyzing error information, and summarizing related knowledge is to supplement training for the weaknesses of the model;

[0070] When the student model is adjusted dynamically in a loop, when the evaluation index, such as accuracy, reaches or exceeds the expected value, the adjustment is ended and the last test is performed on the test set; when the number of loop adjustments is greater than the maximum number of loop adjustments set, that is, Figure 3 If the number of cycles is greater than the maximum value, it is considered that the model has reached the adjustment limit and the adjustment is terminated. In different fields and tasks, the data sets and expected values ​​are also different. For example, in the medical field, a relatively strict accuracy rate is required, and the expected value is high. In the field of e-commerce customer service, the required accuracy rate is not so strict, and the expected value can be relaxed. In general, the expected value is equivalent to the practical evaluation bottom line of the model. If it is difficult to define the expected value artificially, the data obtained from the multiple large language models mentioned in the present invention can be used to obtain the performance indicators of the multiple large language models by taking the mode method. The expected value can be established with reference to the indicator data. The expected value definition method belongs to the conventional prior art and will not be repeated here.

[0071] Example 2

[0072] A training system for multi-model collaborative distillation and dynamic fine-tuning models, including:

[0073] like Figure 1As shown, the data processing module is used to collect and integrate the data set, make the unlabeled data in the data set into labeled data and integrate the labeled data in the data set into a complete data set, and then divide the complete data set into a training set, a validation set and a test set;

[0074] like Figure 2 As shown, the pre-training module inputs the training set into multiple large language models, guides the multiple large language models to generate more extended data through prompt words, adds the extended data into the extended training set, and uses the extended training set to train a student model; wherein the meanings of (prompt word 11+question: answer+reason) / (prompt word 2+question: answer) in the extended training set are respectively to generate the corresponding answer and reason according to the prompt word "generate reasons for choosing the correct answer" (prompt word t1), and the answers generated according to the prompt word "generate more questions" (prompt word 2) are more similar questions.

[0075] like Figure 3 As shown, in the subsequent adjustment module, the trained student model is evaluated on the validation set and a judgment is made based on the evaluation results. If the evaluation results show that the evaluation indicators of the student model do not reach the expected values, the error information is input into multiple large language models for analysis and data generation, and then the student model is adjusted. The student model is dynamically adjusted in this cycle until the evaluation indicators of the student model reach the expected values ​​and the adjustment is ended.

[0076] The data processing module includes a data collection and preliminary processing module, a multi-model voting module and a data integration module; the data collection and preliminary processing module collects unlabeled data related to the target professional field, and then inputs the unlabeled data into multiple different large language models, performs statistics on the generated labeled data, and then takes the mode of the statistical results as the final labeled data, and integrates it with the labeled data in the data set to become a data set, which is divided into three parts: training set, validation set and test set; multiple large language models include gpt, glm, qwen and llama.

[0077] The pre-training module includes a large language model generation module, a data supplementation module and a student model training module; the training set generated by the data processing module is input into the large language model module, and is input into multiple large language models in the large language module through different prompt words to guide multiple large language models to generate extended data; the supplemented extended data is added to the training set as an extended training set; the extended training set is input into the large language model again to train a student model.

[0078] The subsequent adjustment modules include a validation set evaluation module, an evaluation result judgment module, a data analysis generation module and a dynamic adjustment module; the trained student model is evaluated on the validation set evaluation module, and the evaluation result judgment module is used to determine whether the evaluation indicators meet expectations; if expectations are met, the process ends; if expectations are not met, the process enters the data analysis generation module, extracts and analyzes the error information on the learning model, and inputs the data generated based on the error information into the large language model to generate new data, and then adds the accurate data to the training set. Finally, the dynamic adjustment module adjusts the student model based on the supplemented training set to meet model expectations.

[0079] The above description is a detailed description of the preferred feasible embodiment of the present application, but the embodiment is not intended to limit the patent application scope of the present application. All equivalent changes or modified changes made under the technical spirit suggested by the present application should fall within the patent scope covered by the present application.

Claims

1. A training method for a multi-model collaborative distillation and dynamic fine-tuning model, characterized in that: The specific steps include: Step 1: Collect and integrate the data set, input the unlabeled data in the data set into multiple large language models to produce labeled data, and integrate it with the labeled data in the data set into a complete data set, and then divide it into a training set, a validation set, and a test set; the specific method of making unlabeled data into labeled data is: input the unlabeled data into multiple large language models to generate labeled data; perform statistics on the generated labeled data, and then take the mode of the statistical results as the final labeled data, and integrate it with the labeled data in the data set into a data set; Step 2: Input the training set into multiple large language models, guide the multiple large language models to generate extended data through prompt words, add the extended data to the extended training set, and use the extended training set to train a student model; different prompt words are selected in step 2, including "generate reasons for choosing the correct answer" and "generate more questions"; the prompt word "generate reasons for choosing the correct answer" guides multiple large language models to generate explanations and reasons for choosing the answers; the prompt word "generate more questions" guides multiple large language models to generate questions; the extended training set includes the original questions, the generated new questions, the answers to the newly generated questions, and the reasons for generating the answers; Step three: Evaluate the trained student model on the validation set and make a judgment based on the evaluation results. If the evaluation results show that the evaluation index of the student model does not reach the expected value, the error information is input into multiple large language models for analysis and data generation, and then the student model is adjusted based on the analyzed and generated data. The student model is adjusted dynamically in this way until the evaluation index of the student model reaches the expected value and the adjustment is ended. The specific method of analyzing and generating data in step three is: analyze the wrong answers of the student model on the validation set, generate questions and options based on the error information, and input these newly generated questions and options into multiple large language models for training. The large language model regenerates new wrong questions and their answers based on the error information, adds them to the training set used to adjust the student model, and then inputs the training set into the student model for cyclic dynamic adjustment. When the student model is cyclically adjusted, when the number of cyclic adjustments is greater than the set maximum number of cyclic dynamic adjustments, it is considered that the model has reached the adjustment limit and the adjustment is ended.

2. The method for training a multi-model collaborative distillation and dynamic fine-tuning model according to claim 1, characterized in that: In step three, the student model is evaluated on the validation set, and a judgment is made based on the evaluation results. When the evaluation index reaches or exceeds the expected value, the evaluation is ended after the final verification on the test set.

3. The training method of multi-model collaborative distillation and dynamic fine-tuning model according to claim 2, characterized in that: The specific steps for evaluating the student model on the validation set are as follows: Step 1: Predict and solve the validation set. The validation set is , where N is the number of samples in the validation set, is the input text sequence, is the corresponding target output sequence; the student model generates a prediction by generating a probability distribution of words or characters at each position , for each input , the model predicts a probability distribution ; Step 2: Calculate the loss function and use the cross entropy loss to measure the difference between the prediction of the next word generated by the model and the real word. The specific formula of the cross entropy loss is: ； In the formula, N is the number of samples in the validation set, T is the length of the word sequence of each sample, is the true word label of the i-th sample at time step t, is the probability distribution predicted by the i-th sample at time step t; Step 3: Calculate the evaluation indicators, refer to the calculated evaluation indicators, and evaluate and analyze the performance of the student model on the validation set; the evaluation indicators include accuracy, perplexity and BLEU score; the specific solution formula for accuracy is: ； Where N is the number of samples in the validation set, express The value is 1 when it is, otherwise it is 0; The perplexity is used to reflect the model's ability to predict the next word on the validation set. The lower the perplexity value, the more realistic the selection distribution of the model during prediction and the better the performance. The specific solution formula for perplexity is: ； Where N is the number of samples in the validation set, It is the probability of the model predicting the target output sequence corresponding to the target word under a given context input text sequence. exp is an exponential function with the natural constant e as the base in higher mathematics. The BLEU score is used to evaluate the overlap between the generated text and the reference text. The specific solution formula for the BLEU score is: ； Where N is the number of samples in the validation set, BP is the penalty factor used to prevent the model from generating short sentences. is the overlap ratio of n-grams, is the weight of the corresponding n-gram.

4. A multi-model collaborative distillation and dynamic fine-tuning model training system, characterized in that: include: The data processing module is used to collect and integrate the data set, make the unlabeled data in the data set into labeled data, and integrate it with the labeled data in the data set into a complete data set, and then divide the complete data set into a training set, a validation set and a test set; the specific method of making the unlabeled data into labeled data is: input the unlabeled data into multiple large language models to generate labeled data; perform statistics on the generated labeled data, and then take the mode of the statistical results as the final labeled data, and integrate it with the labeled data in the data set into a data set; The pre-training module inputs the training set into multiple large language models, guides the multiple large language models to generate extended data through prompt words, adds the extended data to the extended training set, and uses the extended training set to train a student model; selects different prompt words including "generate reasons for choosing the correct answer" and "generate more questions"; the prompt word "generate reasons for choosing the correct answer" guides multiple large language models to generate explanations and reasons for choosing the answer; the prompt word "generate more questions" guides multiple large language models to generate questions; the extended training set includes the original question, the generated new question, the answer to the newly generated question, and the reason for generating the answer; The subsequent adjustment module evaluates the trained student model on the validation set and makes a judgment based on the evaluation results. If the evaluation results show that the evaluation indicators of the student model do not reach the expected values, the error information is input into multiple large language models for analysis and data generation, and then the student model is adjusted. The student model is adjusted dynamically in this way until the evaluation indicators of the student model reach the expected values ​​and the adjustment is terminated. The specific method of analyzing and generating data is as follows: the incorrect answers of the student model on the validation set are analyzed, and questions and options are generated based on the error information. These newly generated questions and options are input into multiple large language models for training. The large language model regenerates new incorrect questions and their answers based on the error information, and adds them to the training set used to adjust the student model. The training set is then input into the student model for cyclic dynamic adjustment. When the student model is cyclically adjusted, when the number of cyclic adjustments is greater than the set maximum number of cyclic dynamic adjustments, it is considered that the model has reached the adjustment limit and the adjustment is terminated.

5. The multi-model collaborative distillation and dynamic fine-tuning model training system according to claim 4, characterized in that: The data processing module includes a data collection and preliminary processing module, a multi-model voting module and a data integration module; the data collection and preliminary processing module collects unlabeled data in the target professional field, and then inputs the unlabeled data into multiple large language models, performs statistics on the generated labeled data, and then takes the mode of the statistical results as the final labeled data, and integrates it with the labeled data in the data set to become a data set, and divides it into three parts: a training set, a validation set and a test set; the multiple large language models include gpt, glm, qwen and llama.

6. The multi-model collaborative distillation and dynamic fine-tuning model training system according to claim 4, characterized in that: The pre-training module includes a large language model generation module, a data supplementation module and a student model training module; the training set generated by the data processing module is input into the large language model module, and is input into multiple large language models in the large language module through different prompt words to guide the multiple large language models to generate extended data; the supplemented extended data is added to the training set as an extended training set; the extended training set is input into the large language model again to train a student model.

7. The multi-model collaborative distillation and dynamic fine-tuning model training system according to claim 4, characterized in that: The subsequent adjustment modules include a validation set evaluation module, an evaluation result judgment module, a data analysis generation module and a dynamic adjustment module; the trained student model is evaluated on the validation set evaluation module, and the evaluation result judgment module is used to determine whether the evaluation indicators meet expectations; if expectations are met, the process ends; if expectations are not met, the process enters the data analysis generation module, extracts and analyzes the error information on the learning model, and the data generated based on the error information is input into the large language model again to generate new accurate data, and then the accurate data is added to the training set. Finally, the dynamic adjustment module adjusts the student model based on the supplemented training set to meet model expectations.

Images