Strong-weak large model cycle fine-tuning training method for fact-rich conversation content generation

Abstract

The application discloses a strong-weak large model cycle fine-tuning training method for fact-rich conversation content generation, and belongs to the technical field of artificial intelligence. The method comprises the following steps: performing one-time cleaning and cold start on an original fact-rich data set by using a strong large model; mixing the cleaned data and the original data according to a probability to form a mixed data set, and performing initial fine-tuning on a weak large model; performing performance evaluation on the fine-tuned model and calculating a performance value; when the performance value exceeds a dynamic threshold, triggering the weak large model to generate a new answer and updating a historical answer queue; performing multi-round training cycle fine-tuning based on the updated data set, and traversing multiple groups of parameter configurations; and finally saving a performance-optimal model. Through the strong-weak model decoupling cooperation and the cycle self-enhancement mechanism, the application effectively reduces the training cost, avoids overfitting, and improves the performance and generalization ability of the model in the fact-rich conversation generation.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and in particular relates to a method for cyclical fine-tuning training of a strong-weak large model for generating rich factual conversational content. Background Technology

[0002] In the field of rich factual conversational content generation, existing methods primarily rely on generative AI models based on large-scale pre-trained language models, such as BERT and GPT. These models, through pre-training on massive amounts of general text corpora, learn the general rules of language and knowledge representations, and adapt to specific tasks using fine-tuning techniques. They have achieved significant results in scenarios such as intelligent customer service, social media comment generation, and virtual assistants. These methods can generate natural and fluent responses based on the input context and external knowledge base, effectively improving the quality and intelligence level of human-computer interaction, and possessing strong versatility and high coherence of generated content.

[0003] However, existing methods for generating rich factual content based on pre-trained language models still have several significant drawbacks. On the one hand, to ensure the richness and accuracy of the generated content, models with large parameter scales are usually required, resulting in high deployment and fine-tuning costs and slow response times, making it difficult to meet the needs of real-time or high-concurrency scenarios. On the other hand, for specific vertical domains, high-quality, large-scale training corpora rich in factual information are often scarce and the annotation cost is high, making the model prone to overfitting during fine-tuning, resulting in a lack of diversity and sufficient factual basis in the generated content. In addition, although small-scale models have the advantages of low cost and rapid deployment, their limited generalization ability makes it difficult to effectively capture complex semantics and factual relationships in the absence of sufficient high-quality training data, further limiting their application in rich factual conversation generation tasks. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention proposes a method for cyclical fine-tuning training of a strong-weak large model for generating rich factual conversation content, thereby resolving the issues present in the prior art.

[0005] To achieve the above objectives, in a first aspect, the present invention provides a method for iterative fine-tuning training of a strong-weak-large model for generating rich factual conversation content, comprising: Step 1: Clean the original rich fact dataset using a powerful model. The cleaning process includes processing the rich fact text using specialized prompt word templates and regular rules to remove irrelevant information and generate normalized content. Step 2: Mix the cleaned dataset with the original dataset according to a preset probability to form a mixed dataset, and use the mixed dataset to perform the first round of fine-tuning training on the weak model; Step 3: Evaluate the performance of the fine-tuned weak model and calculate the model performance value; Step 4: When the model performance value exceeds the current performance threshold, a new answer is generated by the weak model, and the answer queue is updated, wherein the answer queue is used to store historically generated answers; Step 5: Use the updated dataset to fine-tune the weak model for the next round of training, and repeat steps 3 to 4 for multiple rounds of training until a good-performing weak model is obtained, with different fine-tuning parameter configurations used in each round; Step 6: After completing all rounds of fine-tuning training, save the weak model with the best performance.

[0006] Preferably, the dedicated prompt word template includes system role prompts and user role prompts, wherein the user role prompts contain rich factual text placeholders to be cleaned and cleaning instructions.

[0007] Preferably, the regular expression rules include replacing the URL with a preset marker, replacing the edit timestamp with a preset marker, and removing duplicate punctuation marks.

[0008] Preferably, the mixed dataset is formed by randomly selecting one type of text as an input sample from the original text rich in factual information, the text cleaned by regular expressions, and the text cleaned by a powerful model, according to a preset probability.

[0009] Preferably, the preset probabilities are: the probability of the original text rich in factual information is the first probability value, the probability of the text after regular expression cleaning is the second probability value, and the probability of the text after powerful model cleaning is the third probability value, and the sum of the first probability value, the second probability value and the third probability value is 1.

[0010] Preferably, the model performance value is obtained by calculating the similarity between the fine-tuned model-generated result and the standard result, wherein the similarity calculation uses a word vector encoding model.

[0011] Preferably, the current performance threshold is initially set to a fixed value, and data augmentation is triggered when the model performance value exceeds a preset multiple of the current performance threshold, and then the current performance threshold is updated to the model performance value.

[0012] Preferably, the updated answer queue adopts the first-in, first-out principle, and when the length of the answer queue reaches a preset value, the answer from the earliest round is replaced.

[0013] Preferably, each round of fine-tuning training uses a set of configurations selected from a preset set of fine-tuning parameter configurations, and all configurations are traversed in a loop.

[0014] Preferably, when using the updated dataset, a subset of answers is randomly selected from the answer queue as additional standard answers for training.

[0015] Compared with the prior art, the present invention has the following advantages and technical effects: This invention utilizes a powerful model to perform a one-time cleaning and cold start on the original dataset, while subsequent fine-tuning training is entirely undertaken independently by the weaker model. This avoids the huge computational overhead caused by continuously relying on a powerful model with high parameters, thereby significantly reducing the resource consumption of model deployment and customized training, and effectively reducing the computational resources and time costs of model training.

[0016] This invention dynamically expands and optimizes the training dataset through a weak model self-answer generation and cyclic fine-tuning mechanism, improving sample diversity and quality. This enables the model to learn richer feature patterns from limited data, avoiding overfitting caused by data scarcity.

[0017] This invention introduces a performance evaluation and adaptive threshold triggering mechanism to ensure that data augmentation is only performed when the model performance meets the target. It also combines a historical answer queue to form a multi-objective learning signal, enabling the model to adapt to complex semantics and factual associations. This improves the stability of processing unseen data and enhances the generalization ability and robustness of the augmented model in rich fact conversation generation tasks.

[0018] This invention employs a multi-round training loop fine-tuning and multi-set parameter configuration traversal strategy to systematically explore the hyperparameter space and save the globally optimal model, thereby accelerating the training convergence process and maximizing model performance within a limited number of iterations.

[0019] This invention uses a dedicated prompt word template and regular expression rules to clean and standardize the original text, removing irrelevant information and unifying data representation, providing a high-density, highly consistent input foundation for downstream fine-tuning tasks.

[0020] This invention cleanses the input dataset using a powerful model and rules in one step, and then fine-tunes it multiple times, using a weaker model to learn, update, and optimize the output dataset. This avoids the enormous time and computational costs associated with continuously using a powerful model for data augmentation. The combination of cyclical fine-tuning of the weaker model significantly improves the fine-tuning effect of the large model and shortens the fine-tuning time, even with limited dataset size and lower quality. This results in an expert generation model rich in factual conversational content, demonstrating practicality and feasibility for applications in chatbots, intelligent customer service, and social media sentiment management. It provides a new method for achieving high-quality generation of rich factual conversational content using small- to medium-sized dedicated models. Attached Figure Description

[0021] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of the strong-weak large model cyclic fine-tuning training method for generating rich fact conversation content according to an embodiment of the present invention. Figure 2 This is a flowchart of the weak-large model fine-tuning training and performance evaluation process according to an embodiment of the present invention; Figure 3 This is a flowchart of the weak-large model training dataset hybrid enhancement module according to an embodiment of the present invention; Figure 4 This is a flowchart of the cyclic fine-tuning training and data augmentation module in an embodiment of the present invention. Detailed Implementation

[0022] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0024] Example 1 like Figure 1 As shown, this embodiment provides a method for iterative fine-tuning training of strong and weak models for generating rich factual conversation content. First, a powerful model with high versatility and a large number of parameters is used to initially expand the existing original dataset, focusing on expanding the input and output of the samples in one go to generate semantically similar input text. The expanded dataset is then used to fine-tune the weak model. After fine-tuning, the weak model's performance is evaluated. Based on the model's performance, the dataset is used to generate output text again. The dataset and the newly generated text are combined using a specific method to form a new dataset for the next round of fine-tuning training. The performance of the weak model is monitored in real time throughout the multi-round training and fine-tuning process. After multiple iterations, the fine-tuning training ends, and the model with better performance is saved. This achieves fine-tuning training on datasets with small training sample sizes, low sample quality, and limited output content richness. The method specifically includes: Step 1: Clean the original rich fact dataset using a powerful model. The cleaning process includes processing the rich fact text using specialized prompt word templates and regular rules to remove irrelevant information and generate normalized content. Furthermore, the dedicated prompt word template includes system role prompts and user role prompts, wherein the user role prompts contain rich factual text placeholders to be cleaned and cleansing instructions.

[0025] Furthermore, the regular expression rules include replacing URLs with preset tags, replacing edit timestamps with preset tags, and removing duplicate punctuation marks.

[0026] Specifically, rich factual information includes credible textual data such as news reports, interview transcripts, regulations, and media event information. The original dataset consists of input text (collected rich factual information, events, and issues) and output text, which, along with configurable prompts, are fed into a more powerful model with more parameters for data cleaning. The configurable prompts contain irrelevant, invalid, and harmful information that needs to be removed from the original dataset, and require the large model to maintain consistency in its representation of the original data. During the powerful model's cleaning of the original text data, the textual representation of the original dataset becomes more consistent, the structure more standardized, and the textual information density further improved, laying a high-quality cold-start data foundation for subsequent efficient fine-tuning of the weaker model. The powerful model's cleaning prompt template is shown in Table 1.

[0027] Table 1 Among them, the prompt word templates for the user part can be reset or replaced; "{rich fact text to be cleaned}" is the rich fact information collected from the original dataset. The cleaned rich fact information text will be stored separately to expand the fine-tuned training dataset. See step 2 for specific usage principles.

[0028] In addition, this module will perform simple regular expression processing on the rich factual text to be cleaned, replacing the URL and edit timestamp in the text with "<-Web_URL->" and "<-ED_Time->", and also removing duplicate punctuation marks. The regular expression cleaning principles are shown in Table 2.

[0029] Table 2 Step 2: Mix the cleaned dataset with the original dataset according to a preset probability to form a mixed dataset, and use the mixed dataset to perform the first round of fine-tuning training on the weak model; Furthermore, the mixed dataset is formed by randomly selecting one type of text as an input sample from the original text rich in factual information, the text cleaned by regular expressions, and the text cleaned by a powerful model, according to a preset probability.

[0030] Furthermore, the preset probabilities are as follows: the probability of the original text rich in factual information is the first probability value, the probability of the text after regularity cleaning is the second probability value, and the probability of the text after powerful model cleaning is the third probability value, and the sum of the first probability value, the second probability value and the third probability value is 1.

[0031] Specifically, the weak model fine-tuning training and performance evaluation module mixes the powerful model with the training dataset cleaned using regularization rules. This involves selecting cleaned or uncleaned rich factual information from the input samples based on set probability values, and combining it with events and questions to form a mixed dataset. The weak model will then undergo fine-tuning training based on this mixed dataset. After this fine-tuning training, performance evaluation will be performed. The fine-tuned weak model will generate results on the validation set samples, and the generated results will be compared with the standard results in terms of similarity. The final similarity score will be used as the performance value of the fine-tuned model and passed to the next stage module. A flowchart of this process is shown below. Figure 2 As shown.

[0032] The rules for selecting rich factual information in this process are as follows: (1) in, This indicates the rich factual information that was ultimately selected for this training. This represents the original, factual text. This represents rich factual information cleaned using regular expression rules. This indicates rich factual information after cleaning using a powerful model. This indicates the probability that this type of text will be selected; only one of the aforementioned rich factual information will be selected in a single round of fine-tuning.

[0033] Step 3: Evaluate the performance of the fine-tuned weak model and calculate the model performance value; Furthermore, the model performance value is obtained by calculating the similarity between the fine-tuned model-generated result and the standard result, wherein the similarity calculation uses a word vector encoding model.

[0034] Specifically, the model is fine-tuned during this process. The principles of performance evaluation are as follows: (2) in, This represents the model after this fine-tuning. The numerical values ​​from 0 to 1 represent the model's performance values. This indicates that the similarity between two vectors is calculated. This indicates the result produced by the model. Indicates the theoretical standard result, This indicates the text word vectorization operation using the LongBERT word vector encoding model as the scoring model.

[0035] Step 4: When the model performance value exceeds the current performance threshold, a new answer is generated by the weak model, and the answer queue is updated, wherein the answer queue is used to store historically generated answers; Furthermore, the current performance threshold is initially set to a fixed value, and data augmentation is triggered when the model performance value exceeds a preset multiple of the current performance threshold, and then the current performance threshold is updated to the model performance value.

[0036] Furthermore, the updated answer queue adopts the first-in, first-out principle. When the length of the answer queue reaches a preset value, the answer from the earliest round is replaced.

[0037] Specifically, the weak model training dataset hybrid enhancement module will perform a hybrid enhancement based on the model performance values ​​received in step 3 and the set performance threshold. (Initial value set to 0.5) Compare; if it is greater than the set performance threshold... If the value is 1.05, then the model is fine-tuned. The results from the samples in this training dataset will be generated and combined with the previous fine-tuned answers to form the multi-round training and fine-tuning answer. The multi-round training and fine-tuning answer includes the answers generated by the better fine-tuning model in the previous N rounds. Therefore, the replacement rule is to replace the answer generated by the earliest fine-tuning model, and the replacement principle is as follows: (3) in, This is the training data that will be used in the next round after the current update; This represents the training data used in the previous round, which includes the fine-tuned answers retained from at least the first N rounds; This represents the fine-tuned answer generated in the i-th round; This indicates calculating the length of the sequence. This represents the fine-tuned answer generated in the current round.

[0038] The resulting multi-round training fine-tuning answer library serves as one of the standard answers to be learned during iterative fine-tuning training, and a performance threshold is set. Set as the model performance value for this model The optimal model is set to the currently received fine-tuned model. The flowchart is as follows: Figure 3 As shown.

[0039] Step 5: Use the updated dataset to fine-tune the weak model for the next round of training, and repeat steps 3 to 4 for multiple rounds of training until a good-performing weak model is obtained, with different fine-tuning parameter configurations used in each round; Furthermore, each round of fine-tuning training uses a set of configurations selected from a preset set of fine-tuning parameter configurations, and all configurations are traversed in a loop.

[0040] Furthermore, when using the updated dataset, a subset of answers is randomly selected from the answer queue as additional standard answers for training.

[0041] Specifically, the cyclic fine-tuning training and data augmentation module will use the training dataset, which has undergone the hybrid orchestration and augmentation steps described above, for fine-tuning training. The dataset includes original text data, text data cleaned from the large model, text data cleaned from rules, events and questions, standard answers, and fine-tuned answers. For each input test case, there will be one standard answer and multiple answers randomly selected from the fine-tuned answers, which will be used as multiple samples for learning. This round of fine-tuning will use one of a pre-set series of fine-tuning parameter configurations. After this round of fine-tuning, the newly obtained fine-tuned model will be... The training data, along with the data used, is passed to step 3 for model performance evaluation. After data augmentation in step 4, the fine-tuning dataset is used for the next round of iterative fine-tuning. This process is repeated N times until all user-defined fine-tuning parameters have been tested. A flowchart is shown below. Figure 4 As shown.

[0042] The adjusted answer selection rules are as follows: (4) in, This is the slightly adjusted answer; This represents the fine-tuned answer generated in the i-th round; This indicates the final, slightly adjusted answer. Indicates randomness from sequence Select from One sample, This indicates that the length of the sequence is being calculated.

[0043] Step 6: After completing all rounds of fine-tuning training, save the weak model with the best performance.

[0044] Specifically, in this embodiment, after completing all iterative fine-tuning tasks, the optimal fine-tuned model will be evaluated. The model performance values The training set and other data are saved and stored in a specified local location.

[0045] The beneficial effects of this embodiment: This invention innovatively proposes a fine-tuning method for data augmentation using strong and weak models. This method cleans the input dataset using a strong model and rules in one go, and then, after multiple fine-tuning iterations, the weak model learns and updates its own answers to expand and optimize the dataset output. This achieves efficient fine-tuning training of the weak model even with limited dataset quantity and quality. By combining strong and weak models for data augmentation, this invention effectively reduces the continuous cost of using the strong model. During multiple fine-tuning iterations, the weak model continuously optimizes and expands the dataset through self-learning, reducing the high-frequency dependence on the strong model and thus significantly lowering the demand for computing resources and time costs. The strong model's initial dataset cleaning ensures data quality, while the iterative learning of the weak model further refines and enriches the dataset. This collaborative approach accelerates the data processing flow, enabling faster model training convergence and improving overall efficiency. In the rapidly iterating AI development environment, improving training and inference efficiency is particularly important. Since the weak model learns on its own and continuously extracts new features and patterns from the data, this process helps enhance the model's generalization ability.

[0046] Compared to data augmentation that relies solely on powerful models, the method of this invention can generate more diverse training samples, making the final model more robust when faced with unseen data. This invention provides a method for scenarios with more factual information, such as chat conversations, intelligent customer service Q&A, and social media sentiment management, to generate high-quality responses based on factual information by fine-tuning a weak model on specific tasks or datasets while maintaining a relatively small parameter size and deployment / training cost. This method meets the specific needs of different fields and application scenarios.

[0047] In summary, the innovative fine-tuning method of this invention not only solves the problem of high costs caused by powerful model data augmentation, but also improves the performance, training effect and generalization ability of weak models, providing new solutions and methods for the widespread application and sustainable development of AI technology.

[0048] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for iterative fine-tuning training of a strong-weak model for generating rich factual conversation content, characterized in that, Includes the following steps: Step 1: Clean the original rich fact dataset using a powerful model. The cleaning process includes processing the rich fact text using specialized prompt word templates and regular rules to remove irrelevant information and generate normalized content. Step 2: Mix the cleaned dataset with the original dataset according to a preset probability to form a mixed dataset, and use the mixed dataset to perform the first round of fine-tuning training on the weak model; Step 3: Evaluate the performance of the fine-tuned weak model and calculate the model performance value; Step 4: When the model performance value exceeds the current performance threshold, a new answer is generated by the weak model, and the answer queue is updated, wherein the answer queue is used to store historically generated answers; Step 5: Use the updated dataset to fine-tune the weak model for the next round of training, and repeat steps 3 to 4 for multiple rounds of training until a good-performing weak model is obtained, with different fine-tuning parameter configurations used in each round; Step 6: After completing all rounds of fine-tuning training, save the weak model with the best performance.

2. The method according to claim 1, characterized in that, The dedicated prompt template includes system role prompts and user role prompts, wherein the user role prompts contain placeholders for rich factual text to be cleaned and cleaning instructions.

3. The method according to claim 1, characterized in that, The regular expression rules include replacing URLs with preset tags, replacing edit timestamps with preset tags, and removing duplicate punctuation marks.

4. The method according to claim 1, characterized in that, The hybrid dataset is formed by randomly selecting one type of text as the input sample from the original text rich in factual information, the text cleaned by regular expressions, and the text cleaned by a powerful model, according to a preset probability.

5. The method according to claim 4, characterized in that, The preset probabilities are as follows: the probability of the original text rich in factual information is the first probability value, the probability of the text after regular expression cleaning is the second probability value, and the probability of the text after powerful model cleaning is the third probability value, and the sum of the first probability value, the second probability value and the third probability value is 1.

6. The method according to claim 1, characterized in that, The model performance value is obtained by calculating the similarity between the fine-tuned model's generated result and the standard result, where the similarity calculation uses a word vector encoding model.

7. The method according to claim 1, characterized in that, The current performance threshold is initially set to a fixed value, and data augmentation is triggered when the model performance value exceeds a preset multiple of the current performance threshold, and then the current performance threshold is updated to the model performance value.

8. The method according to claim 1, characterized in that, The updated answer queue adopts the first-in, first-out principle. When the length of the answer queue reaches a preset value, the answer from the earliest round is replaced.

9. The method according to claim 1, characterized in that, Each round of fine-tuning training uses a set of configurations selected from multiple preset sets of fine-tuning parameter configurations, and all configurations are traversed in a loop.

10. The method according to claim 1, characterized in that, When using the updated dataset, a subset of answers is randomly selected from the answer queue as additional standard answers for training.

Images