A model evolution learning method for vertical fields

Abstract

The application provides a model evolution learning method for vertical fields, comprising: obtaining high-quality field data; training a model into a first model with preliminary field knowledge through an instruction fine-tuning method; guiding the evolution of the first model using a large model, including: (1) first model answer generation, (2) the large model gives a systematic evaluation, (3) when the large model determines that the solution of the first model does not meet the expectation, a demonstrative answer is generated, the demonstrative answer, the answer of the last round model and the score of the large model are input into the first model, and S31-S32 are repeatedly executed until the large model finally approves the solution of the first model or the iteration number reaches a preset upper limit; fine-tuning the performance of the first model to generate a second model. Through the application, GPT4 distills high-quality field data, guides the evolution of the model, and uses the method of model self-game to get rid of the dependence on GPT4, and realizes the evolution of the self-capability of the model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing and deep learning, and in particular to a model evolution learning method for vertical fields. BACKGROUND

[0002] In the current field of information technology and computer science, natural language processing technology has made significant progress thanks to the rapid development of large language models. Large language models have shown strong performance in general capabilities, but due to their high resource and cost requirements, they have not been widely applied to various vertical field scenarios. For example, for a vertical field, it is not always necessary for the model to have extensive general knowledge or capabilities, and more importantly, it can focus on educational tasks or educational domain knowledge. Therefore, it is very important to train a model for education and apply it to a vertical field.

[0003] However, due to the lack of vertical field knowledge, the model often needs to evolve its capabilities in the field before it can be applied to the vertical field. In this process, the current method has the following shortcomings:

[0004] (1) Lack of systematic method guidance for model field evolution. Existing methods usually only use a single strategy to improve the performance of the model in the vertical field. This method lacks a gradual performance improvement strategy, making it impossible to optimize for models with different capabilities. And the existing evolution training target method is not clear, lacking systematic training process guidance.

[0005] (2) Lack of high-quality vertical field data. Models often need to inject vertical field knowledge to improve their field capabilities in order to achieve capability evolution in the vertical field. Due to the scarcity and complexity of domain knowledge, it is difficult to obtain large-scale, high-quality domain training data. Distillation from GPT4 is an effective method to obtain domain knowledge, but due to hallucinations and other problems, the resulting domain data often lacks accuracy. How to distill high-quality domain data from GPT4 is the key to model evolution.

[0006] (3) Limitations of GPT4 dependency. In the model evolution process, the model often relies on GPT4 feedback to achieve capability evolution. However, excessive reliance on GPT4 limits the model's ability to learn and optimize independently, making it difficult to explore and innovate independently during training. Secondly, this training method limits the model's ability to improve in the vertical field, making it difficult to break through the limitations of GPT4 itself. SUMMARY

[0007] To solve the above problems, the application provides a model evolution learning method for vertical fields. First, when the model does not have domain knowledge, the method of guiding the strong model by the weak model is used to distill the domain knowledge of the large model and inject it into the model, so that the model has initial domain ability. Then, the large model iteratively feeds back the solution of the model in the field problem, and realizes the evolution of the large model guided model. Finally, the idea of self-game is used to guide the self-evolution of the model, and the dependence on the large model is broken. Finally, the evolution of the model in the vertical field is realized.

[0008] The model evolution learning method for vertical fields provided by the application comprises:

[0009] Step 1: Obtain high-quality domain data;

[0010] Step 2: Train the model to be a first model with preliminary domain knowledge by the instruction fine-tuning method;

[0011] Step 3: Use the large model to guide the evolution of the first model; comprising:

[0012] Step 31: First model answer generation: based on the first model's answer to the field problem, generate the reasoning path and answer;

[0013] Step 32: Large model provides feedback: the large model gives a systematic evaluation of the reasoning path and answer;

[0014] Step 33: When the large model determines that the solution of the first model does not meet the expectations, a demonstrative answer will be generated, the demonstrative answer, the answer of the last round model and the score of the large model are input into the first model, and steps 31-32 are repeated until the large model finally approves the solution of the first model, or the iteration number reaches the preset upper limit;

[0015] Step 34: Fine-tune the performance of the first model to generate a second model.

[0016] Further, step 1 comprises:

[0017] Step 11: Crawl domain data, including domain problems, specific descriptions of the domain problems, user answers and the number of likes of user answers;

[0018] Step 12: Construct a prompt for the domain data, and generate guide information for the domain data through the model;

[0019] Step 13: Concatenate the guide information of each piece of domain data with the domain problem through the prompt and input it into the large model, the large model will generate a corresponding answer for each piece of data, and store the domain problem and the answer pair as distillation data.

[0020] Further, in step 11, it further comprises:

[0021] Ensure the diversity of domain data; diversity includes: domain data length is various, the category of domain data is various;

[0022] De-noising;

[0023] If the number of answers and the number of likes of the domain question exceed the threshold value, the data is retained as high-quality domain data.

[0024] Further, in step 31, the domain question in the distilled data is input into the first model, and the answer is generated by adopting the method of thought chain reasoning.

[0025] Further, in step 32, the systematic evaluation includes evaluation of the current reasoning path, modification of the current reasoning path, evaluation of the answer and modification of the answer.

[0026] Further, in step 33, all the texts generated by the first model and the large model are saved as domain data accumulated based on the multi-round feedback of the large model, which is used for fine-tuning in step 34.

[0027] Further, in step 34, the large model strictly checks all the prompt words and generated content, and converts them into context expressions with logical coherence and fluent language, and uses the Adam optimizer and cross-entropy loss function to continuously fine-tune the first model.

[0028] Further, in step 32, the large model analyzes the rationality of the reasoning path, determines the logical connection between the inference and the intermediate hypothesis in each reasoning step, and the logical loopholes or unreasonable assumptions in the reasoning.

[0029] Further, it also includes using the self-game method to evolve the second model, including:

[0030] Step 41, input the domain data into the second model, and generate two types of reasoning paths and answer pairs through the greedy search strategy and beam search strategy;

[0031] Step 42, the second model uses the reasoning path and answer pair generated by the greedy search and beam search to train, so that the result generated by the second model is close to the beam search strategy and far away from the greedy search strategy.

[0032] Further, the high FLOPs path is assigned a positive score, and the low FLOPs path is assigned a negative score.

[0033] The model evolution learning method for vertical fields proposed in the application has the following beneficial effects:

[0034] (1) Model capability improvement: through domain knowledge distillation, GPT4 guides model evolution and model self-game, and realizes the improvement of model capability in vertical fields;

[0035] (2) High-quality knowledge distillation: using the idea of guiding strong models with weak models, distilling higher-quality field data from GPT4 to provide models for field learning;

[0036] (3) Model self-capability breakthrough: using the method of model self-game, breaking away from the dependence on GPT4, realizing the breakthrough of model self-capability. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor.

[0038] Figure 1 The flowchart of the model evolution learning method of one embodiment of the present application;

[0039] Figure 2 The flowchart of the distillation method of guiding strong models with weak models of one embodiment of the present application;

[0040] Figure 3 The flowchart of the GPT4 guided model evolution of one embodiment of the present application;

[0041] Figure 4 The flowchart of the model self-game of one embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical scheme and advantages of the embodiments of the present application more clear, the technical scheme of the embodiments of the present application will be described in detail below with the drawings of the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0043] The present application proposes a model evolution learning method for vertical fields, aiming to propose a comprehensive solution for model evolution in vertical fields, so that the model can fully understand the field-specific problems existing in the vertical field and can give targeted solutions. The field capability generalization of the general model in the vertical field shows the practicability and effectiveness of the model in the vertical field. As shown in Figure 1 , the model evolution learning method for vertical fields, as shown in Figure 1 , includes:

[0044] Step 1: Obtain high-quality domain data;

[0045] Step 2: Train the model as a first model with preliminary domain knowledge through the instruction fine-tuning method;

[0046] Step 3: Use the large model to guide the first model to evolve into a second model.

[0047] In step 1, the model evolution method is used to make the model (i.e., the model to be evolved) have the ability to answer questions in a vertical field. First, the model must have preliminary domain knowledge. Therefore, the method collects high-quality domain data, including high diversity, high-quality domain questions, and high-quality domain question answers.

[0048] Step 11: Domain knowledge question crawling

[0049] Taking the computer science vertical field as an example, we select high-quality domain questions from the StackOverFlow website (Stack Overflow is a well-known software developer forum operated by StackExchange, where users can ask and answer various programming questions). The website contains a large amount of data related to computer science, and each data has multiple artificial answer schemes, covering various categories in the computer science field, so it is suitable for use as training data in this field.

[0050] Vertical domain data usually contains four parts: 1. Domain question. 2. Specific description of the domain question. 3. User's answer. 4. Number of likes for the user's answer.

[0051] After crawling the domain data, preprocessing is required. First, the higher the diversity of the domain data, the stronger the generalization of the trained model, and the more adaptable the model is to various scenarios in the vertical field. In an embodiment, diversity includes data length diversity and data category diversity. Then, noise removal and other processing are performed.

[0052] (1) Use data of different lengths to determine the diversity of domain data. The following examples are provided for illustration. If the model to be evolved is Llama3-8B-Chat (or Qwen2-7B-Chat, but not limited to these two models), according to the maximum length of the model, the question data is divided into four parts: length less than 128 tokens, length between 128 tokens and 256 tokens, length between 256 tokens and 512 tokens, and length greater than 512 tokens. For each length of data, an equal amount of data will be left to ensure the diversity of domain data length.

[0053] (2) The domain model should be able to be applied to more scenarios in the field, so the category diversity of the domain data is important. For example, in the computer vertical field, in order to ensure the category diversity of the domain data, we divide the data into four categories according to the classification of the user's question: machine learning, deep learning, natural language processing, and computer vision. While ensuring the length diversity of the domain data, we also ensure the category diversity of the domain data.

[0054] (3) Data quality is crucial for model learning, especially when we need the model to learn domain knowledge from a small amount of high-quality data. Therefore, unlike large-scale text data used for pre-training, all noise should be removed from the domain data as much as possible. Therefore, using the web parser Beautiful Soup, we use its powerful parsing ability to remove noise from the title <h1>To< / h1> <h6>), paragraph ( ), span ( ), block ( ​ ) and list ( extract the text content in the webpage. To ensure the purity of the extraction results, we perform a post-judgment on the extracted results. If the data contains non-text elements such as HTML, CSS, JS, expressions, images, and videos, we will delete the data.

[0055] In addition, there are also a large number of meaningless domain questions on the website. These questions and answers may not contain much domain information, and training the model using these data may not improve the domain ability of the model. However, if the number of answers to the question is high and there are high-upvoted answers, it means that this data is discussed by many people and is a more meaningful data content. Therefore, we can define the threshold of the number of answers to a question as 5 and the highest upvoted number of questions as 10. When the corresponding content of the data exceeds the threshold, we retain the data as high-quality domain data. Table 1 shows the total amount and categories of data after crawling and processing the data from the StackOverFlow website in the computer domain.

[0056] Table 1: Classification and size of raw data

[0057] Category Machine Learning Deep Learning Natural Language Processing Computer Vision Total Scale 4605 2092 1881 1698 10276

[0058] Step 12: The model generates guideline information according to the prompt

[0059] Since there is a cognitive bias between the large model (GPT4 is used as an example in the following) and the model (i.e., the model to be evolved), directly inputting the data crawled in step 11 into GPT4 to distill the domain knowledge of GPT4 may not necessarily make the model learn efficiently. This method will adopt the strategy of weak model guiding strong model generation (as shown in Figure 2 To obtain the cognitive way of the model to the domain question, and use this cognitive way to guide GPT4 to generate data, which can to some extent alleviate the cognitive bias between GPT4 and the model.

[0060] To obtain the model's understanding of the problem in this field, the method encapsulates a prompt to let the model output what steps it thinks should be taken to solve the problem in this field, which are called guideline information. After obtaining the guideline information, our purpose is to let the large model generate the answer to the problem in the field according to the guideline information. Therefore, in this step, the method inputs the field problem in the high-quality field data generated in step 11 through a prompt, i.e., "You are an expert in the field of [a certain special field category]. I have a question that needs to be answered: [the user question crawled]. You do not need to answer this question yourself, as I will use a more powerful model, GPT-4, to provide the answer. However, to ensure that GPT-4 can answer the question more accurately, please provide a detailed guideline to help it respond more precisely. The guideline is:"

[0061] For example: You are an expert in the field of Machine Learning. I have a question that needs to be answered: "Title: Best Practices for Handling Missing Values ​​in Machine Learning?\n\n Body: I ​​have a dataset with missing values ​​in both numerical and categorical features. What are some recommended strategies for handling these missing values ​​before training a machine learning model using Python and scikit-learn? Should I treat numerical and categorical features differently? Any advice or example code would be helpful. Thanks!" You do not need to answer this question yourself, as I will use a more powerful model, GPT-4, to provide the answer. However, to ensure that GPT-4 can answer the question more accurately, please provide detailed guidelines to help it respond more precisely. The guidelines are: "Have the weak model generate the corresponding guideline information. If the generated result is a 'repeated output' problem, have the model regenerate the guideline information for that data pair until all data pairs generate the corresponding guideline information."

[0062] At this point, the guidance information generated by the model is:

[0063] 1. Context Understanding: The question is about preprocessing data in Python using scikit-learn, specifically handling missing values in both numerical and categorical features.

[0064] 2. Strategies Overview.

[0065] 3. Code Snippets: Provide simple scikit-learn examples for imputation.

[0066] 4. Best Practices: Recommend validating the impact of imputation on model performance and testing different methods.

[0067] 5. Additional Tips: Mention that some models handle missing values natively (e.g., XGBoost).

[0068] Step 13: Large Model Field Knowledge Distillation Based on Guidelines Information

[0069] The field problem is input into a large model (such as GPT4, which is used as an example below), and the large model outputs the answer to the question. The evolved field model and GPT4 may have cognitive bias problems. The guideline information generated in step 12 for each piece of data is spliced with the field problem (user question) using the following prompt: "Below is a guideline provided by an expert in the field of [specific field], followed by a specific question that needs to be answered. Please use the provided guideline to generate a precise and accurate answer to the question. guideline: [Insert guideline here] Question: [specific question]", and then input into a large model (such as GPT4). The large model will generate a corresponding answer for each piece of data. Store the question and answer pairs as distillation data.

[0070] Use a large model (such as GPT4) to score the distillation data. By comparing the GPT Score of distillation data with and without guideline information, the effectiveness of the weak model guiding the strong model strategy in improving the quality of distillation data is demonstrated. Table 2 reflects the comparison of the effects of knowledge distillation based on guideline information and knowledge distillation not based on guideline information.

[0071] Table 2: Comparison of the effects of knowledge distillation based on guideline information and knowledge distillation not based on guideline information

[0072]

[0073] Step 2: Use the instruction fine-tuning method to generate a preliminary evolved model, called the first model.

[0074] Based on the distillation data, the method of instruction fine-tuning is adopted, and the Adam optimizer and cross-entropy loss function are used to train and fine-tune the model, generating the first model with preliminary vertical field answering ability. The specific training and fine-tuning method is a prior art, which will not be described here. Input the first model field problem, the first model will generate the corresponding answer. The accuracy score of the answer can be evaluated using GPT4 to determine the quality of the answer; then the average score of the answer score is used as the score of the model to evaluate the pros and cons of the model. In addition, in order to further prove the effectiveness of the weak model guiding the strong model strategy, the scores of the distillation data fine-tuned model with and without guideline information are compared, and the results are shown in Table 3.

[0075] Table 3: Average scores of distillation data fine-tuned models with and without guideline information

[0076]

[0077] Table 3: Experimental results, the performance of the model fine-tuned by distillation data with guideline information is stronger, further verifying the effectiveness of the weak model guiding the strong model strategy.

[0078] Step 3: Large model guides the evolution of the first model to generate the second model.

[0079] In order to further enhance the ability of the first model in the vertical field, the present application designs a multi-round iteration method of guiding the evolution of the first model by the large model, as shown in Figure 3 which includes four steps of first model answer generation, GPT4 feedback, multi-round iteration correction, and model performance fine-tuning.

[0080] Step 31: First model answer generation: based on the first model's answer to the field problem, generate the reasoning path and answer;

[0081] In this step, we input the field problem (or user question) in the distillation data into the first model, and use the thought chain reasoning method to generate the answer. Thought chain reasoning is a method of gradually expanding the reasoning process, which can help the model to decompose complex problems into easy-to-understand sub-steps. This method can significantly improve the logical consistency and coherence of the reasoning structure of the model when solving problems. Specifically, the first model first receives the question and generates a reasoning path using its domain knowledge. Each step of reasoning explicitly generates intermediate hypotheses and inferences, thereby avoiding missing key details or jumping in logic. Compared with the traditional direct answer generation method, the thought chain reasoning method not only improves the accuracy of the answer, but also enhances the sensitivity and interpretability of the model to reasoning problems. In addition, this mechanism promotes the model to continuously correct itself in the subsequent reflection process, and finally generates an answer with complete structure and accurate content.

[0082] Step 32, GPT4 provides model feedback: GPT4 gives an evaluation for the reasoning path and the answer, and scores the answer;

[0083] GPT4 conducts a systematic evaluation of the first model's output, including the answer and the reasoning path. GPT4 analyzes the rationality of the reasoning path, reviews the logical connection between the inference and the intermediate hypothesis in each reasoning step, and explicitly points out logical flaws or unreasonable assumptions in the reasoning. In addition, GPT4 scores the final answer through preset accuracy indicators. The systematic evaluation given by GPT4 includes evaluation of the current reasoning path, modification of the current reasoning path, evaluation of the answer, and modification of the answer. This systematic evaluation has a dual effect: on the one hand, it helps the model identify and correct weaknesses in reasoning; on the other hand, it provides clear guidance for subsequent iterative corrections. This mode simulates the process of receiving expert advice in human learning, and promotes the continuous improvement of the model's logical reasoning and vertical field adaptation ability.

[0084] Step 33, if the large model judges that the answer does not meet the expectations, generate a demonstrative answer, and then perform S31, S32.

[0085] The demonstrative answer is provided by GPT4 based on existing knowledge and reasoning logic. The demonstrative answer is used as part of the prompt, and then steps S31-S32 are performed until GPT4 finally approves the model's solution, or the number of iterations reaches the preset upper limit, completing the iteration and saving all text generated by the first model and GPT4 as domain data based on GPT4 multi-round feedback accumulation, which is used for fine-tuning in step 34. During the execution of step 31, the first model will modify the reasoning path generated in the previous round according to the demonstrative answer.

[0086] Multiple iterations expose the performance defects of the model in different reasoning scenarios and guide it to learn and correct in a targeted manner, ensuring gradual improvement of problem-solving ability. This process not only accelerates the learning curve of the model, but also significantly enhances its performance in complex reasoning tasks, laying a solid foundation for future applications.

[0087] Step 34, fine-tune the performance of the first model to generate the second model: fine-tune the first model based on the domain data accumulated by GPT4 multi-round feedback to improve its performance;

[0088] The entire reasoning path, answer, and GPT4 feedback are used as training data to fine-tune the model. In one embodiment, to ensure the effectiveness of the data, the large model strictly checks all prompt words and generated content and converts them into logically coherent and linguistically fluent contextual expressions. This processing step not only optimizes the structure of the prompt words, but also ensures the high quality and availability of subsequent training data. After accumulating a number of (e.g., 10) samples, the first model is continuously fine-tuned using the Adam optimizer and cross-entropy loss function. The Adam optimizer, with its efficient gradient update mechanism, can improve the generalization ability and problem-solving accuracy of the model in a relatively short time. Through this fine-tuning process, the first model gradually absorbs and integrates knowledge and continuously improves its performance in solving domain problems, providing stable and efficient reasoning and answering capabilities for subsequent applications.

[0089] Table 4 shows that after completing step 3, GPT4 is used to evaluate the accuracy, completeness, relevance, logic, and reliability of the answers generated by the pre-evolution and post-evolution models, and to select which answer is more in line with expectations. From the performance of each dimension, the post-evolution model has significantly improved, indicating the effectiveness of the method.

[0090] Table 4: Comparison of GPT4-guided model evolution performance before and after

[0091]

[0092]

[0093] In one embodiment, to enable the second model to diversify and optimize itself, the present application further includes:

[0094] Step 4, evolving the second model using a self-game method based on different reasoning complexities, including generating different reasoning paths and training based on contrastive learning, as shown in Figure 4 .

[0095] Step 41, generating diverse reasoning paths based on different reasoning FLOPs (floating-point operations per second).

[0096] In this step, the domain data is input into the second model, and both greedy search and beam search strategies are used to generate diverse reasoning paths to meet the needs of different reasoning complexity. Specifically, the greedy search strategy selects the option with the highest probability at each step, generating a higher efficiency but possibly lacking global optimality, suitable for handling simple or having obvious logic tasks. This strategy uses low FLOPs computing resources to quickly give approximate solutions, which is an ideal choice for resource-limited applications. However, to deal with complex reasoning tasks and deep logical analysis, the beam search strategy can be used, which has a higher FLOPs consumption and is suitable for scenarios that require deep reasoning. Beam search ensures a wider global search space by retaining multiple candidate solutions (beam width set to 10) at each step, thereby improving the accuracy and reasonableness of the answer. This strategy retains multiple possible paths during reasoning and selects and optimizes them based on the principle of global optimality.

[0097] This step will generate two reasoning paths and answer pairs for the input question to the model: one is the reasoning path and answer pair generated by greedy search, and the other is the reasoning path and answer pair generated by beam search.

[0098] Step 42, training based on contrastive learning.

[0099] After generating diverse reasoning paths, the second model uses contrastive learning to optimize its reasoning ability. Contrastive learning is a deep learning paradigm that aims to improve the model's discrimination ability through positive and negative sample comparison. In this stage, the second model is trained using the answers generated by both greedy search and beam search. According to the scaling law in the reasoning process of the second model, the higher the FLOPs used in the reasoning process, the better the reasoning performance will be. Therefore, the answers generated by the beam search strategy are naturally better than those generated by the greedy search strategy. Therefore, during the self-evolution of the second model, we make the results generated by the second model approach the beam search strategy and move away from the greedy search strategy, so as to optimize its reasoning ability to a state with a more global perspective and logical consistency.

[0100] Specifically, the cross-entropy loss function is used to compare and score the paths generated by greedy search and beam search: high FLOPs paths are given positive scores due to their reasoning depth and potential accuracy, while low FLOPs paths are given negative scores due to their relatively simple reasoning process. The specific training is a prior art, which will not be described here. Through contrastive learning, the model continuously self-strengthens learning in self-game, gradually improving its performance in various reasoning scenarios. Through this self-game mechanism, the model can effectively extract efficiency advantages from low FLOPs paths, while learning complex reasoning logic from high FLOPs paths, forming the ability of self-evolution.

[0101] Table 5 shows the accuracy, completeness, relevance, logic, and reliability of the answers selected by GPT4 before and after the model self-evolution. From the performance of each dimension, the model after self-evolution has a significant improvement, indicating the effectiveness of the model self-evolution.

[0102] Table 5: Comparison of GPT4 guided model evolution performance before and after

[0103]

[0104] Those skilled in the art can understand that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope defined by the claims of the present application. < / h6>

Claims

1. A model evolution learning method for vertical domains, characterized in that, The method includes: Step 1: Obtain high-quality domain data; Step 2: Train the model into a first model with preliminary domain knowledge using the instruction fine-tuning method; Step 3: Use the large model to guide the evolution of the first model; including: Step 31: First Model Answer Generation: Based on the first model, answer the domain question and generate reasoning paths and answers; Step 32: The large model provides feedback: The large model provides a systematic evaluation of the reasoning path and the answer; Step 33: When the large model determines that the solution of the first model does not meet expectations, it will generate a model answer. The model answer, the answer of the previous model, and the score of the large model will be input into the first model. Steps 31-32 will be repeated until the large model finally approves the solution of the first model, or the number of iterations reaches the preset limit. Step 34: Fine-tune the performance of the first model to generate the second model; Step 4: Evolve the second model using a self-game approach, including: Step 41: Input domain data into the second model and generate two types of reasoning paths and answer pairs using a greedy search strategy and a bundle search strategy; Step 42: The second model is trained using inference paths and answer pairs generated by greedy search and bundle search, so that the results generated by the second model are closer to the bundle search strategy and farther away from the greedy search strategy.

2. The model evolution learning method according to claim 1, characterized in that, Step 1 includes: Step 11: Crawl domain data, which includes domain questions, detailed descriptions of those questions, user answers, and the number of likes on those answers; Step 12: Build hints for domain data, and generate guidance information for domain data through the model; Step 13: Concatenate the guidance information and domain question for each piece of domain data with the prompts and input them into the large model. The large model will generate a corresponding answer for each piece of data and store the domain question and answer pair as distilled data.

3. The model evolution learning method according to claim 2, characterized in that, Step 11 also includes: Ensure the diversity of domain data; diversity includes: diverse domain data length and diverse domain data categories; Noise reduction; If the number of answers and likes for a domain question exceeds a threshold, the data is retained as high-quality domain data.

4. The model evolution learning method according to claim 1, characterized in that, In step 31, the domain problem in the distillation data is input into the first model, and the solution is generated using the thought chain reasoning method.

5. The model evolution learning method according to claim 1, characterized in that, In step 32, the systematic evaluation includes evaluating the current reasoning path, modifying the current reasoning path, evaluating the answer, and modifying the answer.

6. The model evolution learning method according to claim 1, characterized in that, In step 33, all the text generated by the first model and the large model is saved as domain data accumulated based on multiple rounds of feedback from the large model, which is used for fine-tuning in step 34.

7. The model evolution learning method according to claim 1, characterized in that, In step 34, the large model rigorously verifies all prompts and generated content, transforms them into logically coherent and fluent contextual expressions, and continuously fine-tunes the first model using the Adam optimizer and cross-entropy loss function.

8. The model evolution learning method according to claim 1, characterized in that, Step 32: Analyze the rationality of the reasoning path in the large model analysis, and determine the logical connection between the inference and intermediate assumptions in each reasoning step, as well as logical loopholes or unreasonable assumptions in the reasoning.

9. The model evolution learning method according to claim 1, characterized in that, In step 42, high FLOPs paths are assigned positive scores, and low FLOPs paths are assigned negative scores.

Images