Model evolution learning method oriented to vertical field

Abstract

The invention provides a vertical domain-oriented model evolution learning method. The method comprises the steps of obtaining high-quality domain data; training the model into a first model with preliminary domain knowledge through an instruction fine tuning method; guiding the evolution of the first model by utilizing the large model, including (1) generating answers of the first model, (2) performing systematic evaluation on the large model, and (3) when the large model judges that the answers of the first model do not conform to expectation, generating demonstrative answers, inputting the demonstrative answers, answers of the previous round of model and scores of the large model into the first model, and repeatedly executing S31-S32, until the large model ultimately accepts the solution of the first model, or the number of iterations reaches a preset upper limit; and finely adjusting the performance of the first model to generate a second model. According to the method, the GPT4 distills high-quality domain data to guide the model evolution, and a model self-gaming method is utilized to get rid of the dependence on the GPT4 and realize the evolution of the self-ability of the model.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing and deep learning technology, and in particular to a model evolutionary learning method for a vertical field. Background Art

[0002] In the current field of information technology and computer science, natural language processing technology has made significant progress thanks to the rapid development of large language models. Large language models have shown strong performance in general capabilities, but due to their high resource and cost requirements, they have not been widely used in scenarios in various vertical fields. For example, for a certain vertical field, it is not always necessary for the model to have extensive general knowledge or capabilities. What is more important is that it can focus on educational tasks or domain knowledge of education. Therefore, it is very important to train models for education and apply them to vertical fields.

[0003] However, due to the lack of vertical domain knowledge, models often need to evolve their capabilities in the field before they can be applied to vertical fields. In this process, current methods have the following shortcomings:

[0004] (1) Model domain evolution lacks systematic methodological guidance. Existing methods usually only use a single strategy to improve the performance of the model in a vertical field. This method lacks a progressive performance improvement strategy, resulting in the inability to make targeted optimizations for models with different capabilities. In addition, the existing evolutionary training target methods are not clear and lack systematic training process guidance.

[0005] (2) Lack of high-quality vertical domain data. Models usually need to be injected with vertical domain knowledge to improve their domain capabilities in order to achieve capability evolution in the vertical field. Due to the scarcity and complexity of domain knowledge, it is difficult to obtain large-scale, high-quality domain training data. Distillation from GPT4 is an effective way to obtain domain knowledge. However, due to problems such as hallucinations, the generated domain data is often inaccurate. How to distill high-quality domain data from GPT4 is the key to achieving model evolution.

[0006] (3) Limitations of GPT4 dependency. During the model evolution process, the model often relies on the feedback of GPT4 to achieve capability evolution. However, over-reliance on GPT4 limits the model's autonomous learning and optimization capabilities, causing it to lack independent exploration and innovation mechanisms during the training process. Secondly, this training method limits the model's ability to improve in vertical fields, making it difficult to break through the limitations of GPT4 itself. Summary of the invention

[0007] In order to solve the above problems, the present invention proposes a model evolution learning method for vertical fields. First, when the model does not have domain knowledge, the domain knowledge of the large model is distilled by the method of weak model guiding strong model, and injected into the model, so that the model has initial domain capabilities. Then, the large model iteratively feeds back the model's answers to domain problems, so that the large model guides the model evolution. Finally, the idea of ​​self-game is used to guide the model's self-evolution and get rid of the dependence on the large model. Finally, the evolution of the model in the vertical field is realized.

[0008] The vertical field-oriented model evolution learning method proposed in the present invention includes:

[0009] Step 1: Obtain high-quality domain data;

[0010] Step 2: Train the model to be a first model with preliminary domain knowledge through instruction fine-tuning method;

[0011] Step 3: Use the large model to guide the evolution of the first model; including:

[0012] Step 31: First model answer generation: answer the domain question based on the first model, generate reasoning paths and answers;

[0013] Step 32: The big model provides feedback: The big model gives a systematic evaluation of the reasoning path and the answer;

[0014] Step 33: When the large model determines that the answer of the first model does not meet expectations, it will generate a demonstrative answer, input the demonstrative answer, the answer of the previous model, and the score of the large model into the first model, and repeat steps 31-32 until the large model finally recognizes the answer of the first model or the number of iterations reaches the preset upper limit;

[0015] Step 34, fine-tune the performance of the first model to generate a second model.

[0016] Furthermore, step 1 includes:

[0017] Step 11: Crawl domain data, which includes domain questions, specific descriptions of the domain questions, user answers, and the number of likes for user answers;

[0018] Step 12: Construct prompts for domain data and generate guidance information for domain data through the model;

[0019] Step 13: The guidance information and domain questions of each domain data are spliced ​​together through prompts and input into the big model. The big model will generate corresponding answers for each data and store the domain question and answer pairs as distilled data.

[0020] Furthermore, in step 11, it also includes:

[0021] Ensure the diversity of domain data; diversity includes: diverse length of domain data and diverse categories of domain data;

[0022] Denoising;

[0023] If the number of answers and likes to a domain question exceeds the threshold, the data is retained as high-quality domain data.

[0024] Furthermore, in step 31, the domain problems in the distilled data are input into the first model, and the solution is generated by using the thought chain reasoning method.

[0025] Furthermore, in step 32, the systematic evaluation includes evaluation of the current reasoning path, modification of the current reasoning path, evaluation of the answer, and modification of the answer.

[0026] Furthermore, in step 33 , all texts generated by the first model and the large model are saved as domain data accumulated based on multiple rounds of feedback from the large model, and used for fine-tuning in step 34 .

[0027] Furthermore, in step 34, the large model strictly verifies all prompt words and generated content, and converts them into logically coherent and fluent contextual expressions, and uses the Adam optimizer and cross entropy loss function to continuously fine-tune the first model.

[0028] Furthermore, in step 32, the large model analyzes the rationality of the reasoning path, determines the logical connection between the inference and the intermediate assumptions in each reasoning step, and the logical loopholes or unreasonable assumptions in the reasoning.

[0029] Furthermore, the method further includes using a self-game method to evolve the second model, including:

[0030] Step 41, inputting domain data into the second model, and generating two types of reasoning paths and answer pairs through a greedy search strategy and a beam search strategy;

[0031] Step 42, the second model is trained using the reasoning paths and answer pairs generated by the greedy search and the beam search, so that the results generated by the second model are close to the beam search strategy and away from the greedy search strategy.

[0032] Furthermore, high FLOPs paths are assigned positive scores, and low FLOPs paths are assigned negative scores.

[0033] The vertical field-oriented model evolution learning method proposed in the present invention has the following beneficial effects:

[0034] (1) Model domain capability improvement: Through domain knowledge distillation, GPT4 guides model evolution and model self-game to improve the model's capabilities in vertical fields;

[0035] (2) High-quality knowledge distillation: Using the idea of ​​weak models guiding strong models, higher-quality domain data is distilled from GPT4 to provide models for domain learning;

[0036] (3) Breakthrough in model self-capability: Utilize the method of model self-game to get rid of the dependence on GPT4 and achieve a breakthrough in the model’s self-capability. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 A schematic diagram of a flow chart of a model evolution learning method according to an embodiment of the present invention;

[0039] Figure 2 A schematic diagram of a distillation method in which a weak model guides a strong model according to an embodiment of the present invention;

[0040] Figure 3 A schematic diagram of the process of GPT4 guided model evolution according to an embodiment of the present invention;

[0041] Figure 4 The figure is a flow chart of the model self-game according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be checked and described in full in combination with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] This paper proposes a model evolution learning method for vertical fields, aiming to propose a comprehensive solution for model evolution in vertical fields, so that the model can fully understand the domain-specific problems existing in vertical fields and provide targeted solutions. The generalization of the domain capabilities of the general model in vertical fields demonstrates the practicality and effectiveness of the model in vertical fields. Figure 1 As shown in the figure, the model evolution learning method for vertical fields, such as Figure 1 As shown, including:

[0044] Step 1: Obtain high-quality domain data;

[0045] Step 2: Train the model to be a first model with preliminary domain knowledge through instruction fine-tuning method;

[0046] Step 3: Use the large model to guide the first model to evolve into the second model.

[0047] In step 1, the present invention uses the model evolution method to enable the model (i.e. the model to be evolved) to have the ability to answer questions in a vertical field. First, the model must initially have domain common sense. Therefore, this method needs to collect high-quality domain data, including high-diversity, high-quality domain questions and high-quality answers to domain questions.

[0048] Step 11: Domain knowledge question crawling

[0049] Taking the vertical field of computer science as an example, we crawled high-quality domain questions from the StackOverFlow official website (Stack Overflow is a well-known software developer forum operated by StackExchange, where users can ask and answer various programming questions). This includes a large amount of data related to computer science, and each piece of data also has multiple manual solutions, covering various categories in the field of computer science. Therefore, it is suitable for use as training data in this field.

[0050] Vertical field data usually consists of four parts: 1. Field issues. 2. Detailed description of the field issues. 3. User answers. 4. Number of likes for user answers.

[0051] After crawling, the domain data needs to be preprocessed. First, the higher the diversity of the domain data, the stronger the generalization of the trained model, and the more the model can adapt to various scenario problems in the vertical field. In one embodiment, the diversity includes data length diversity and data category diversity. Then, denoising and other processing are performed.

[0052] (1) Determine the diversity of domain data by using data of different lengths. The following is a specific example. If the model to be evolved is Llama3-8B-Chat (or Qwen2-7B-Chat, but not limited to these two models), divide the problem data into four parts according to the maximum length of the model: length less than 128 tokens, length between 128 tokens and 256 tokens, length between 256 tokens and 512 tokens, and length greater than 512 tokens. For data of each length, an equal amount of data will eventually be left to ensure the diversity of domain data length.

[0053] (2) The domain model should be applicable to more scenarios in the domain, so the diversity of domain data is extremely important. Taking the computer vertical field as an example, in order to ensure the diversity of domain data, we divided the data into four categories: machine learning, deep learning, natural language processing, and computer vision according to the classification of user questions. This ensures the diversity of domain data while ensuring the length of the data.

[0054] (3) Data quality is crucial for model learning, especially when we need the model to learn domain knowledge from a small amount of high-quality data. Therefore, unlike the large-scale text data used for pre-training, domain data should try to remove all noise. Therefore, we use the web page parser Beautiful Soup to use its powerful parsing capabilities to extract domain knowledge from the title ( <h1> to< / h1> <h6>),paragraph( ),span( ),piece( ) and lists ( ) to extract text content. In order to ensure the purity of the extraction results, we post-judge the extracted results. If the data contains non-text elements such as HTML, CSS, JS, emoticons, images, videos, etc., the data will be deleted.

[0055] In addition, the website also contains a large number of meaningless domain questions. These questions and answers may not contain too much domain information. Using these data to train the model may not improve the model's domain capabilities. However, if the number of answers to the question is relatively high and there are highly praised answers, it means that this data is discussed more by everyone and is a more meaningful data content. Therefore, you can define the threshold of the number of answers to the question as 5 and the threshold of the highest number of likes for the question as 10. When the corresponding content of the data exceeds the threshold, the data is retained as high-quality domain data. Table 1 shows the total amount and category of data after crawling and processing from the StackOverFlow official website, taking the computer field as an example.

[0056] Table 1: Raw data classification and its size

[0057] category Machine Learning Deep Learning Natural Language Processing Computer Vision total scale 4605 2092 1881 1698 10276

[0058] Step 12: The model generates guidelines based on the prompt

[0059] Since there is a cognitive bias between the large model (GPT4 is used as an example later) and the model (i.e., the model to be evolved), directly inputting the crawled data in step 11 into GPT4 to distill GPT4's domain knowledge may not necessarily produce results that enable the model to learn efficiently. This method will adopt a method strategy of weak model guiding strong model generation (such as Figure 2 As shown in the figure, we first need to obtain the model's cognitive approach to domain problems and use this cognitive approach to guide GPT4 to generate data, which can alleviate the cognitive bias between GPT4 and the model to a certain extent.

[0060] In order to obtain the model's understanding of the problem in this field, this method encapsulates prompts to let the model output the steps it thinks should be followed to solve the problem in this field. These steps are called guide information. Our goal is to obtain the guide information and let the large model generate answers to the domain problems based on the guide information. Therefore, in this step, this method inputs the domain problems in the high-quality domain data generated in step 11 into the model through prompts, so that the model generates guide information. The prompt is: "You are an expert in the field of [a special domain category]. I have a question that needs to be answered: [crawled user questions]. You do not need to answer this question yourself, as I will use a more powerful model, GPT-4, to provide the answer. However, to ensure that GPT-4 can answer the question more accurately, please provide a detailed guideline to help it respond more precisely. The guideline is:”

[0061] For example: You are an expert in the field of Machine Learning. I have a question that needs to be answered: "Title: Best Practices for Handling Missing Values ​​in Machine Learning?" Body: I ​​have a dataset with missing values ​​in both numerical and categorical features. What are some recommended strategies for handling these missing values ​​before training a machine learning model using Python and scikit-learn? Should I treat numerical and categorical features differently? Any advice or example code would be helpful. Thanks!". You do not need to answer this question yourself, as I will use a more powerful model, GPT-4, to provide the answer. However, to ensure that GPT-4 can answer the question more accurately, please provide adetailed guideline to help it respond more precisely. The guideline is: "Let the weak model generate the corresponding guide information. If the generation result repeater problem occurs, let the model re-generate the guide information for the data until all data pairs generate corresponding guide information data.

[0062] At this point, the guidance information generated by the model is:

[0063] 1.Context Understanding: The question is about preprocessing data in Python using scikit - learn, specifically handling missing values in both numerical and categorical features.

[0064] 2.Strategies Overview.

[0065] 3.Code Snippets: Provide simple scikit - learn examples for imputation.

[0066] 4.Best Practices: Recommend validating the impact of imputation on model performance and testing different methods.

[0067] 5.Additional Tips: Mention that some models handle missing values natively (e.g., XGBoost).

[0068] Step 13: Knowledge Distillation in the Field of Large Models Based on Guide Information

[0069] Input the domain question into the big model (such as GPT4, GPT4 is used as an example below) and let the big model output the answer to the question. The domain model to be evolved and GPT4 may have cognitive bias problems. Use the following prompts to stitch together the guide information of each data generated in step 12 with the domain question (user question): "Below is a guideline provided by an expert in the field of [specific field], followed by a specific question that needs to be answered. Please use the provided guideline to generate a precise and accurate answer to the question.\n guideline: [Insert guideline here]\n Question: [specific question]", and then input it into the big model (such as GPT4). The big model will generate the corresponding answer for each data. Store the question and answer pairs as distilled data.

[0070] Use a large model (such as GPT4) to score the distilled data. By comparing the GPT Score of the distilled data with and without guideline information, the effectiveness of the weak model-guided strong model strategy in improving the quality of distilled data is demonstrated. Table 2 shows the comparison of the effects of knowledge distillation based on guideline information and knowledge distillation without guideline information.

[0071] Table 2: Comparison of the effects of knowledge distillation based on guide information and knowledge distillation not based on guide information

[0072]

[0073] Step 2: Use the instruction fine-tuning method to generate a preliminary evolved model, called the first model.

[0074] Based on the distilled data, the instruction fine-tuning method is adopted, and the Adam optimizer and the cross entropy loss function are used to train and fine-tune the model to generate a first model with the ability to answer questions in a vertical field. The specific training and fine-tuning methods are existing technologies and will not be repeated here. Enter the domain question of the first model, and the first model will generate the corresponding answer. GPT4 can be used to evaluate the accuracy score of the answer to judge the quality of the answer; then the average score of the answer is used as the score of the model to evaluate the quality of the model. In addition, in order to further prove the effectiveness of the weak model guiding the strong model strategy, the scores of the fine-tuned model with guide information distilled data and the fine-tuned model without guide information are compared. The results are shown in Table 3.

[0075] Table 3: Average scores of models fine-tuned on distilled data with and without guide information

[0076]

[0077] Table 3 shows the experimental results. The model fine-tuned with the distilled data with guide information has stronger performance, which further verifies the effectiveness of the strategy of weak model guiding strong model.

[0078] Step 3: The large model guides the evolution of the first model to generate the second model.

[0079] In order to further enhance the capability of the first model in the vertical field, the present invention designs a multi-round iterative method in which the large model guides the evolution of the first model, such as Figure 3 As shown, it includes four steps: first model answer generation, GPT4 providing feedback, multiple rounds of iterative correction, and model performance fine-tuning.

[0080] Step 31, first model answer generation: answer the domain question based on the first model, generate a reasoning path and an answer;

[0081] In this step, we input the domain questions (or user questions) in the distilled data into the first model and generate answers using the chain of thought reasoning method. Chain of thought reasoning is a method of gradually unfolding the reasoning process, which can help the model decompose complex problems into easy-to-understand sub-steps. This method can significantly improve the logical consistency of the model when solving problems and the coherence of the reasoning structure. Specifically, the first model first receives the problem and uses its own domain knowledge to generate a reasoning path. Each step of reasoning explicitly generates intermediate assumptions and inferences to avoid missing key details or logical jumps. Compared with the traditional direct answer generation method, the chain of thought reasoning method not only improves the accuracy of the answer, but also enhances the model's sensitivity and interpretability to reasoning problems. In addition, this mechanism prompts the model to continuously correct itself in the subsequent reflection process, and finally generates a complete structure and accurate content answer.

[0082] Step 32: GPT4 provides model feedback: GPT4 evaluates the reasoning path and answer and scores the answer;

[0083] GPT4 conducts a systematic evaluation of the output of the first model (including answers and reasoning paths). GPT4 analyzes the rationality of the reasoning path, examines the logical connection between the inference and the intermediate assumptions in each reasoning step, and clearly points out the logical loopholes or unreasonable assumptions in the reasoning; in addition, GPT4 also scores the final answer through preset accuracy indicators. The systematic evaluation given by GPT4 includes the evaluation of the current reasoning path, the modification of the current reasoning path, the evaluation of the answer, and the modification of the answer. This systematic evaluation has a dual role: on the one hand, it helps the model identify and correct weak links in reasoning; on the other hand, it provides clear guidance for subsequent iterative corrections. This model simulates the process of accepting expert advice in the human learning process, and promotes the model to continuously improve its logical reasoning and vertical field adaptability.

[0084] Step 33: If the large model determines that the answer does not meet expectations, a demonstrative answer is generated, and then S31 and S32 are executed.

[0085] The exemplary answer is provided by GPT4 based on existing knowledge and reasoning logic, and the exemplary answer is used as part of the prompt, and then steps S31-S32 are executed until GPT4 finally approves the model's answer, or the number of iterations reaches the preset upper limit, the iteration is completed and all the texts generated by the current first model and GPT4 are saved as domain data based on GPT4's multiple rounds of feedback accumulation, which is used for fine-tuning in step 34. In the process of executing step 31, the first model will re-modify the reasoning path generated in the previous round according to the exemplary answer.

[0086] Multiple rounds of iterations continuously expose the model's performance defects in different reasoning scenarios and guide it to conduct targeted learning and corrections, ensuring that its problem-solving ability is gradually improved. This process not only accelerates the model's learning curve, but also significantly enhances its performance in complex reasoning tasks, laying a solid foundation for future applications.

[0087] Step 34, fine-tune the performance of the first model to generate a second model: fine-tune the first model based on the domain data accumulated from multiple rounds of feedback from GPT4 to improve its performance;

[0088] The entire reasoning path, the answer, and the feedback from GPT4 are used as training data to fine-tune the model. In one embodiment, in order to ensure the validity of the data, the large model will strictly verify all prompt words and generated content, and convert them into logically coherent and fluent contextual expressions. This processing step not only optimizes the structure of the prompt words, but also ensures the high quality and availability of subsequent training data. After accumulating a number of samples (for example, 10), the Adam optimizer and the cross entropy loss function are used to continuously fine-tune the first model. The Adam optimizer, with its efficient gradient update mechanism, can improve the generalization ability and problem-solving accuracy of the model in a relatively short time. Through this fine-tuning process, the first model continuously improves its performance in solving domain problems on the basis of gradually absorbing and integrating knowledge, providing stable and efficient reasoning and problem-solving capabilities for subsequent applications.

[0089] Table 4 shows that after completing step 3, GPT4 was used to evaluate the accuracy, completeness, relevance, logic, and reliability of the answers generated by the models before and after evolution, and which answer was more in line with expectations. From the performance of each dimension, the evolved model has been significantly improved, indicating the effectiveness of this method.

[0090] Table 4: Comparison of GPT4-guided model evolution performance before and after

[0091]

[0092]

[0093] In one embodiment, in order to enable the second model to achieve diversification and self-optimization, the present invention further includes:

[0094] Step 4, using the self-game method based on different reasoning complexities to evolve the second model, including generating different reasoning paths and training based on contrastive learning, such as Figure 4 shown.

[0095] Step 41 , generating diversified inference paths based on different inference FLOPs (floating point operations per second).

[0096] In this step, the domain data is input into the second model, and two strategies, greedy search and beam search, are used to generate diverse reasoning paths to meet the needs of different reasoning complexities. Specifically, the greedy search strategy adopts the option with the highest probability at each step, which has high generation efficiency but may lack global optimality. It is suitable for processing simple tasks or tasks with obvious logic. This strategy uses low FLOPs computing resources and can quickly give approximate answers. It is an ideal choice for application under resource-constrained conditions. However, in order to cope with complex reasoning tasks and deep logical analysis, a beam search strategy can be used, whose reasoning path has high FLOPs consumption and is suitable for scenarios requiring deep reasoning. Beam search ensures a wider global search space by retaining multiple candidate solutions at each step (the beam width is set to 10), thereby improving the accuracy and rationality of the answer. This strategy retains multiple possible paths during the reasoning process and screens and optimizes them according to the principle of global optimality.

[0097] This step will generate two reasoning paths and answer pairs for the questions input to the model: one is the reasoning path and answer pair generated by greedy search, and the other is the reasoning path and answer pair generated by beam search.

[0098] Step 42, training based on contrastive learning.

[0099] After the diversified reasoning paths are generated, the second model uses contrastive learning to optimize its reasoning ability. Contrastive learning is a deep learning paradigm that aims to improve the model's ability to distinguish by comparing positive and negative samples. At this stage, the second model is trained using answers generated by both greedy search and beam search. According to the scaling law in the second model's reasoning process, the higher the FLOPs used in the reasoning process, the better its reasoning performance will be. Therefore, the answer generated by the beam search reasoning strategy will naturally be better than the greedy search. Therefore, during the self-evolution of the second model, we make the results generated by the second model close to the beam search strategy and away from the greedy search strategy to optimize its reasoning ability to a state with more global vision and logical consistency.

[0100] Specifically, the cross-entropy loss function is used to compare and score the paths generated by greedy search and beam search: high FLOPs paths are given positive scores due to their reasoning depth and potential accuracy, while low FLOPs paths are given negative scores due to their simpler reasoning process. The specific training is an existing technology and will not be described here. Through comparative learning, the model continuously conducts self-reinforcement learning in self-game, and gradually improves its performance in various reasoning scenarios. Through this self-game mechanism, the model can effectively extract efficiency advantages from low FLOPs paths, while learning complex reasoning logic from high FLOPs paths, forming the ability of self-evolution.

[0101] Table 5 shows that before and after the model self-evolution, GPT4 evaluated the accuracy, completeness, relevance, logic, and reliability of the answers, and selected which answer was more in line with expectations. From the performance of each dimension, the model after self-evolution has been significantly improved, indicating the effectiveness of the model's self-evolution.

[0102] Table 5: Comparison of GPT4-guided model evolution performance before and after

[0103]

[0104] Those skilled in the art will appreciate that the foregoing embodiments are merely intended to illustrate the technical solutions of the present invention rather than to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art will appreciate that the technical solutions described in the foregoing embodiments may be modified or some or all of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope defined by the claims of the present invention. < / h6>

Claims

1. A model evolution learning method for vertical fields, characterized in that: The generation method comprises: Step 1: Obtain high-quality domain data; Step 2: Train the model to be a first model with preliminary domain knowledge through instruction fine-tuning method; Step 3: Use the large model to guide the evolution of the first model; including: Step 31: First model answer generation: answer the domain question based on the first model, generate reasoning paths and answers; Step 32: The big model provides feedback: The big model gives a systematic evaluation of the reasoning path and answer; Step 33: When the large model determines that the answer of the first model does not meet expectations, it will generate a demonstrative answer, input the demonstrative answer, the answer of the previous model, and the score of the large model into the first model, and repeat steps 31-32 until the large model finally recognizes the answer of the first model or the number of iterations reaches the preset upper limit; Step 34, fine-tune the performance of the first model to generate a second model.

2. The model evolution learning method according to claim 1, characterized in that: Step 1 includes: Step 11: Crawl domain data, which includes domain questions, specific descriptions of the domain questions, user answers, and the number of likes for user answers; Step 12: Construct prompts for domain data and generate guidance information for domain data through the model; Step 13: The guidance information and domain questions of each domain data are spliced ​​together through prompts and input into the big model. The big model will generate corresponding answers for each data and store the domain question and answer pairs as distilled data.

3. The model evolution learning method according to claim 2, characterized in that: In step 11, it also includes: Ensure the diversity of domain data; diversity includes: diverse length of domain data and diverse categories of domain data; Denoising; If the number of answers and likes to a domain question exceeds the threshold, the data is retained as high-quality domain data.

4. The model evolution learning method according to claim 1, characterized in that: In step 31, the domain questions in the distilled data are input into the first model, and the solution is generated by using the thought chain reasoning method.

5. The model evolution learning method according to claim 1, characterized in that: In step 32, the systematic evaluation includes evaluation of the current reasoning path, modification of the current reasoning path, evaluation of the answer, and modification of the answer.

6. The model evolution learning method according to claim 1, characterized in that: In step 33 , all texts generated by the first model and the large model are saved as domain data accumulated based on multiple rounds of feedback from the large model for fine-tuning in step 34 .

7. The model evolution learning method according to claim 1, characterized in that: In step 34, the large model strictly verifies all prompt words and generated content, and converts them into logically coherent and fluent contextual expressions, and uses the Adam optimizer and cross entropy loss function to continuously fine-tune the first model.

8. The model evolution learning method according to claim 1, characterized in that: Step 32, the large model analyzes the rationality of the reasoning path, determines the logical connection between the inference and the intermediate assumptions in each reasoning step, and the logical loopholes or unreasonable assumptions in the reasoning.

9. The model evolution learning method according to claim 1, characterized in that: It also includes using a self-game method to evolve the second model, including: Step 41, inputting domain data into the second model, and generating two types of reasoning paths and answer pairs through a greedy search strategy and a beam search strategy; Step 42, the second model is trained using the reasoning paths and answer pairs generated by the greedy search and the beam search, so that the results generated by the second model are close to the beam search strategy and away from the greedy search strategy.

10. The model evolution learning method according to claim 9, characterized in that: In step 42, a high FLOPs path is assigned a positive score, and a low FLOPs path is assigned a negative score.

Images