Method and system for reducing influence of large language model fine tuning on generalization ability in combination with system prompt

Through system propt and data mixing technology, the problem of forgetting after fine-tuning of large language models is alleviated, and common capabilities are retained in professional applications are realized, catastrophic forgetting problems are solved, and the application scalability of the model is improved.

CN119067236APending Publication Date: 2024-12-03HARBIN INST OF TECH

Patent Information

Application Number
CN202411165771.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-23
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Large language models are prone to catastrophic forgetting after fine-tuning, forgetting the knowledge they have learned before, limiting their universality and scalability in practical applications.

Method used

By designing the system prompt and constructing training data templates, mixing own data and open source data, adjusting the data ratio, fine-tuning the model with full parameter instructions, providing inference interfaces in different fields, alleviating catastrophic forgetting problems, and retaining the original capabilities of the model.

Benefits of technology

While meeting the application of professional fields, the general ability and pre-training knowledge of the model are retained, effectively solving the problem of forgetting large models after fine-tuning, and improving the maintenance of the model's ability in multiple rounds of dialogue and document generation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067236A_ABST
    Figure CN119067236A_ABST
Patent Text Reader

Abstract

The invention discloses a method and a system for reducing influence of fine tuning of a large language model on generalization ability in combination with system prompt, belongs to the technical field of natural language processing, and solves the problem of disastrous forgetting after fine tuning of the large model. The method comprises the following steps: S1, constructing a training data template according to a professional field problem needing to be solved; s2, obtaining a plurality of training data according to the training data template; s3, the training data and the open source data of the model to be finely tuned are mixed, the mixing proportion of the training data and the open source data is adjusted, and the optimal mixing proportion is obtained; and S4, according to the optimal mixing proportion, carrying out fine tuning on the model to be subjected to fine tuning to obtain a final large language model. The method provided by the invention can be applied to the situations that the problems in some professional fields need to be solved and the normal question-answering ability of a large model is expected to be met.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to a method for reducing the influence of fine-tuning of a large language model on generalization ability. Background Art

[0002] ChatGPT quickly became popular, triggering an era of change in big models. One of the problems faced by big models is the specialization of big models in some professional fields, and many existing general big models cannot meet the processing needs of some professional field problems. In addition, because the number of parameters of big models is very large and the training cost is very high, when the effect of prompt engineering cannot meet the requirements, enterprises or individuals have better own data and can use their own data to better improve the ability of big models in specific fields. Fine-tuning big models has become a very suitable choice.

[0003] Fine-tuning refers to the process of adjusting the parameters of a large language model to adapt it to a specific task. This is done by training the model on a dataset relevant to the task. The amount of fine-tuning required depends on the complexity of the task and the size of the dataset. However, fine-tuning of large models suffers from the problem of catastrophic forgetting, that is, after fine-tuning a new task, the large model may forget what it has learned previously. This phenomenon may limit the versatility and scalability of fine-tuned models in practical applications. Summary of the invention

[0004] The present invention proposes a method and system for reducing the influence of fine-tuning of a large language model on generalization ability in combination with a system prompt, with the aim of solving the problem of catastrophic forgetting after fine-tuning of a large model.

[0005] The present invention proposes a method for reducing the influence of fine-tuning of a large language model on generalization ability in combination with a system prompt, including:

[0006] S1: Construct training data templates based on the professional field problems that need to be solved;

[0007] S2: obtaining a number of training data according to the training data template;

[0008] S3: Mix the training data with the open source data of the model to be fine-tuned, and adjust the mixing ratio of the training data and the open source data to obtain the optimal mixing ratio;

[0009] S4: Fine-tune the model to be fine-tuned according to the optimal mixing ratio to obtain a final large language model.

[0010] Furthermore, a preferred solution is provided: S1 comprises:

[0011] S1.1: Design system prompts based on the professional problems that need to be solved;

[0012] S1.2: Construct a training data template according to the designed system prompt.

[0013] Furthermore, a preferred solution is provided: S1.1 includes:

[0014] S1.1.1: Extract keywords of professional problems that need to be solved, and match the keywords with formulas in the formula library;

[0015] According to the matched formula, query the specific value, bring the specific value into the formula for calculation, and obtain the calculation result.

[0016] Furthermore, a preferred solution is provided: S2 includes:

[0017] Use the existing large model, construct prompt words, and generate a number of training data in batches.

[0018] Furthermore, a preferred solution is provided: the existing large models include: GPT4 and GLM4.

[0019] Furthermore, a preferred solution is provided: S3 includes:

[0020] S3.1: Obtain open source data of the model to be fine-tuned, and combine the open source data and the plurality of training data to obtain a training data set;

[0021] S3.2: Screening the training data set to select valid training data;

[0022] S3.3: Mix the valid training data with the open source data of the model to be fine-tuned to obtain an optimal mixing ratio.

[0023] Furthermore, a preferred solution is provided: in S4, the model to be fine-tuned is fine-tuned using full parameter instruction fine-tuning.

[0024] The present invention also proposes a system for reducing the influence of fine-tuning of a large language model on generalization ability in combination with a system prompt, the system comprising:

[0025] Template construction module: used to construct training data templates according to the professional field problems that need to be solved;

[0026] Data generation module: used to obtain a number of training data according to the training data template;

[0027] Data mixing module: used to mix training data with the open source data of the model to be fine-tuned, and adjust the mixing ratio of training data and open source data to obtain the optimal mixing ratio;

[0028] Model fine-tuning module: used to fine-tune the model to be fine-tuned according to the optimal mixing ratio to obtain the final large language model.

[0029] The present invention also proposes a computer device, including a memory and a processor, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes a method for reducing the impact of fine-tuning of a large language model on generalization ability in combination with a system prompt as described in any combination of the above-mentioned solutions.

[0030] The present invention also proposes a computer-readable storage medium, which is used to store a computer program, and the computer program executes the steps of a method for reducing the impact of fine-tuning of a large language model on generalization ability in combination with a system prompt as described in any combination of the above-mentioned solutions.

[0031] Compared with the prior art, the advantages of the present invention are:

[0032] The present invention aims to solve the problem that a large model may forget previously learned knowledge after fine-tuning a new task. For the fine-tuning data in a vertical field, a corresponding system prompt is designed. During fine-tuning, the system prompt is fine-tuned together with the data in the vertical field. The method described in the present invention provides different reasoning interfaces for the large model to complete answers in different fields. This method is used to alleviate the catastrophic forgetting problem after fine-tuning the large model and retain the original capabilities of the large model.

[0033] The method described in the present invention can satisfy the application of large models in professional fields while retaining the original general capabilities and pre-trained knowledge, such as document generation, multi-round dialogues, etc.

[0034] The method described in the present invention can be applied in situations where there is a need to solve some professional problems while also wanting to satisfy the normal question-answering capabilities of the large model. For example, this method can be used to solve the professional problems in the financial field of the large model while also retaining the large model's capabilities in document generation, multi-round dialogue, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0036] Figure 1This is a flowchart of a method for reducing the impact of fine-tuning of a large language model on generalization ability in combination with a system prompt as described in a specific implementation mode of the present invention. DETAILED DESCRIPTION

[0037] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0038] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0039] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification and the appended claims, the singular forms "a", "an" and "the" are intended to include plural forms unless the context clearly indicates otherwise.

[0040] The following is a clear and complete description of the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0041] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present application. Therefore, the present application is not limited to the specific implementation methods disclosed below.

[0042] Implementation method 1:

[0043] Reference Figure 1 This embodiment will be described.

[0044] A method for reducing the influence of fine-tuning of a large language model on generalization ability by combining system prompt in this embodiment includes:

[0045] S1: Construct training data templates based on the professional field problems that need to be solved;

[0046] S1.1: Design system prompts based on the professional field problems that need to be solved; for example: "The user is a financial question answerer. The user's task is to answer the question according to the following ideas. The answer idea is: first extract keywords and match the extracted keywords with formulas. Then query the specific value according to the matched formula. Finally, substitute the obtained value into the formula and calculate.\nWhen answering questions, you need to refer to the following format\nThought: answer content, Action: name of the calling tool, Observation: the return result of calling the tool.\n###The questions that need to be answered are: <human>:{input}\n <bot>:".

[0047] S1.1.1: Extract keywords of professional problems to be solved, and perform formula matching on the keywords in the formula library, wherein the formula matching is implemented through character strings; specifically, the formula library is designed according to indicators in a specific field, for example, in the financial field, a financial formula library is designed according to indicators in the three major financial statements, and the formula library contains the calculation formula for each indicator;

[0048] S1.1.2: Query the specific value according to the matched formula, and put the specific value into the formula to calculate and obtain the calculation result;

[0049] S1.2: Construct a training data template based on the designed system prompt; specifically, replace {input} in the system prompt with the specific content of the user's question, such as replacing {input} with: "How much is the monetary funds in 2017?" This step is based on the system prompt designed in S1.1, and combines it with specific questions and answers to construct a complete and unified training data template.

[0050] Such as:

[0051] {"instruction":

[0052] "output":

[0053] }

[0054] S2: obtaining a number of training data according to the training data template, wherein the number of the obtained training data is not less than 2k;

[0055] S2.1: Based on the training data template designed in S1, artificially generate a small amount of high-quality training data samples that meet the requirements;

[0056] S2.2: Write a script or use existing high-performance large models, such as GPT4, GLM4, etc., to construct appropriate prompt words, and batch generate the required amount of training data. The prompt words are constructed according to the content of professional field problems, and the task scenario and task content are obtained according to the professional field problems. The prompt words are: "The scenario involved is {task scenario}, and the document outline to be written is {task content}. Please complete the expansion according to the above content", and replace the content in {} with the obtained task scenario and task content. For example: Taking law as an example, the prompt used to construct data is as follows:

[0057] You are a person engaged in finance-related work. You need to complete many documents in your work. Please complete the corresponding documents based on some information. \nThe word count is about 500 words. "

[0058] "The scenario involved is {task scenario}, and the document outline to be written is {task content}. Please complete the expansion based on the above content.

[0059] Use the above method to construct some data and mix it with the data in S1 to prepare for S3.

[0060] S2.3: Screen appropriate high-quality training data;

[0061] S3: Mix the training data with the open source dataset of the model to be fine-tuned, and adjust the mixing ratio of the training data and the original data to obtain the optimal mixing ratio;

[0062] S3.1: Obtain the original instructions of the model to be fine-tuned (which can be obtained from open source projects on GitHub, etc., i.e., open source datasets), and combine the open source datasets and the plurality of training data to obtain a training dataset;

[0063] S3.2: Screen the training data set to select valid training data, ensure that the quantity and type of training data are appropriate, and maintain the diversity of training data; the evaluation method of the valid data is: for the several training data obtained in S2, after tf-idf vectorization, use cosine similarity to remove duplicates, and if the cosine similarity is greater than 0.8, it is considered that the duplicates need to be removed, and the data finally screened out is the valid training data.

[0064] S3.3: Mix the effective training data with the original data of the model to be fine-tuned to obtain the optimal mixing ratio; the optimal mixing ratio is verified by experiments, the mixing ratio is pre-set, the number of training data constructed in S2 is about 2k, and the ratio of the training data constructed in S2 to the effective training data is 1:1 or 1:1.5 for trial. The method for determining the optimal ratio is to test on an open source large model general capability test dataset, such as CMMUL.

[0065] S4: Fine-tune the model to be fine-tuned according to the optimal mixing ratio to obtain a final large language model.

[0066] Specifically:

[0067] S1 described in this embodiment constructs a suitable training data template according to the specific professional field problem to be solved.

[0068] According to different problem requirements, it is necessary to design appropriate prompt word templates to combine the user's questions. In the field of artificial intelligence, especially in the application of large language models (LLMs), system prompt is a common technical means for guiding the behavior and output of the model. System prompts are usually designed as part of the model input, and their purpose is to influence the model's understanding and way of generating text. In this embodiment, the system prompt is used as a prefix of the user input. This means that when the user asks a question or request to the model, the system prompt will appear before the user's actual input. For example, the system prompt may be a text that clearly instructs the model to answer questions in a specific role or style. The advantage of this method is that it is direct and clear, because it ensures that the model has been guided before processing the user input.

[0069] The S2 batch-produces training data of sufficient quantity and quality according to the training data template determined by S1.

[0070] Almost all the knowledge in large language models is learned in pre-training, and only limited fine-tuning data is needed to teach the model to produce high-quality output. However, fine-tuning still requires a sufficient amount of training data, and the quality of training data is critical. When the amount of data is expanded without simultaneously expanding the diversity of prompts, the effect will be greatly reduced, while when optimizing the data quality, the benefits will be greatly increased.

[0071] The S3 searches for original fine-tuning data of the appropriate proportion of the hybrid model and fine-tunes the model.

[0072] Different proportions of training data have a great impact on model performance. If there is too much original training data, the model may not be able to learn how to handle special system prompts. Conversely, it may lead to catastrophic forgetting of the model. The ratio of training data to original training data can be adjusted multiple times to fine-tune the model and find the ratio that has the least impact on the performance of the original model while gaining the ability to handle problems in specific fields.

[0073] This implementation also constructs a dedicated reasoning interface for special problems.

[0074] This interface will add special format prompt words to the input of the big model. After inputting the fine-tuned big model, the big model will generate answers in a specific format. Write appropriate processing scripts to process these answers in a specific format and return the processing results to the big model to obtain the final output. If you are not asking special questions, the big model can perform normal question-answering without calling this interface, and the original performance loss is small.

[0075] Specifically:

[0076] First, use Python to write and select a suitable large model reasoning framework, such as vLLM;

[0077] Secondly, we add a system prompt to the user's question and some special tokens as input to the big model.

[0078] Finally, the large model reasoning content is processed, and methods such as external retrieval and database query can be used to meet the needs of problem solving, and the final results are returned to the user.

[0079] Implementation Method 2

[0080] This implementation is a further example of a method described in Implementation 1 that combines system prompts to reduce the impact of fine-tuning a large language model on generalization ability.

[0081] This implementation is the performance of the model in professional and general fields after fine-tuning in combination with the system prompt.

[0082] Question: What was the investment income from associated companies and joint ventures in 2018?

[0083] Answer: Thought: According to the question, the key information extracted is the investment income of associates and joint ventures, and the complete formula needs to be found. \nAction:find_fommula[Investment income of associates and joint ventures]\nObservation: Investment income of associates and joint ventures = Secondary account investment income_Investment income of associates and joint ventures\nThought: According to the formula information found, it is necessary to know the specific value in the formula in 2018. \nAction:get values[2018, [Secondary account investment income_Investment income of associates and joint ventures]]\nObservation:{Secondary account investment income_Investment income of associates and joint ventures: 0}\nThought: After obtaining the specific value, it is necessary to substitute it into the formula: Investment income of associates and joint ventures = Secondary account investment income_Investment income of associates and joint ventures, and then calculate it through the calculator. \nAction:calculate[0]\nObservation:0\nThought:According to the calculation results, the investment income from associates and joint ventures is 0, so the investment income from associates and joint ventures in 2018 is 0\nAction:Finish[The investment income from associates and joint ventures in 2018 is 0]

[0084] Question: Is there a minimum registered capital requirement when setting up a limited liability company? Is it necessary to pay the registered capital in full?

[0085] Answer: At present, my country's registered capital system has been changed from a paid-in system to a subscribed system, that is, when establishing a limited liability company, it is no longer necessary to actually pay the registered capital, but only to bear limited liability within the scope of the registered capital registered with the industrial and commercial authorities. However, it should be noted that according to relevant laws and regulations and the company's articles of association, the company's shareholders still need to pay the registered capital within the time limit.

[0086] It can be seen that the method is applied to solve the problem of large models in the financial field. While being able to effectively answer financial questions, it can also normally answer general questions. The large model can answer financial query questions based on the system prompt and the special reasoning interface. At the same time, it can also generate text generation instructions normally, effectively alleviating catastrophic forgetting.

[0087] Implementation method three:

[0088] This implementation is a further example of a method described in Implementation 1 that combines system prompts to reduce the impact of fine-tuning a large language model on generalization ability.

[0089] 1. The following are examples from the financial field (system prompt):

[0090] Instruction: You are a respondent to a financial calculation question. Your task is to answer the question according to the following ideas. The answer idea is: first extract keywords and match the extracted keywords with formulas. Then query the specific value based on the matched formula. Finally, substitute the obtained value into the formula and calculate. \nWhen answering questions, you need to refer to the following format\nThought: answer content, Action: name of the tool called, Observation: the return result of the tool called. \n###The question to be answered is: <human>:How much is the monetary capital in 2017? \n <bot>:

[0091] Output: Thought: According to the question, the key information extracted is monetary funds, and we need to find the complete formula. \nAction: find_formula [monetary funds]\nObservation: Monetary funds = cash on hand + deposits with the central bank in the first-level account + deposits with peers in the first-level account + funds between financial institutions in the first-level account + settlement reserves in the first-level account + deposits with margin in the first-level account + trading financial assets in the first-level account + financial assets purchased under resale agreements in the first-level account\nThought: According to the formula information found, we need to know the specific value in the formula in 2017. \nAction:get_values[2017,[Cash on hand, First-level account deposits with the central bank, First-level account deposits with peers, First-level account funds transfer between internal financial institutions, First-level account settlement reserves, First-level account deposit margin, First-level account trading financial assets, First-level account purchase and resale financial assets]]\nObservation:{Cash on hand: 0, First-level account deposits with the central bank: 0, First-level account deposits with peers: 0, First-level account funds transfer between internal financial institutions: 1742285.91, First-level account settlement reserves: 0, First-level account deposit margin: 0, First-level account trading financial assets: 0, First-level account purchase and resale financial assets: 0}\nThought:After getting the specific value, you need to substitute it into the formula: Monetary funds = cash on hand + First-level account deposits with the central bank + First-level account deposits with peers + First-level account funds transfer between internal financial institutions + First-level account settlement reserves + First-level account deposit margin + First-level account trading financial assets + First-level account purchase and resale financial assets, and then calculate it through the calculator. \nAction:calculate[0+0+0+1742285.91+0+0+0+0]\nObservation:1742285.91\nThought:According to the calculation results, the monetary funds are 1742285.91, so the monetary funds in 2017 are 1742285.91\nAction:Finish[The monetary funds in 2017 are 1742285.91]\n\n < / s。

[0092] 2. The following are examples of general use cases (without system prompts):

[0093] Instruction: When investing, how do you evaluate the risk-return ratio and develop a corresponding investment strategy?

[0094] Output: Assessing the risk-return ratio is a key step in investment decision-making. Here are some suggestions: 1. Understand investment products: Study the characteristics of different investment products, such as stocks, bonds, funds, etc., and understand their sources of income, risk level and liquidity. 2. Analyze the market environment: Pay attention to the macro-economy, industry policies and market trends, and evaluate the overall market risks and investment opportunities. 3. Evaluate investment goals: Choose suitable investment products based on your own risk tolerance, investment period and expected returns. 4. Calculate expected returns: Analyze the historical return data of investment products, combine market expectations, and estimate future returns. 5. Analyze risks: Analyze the potential risks of investment products, such as market fluctuations, credit risks, liquidity risks, etc. 6. Formulate investment strategies: According to the risk-return ratio, choose suitable investment products, diversify investments, and reduce overall risks. 7. Regularly adjust: Pay attention to the performance of the investment portfolio, and adjust the investment strategy in a timely manner according to market changes and your own needs. In the investment process, we must strictly abide by relevant laws and regulations, such as my country's "Securities Law" and "Fund Law", to ensure that investment behavior is legal and compliant. At the same time, investors should maintain a cautious investment mentality, do not blindly pursue high returns, follow the principle of risk-return balance, and achieve long-term and stable investment returns. \n

[0095] Table 1. CMMLU test results of the method combining system prompt to reduce the impact of large language model fine-tuning on generalization ability

[0096] CMMLU Science and Technology Humanities sociology other Local characteristics overall Baichuan2 0.429 0.578 0.577 0.573 0.547 0.545 1:1 0.523 0.523 0.560 0.549 0.542 0.539 1:1.5 0.393 0.540 0.548 0.539 0.518 0.512

[0097] In the table, Baichuan2 refers to the original model, and 1:1 and 1:1.5 refer to the mixed ratio of data with system prompt and data without system prompt.

[0098] Those skilled in the art will appreciate that the above are only preferred embodiments of the present invention, and the various embodiments of the present disclosure and / or the features described in the claims may be combined or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. It is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art may still modify the technical solutions described in the aforementioned embodiments, or perform equivalent substitutions on some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

[0099] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0100] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0101] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0102] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram and the combination of processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0103] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure rather than to limit its protection scope. Although the present disclosure has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that after reading the present disclosure, those skilled in the art can still make various changes, modifications or equivalent substitutions to the specific implementation methods of the invention, but these changes, modifications or equivalent substitutions are all within the protection scope of the disclosed claims to be approved.< / bot> < / human> < / bot> < / human>

Claims

1. A method for reducing the impact of fine-tuning a large language model on generalization ability in combination with a system prompt, characterized in that: The method comprises: S1: Construct training data templates based on the professional field problems that need to be solved; S2: obtaining a number of training data according to the training data template; S3: Mix the training data with the open source data of the model to be fine-tuned, and adjust the mixing ratio of the training data and the open source data to obtain the optimal mixing ratio; S4: Fine-tune the model to be fine-tuned according to the optimal mixing ratio to obtain a final large language model.

2. According to claim 1, a method for reducing the influence of fine-tuning of a large language model on generalization ability in combination with a system prompt, characterized in that: The S1 includes: S1.1: Design system prompts based on the professional problems that need to be solved; S1.2: Construct a training data template according to the designed system prompt.

3. According to claim 2, a method for reducing the influence of fine-tuning of a large language model on generalization ability in combination with a system prompt, characterized in that: The S1.1 includes: S1.1.1: Extract keywords of professional problems that need to be solved, and match the keywords with formulas in the formula library; S1.1.2: According to the matched formula, query the specific value, substitute the specific value into the formula to calculate and obtain the calculation result.

4. According to claim 1, a method for reducing the influence of fine-tuning of a large language model on generalization ability in combination with a system prompt, characterized in that: The S2 includes: Use the existing large model, construct prompt words, and generate a number of training data in batches.

5. According to claim 4, a method for reducing the influence of fine-tuning of a large language model on generalization ability in combination with a system prompt, characterized in that: The existing large models include: GPT4 and GLM4.

6. The method of reducing the influence of fine-tuning of a large language model on generalization ability in combination with a system prompt according to claim 1, characterized in that: The S3 includes: S3.1: Obtain open source data of the model to be fine-tuned, and combine the open source data and the plurality of training data to obtain a training data set; S3.2: Screening the training data set to select valid training data; S3.3: Mix the valid training data with the open source data of the model to be fine-tuned to obtain an optimal mixing ratio.

7. The method of reducing the influence of fine-tuning of a large language model on generalization ability in combination with a system prompt according to claim 1, characterized in that: In S4, the model to be fine-tuned is fine-tuned using full parameter instruction fine-tuning.

8. A system that combines system prompt to reduce the impact of fine-tuning of large language models on generalization ability, characterized in that: The system comprises: Template construction module: used to construct training data templates according to the professional field problems that need to be solved; Data generation module: used to obtain a number of training data according to the training data template; Data mixing module: used to mix training data with the open source data of the model to be fine-tuned, and adjust the mixing ratio of training data and open source data to obtain the optimal mixing ratio; Model fine-tuning module: used to fine-tune the model to be fine-tuned according to the optimal mixing ratio to obtain the final large language model.

9. Computer device, characterized in that The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the processor runs the computer program stored in the memory, the processor executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store a computer program, and the computer program executes the steps of any one of claims 1-7.

Citation Information

Patent Citations

  • Model training method, data processing method, equipment and storage medium

    CN117520842A

  • Vertical domain large language model training method and related equipment

    CN117828401A

  • A ship test training scenario question and answer method, system, device and medium

    CN118113855A

  • Modular reasoning, knowledge, and language systems

    WO2023200762A1

Cited By

  • Model generalization ability optimization method and system based on hybrid experts

    CN121581146A

  • A method and system for optimizing model generalization ability based on mixed experts

    CN121581146B