A method, device and storage medium for generating fine-tuning data of intelligent agent

Abstract

The present application relates to a method for generating fine-tuning data for an intelligent agent, which comprises obtaining a first knowledge base; inputting the first knowledge base into a first large language model in combination with prompt information to generate questions related to the first knowledge base; inputting the questions into a second large language model to generate answers; determining whether the answers are answers of a tool call type; if so, calling the corresponding tool execution component to perform tool execution on the answers; determining whether the tool execution is successful; inputting the answers and historical conversation record information into a large language model that has been fine-tuned in advance to obtain a confirmatory analysis result; when the confirmatory analysis result indicates that the problem has been solved, the complete historical conversation record information is used as valid data and entered into the intelligent agent fine-tuning data set. The present application solves the problem of a short reasoning link in the method of automatically generating fine-tuning data based on a large language model by introducing a strong and weak model self-interaction mechanism.

Description

Technical Field

[0001] The present application belongs to the field of large language model interaction technology, and specifically relates to a method, device and storage medium for generating fine-tuning data for an intelligent agent. Background Art

[0002] With the rapid development of artificial intelligence and big data technologies, the new generation of artificial intelligence technology represented by large language models has brought revolutionary breakthroughs. Large language models have shown excellent performance in various applications.

[0003] However, the training of large language models requires a large amount of high-quality training data, especially supervised fine-tuning data, which often needs to rely on manual labeling and is very costly. The intelligent agent (Agent) fine-tuning task was originally designed to improve the ability of large models to solve complex problems. The fine-tuning data labeling is more difficult and more costly.

[0004] Therefore, how to automatically construct high-quality fine-tuning data for intelligent agents is a research direction of great significance. Summary of the invention

[0005] In view of the above analysis, the embodiments of the present invention aim to provide a method, device and storage medium for generating fine-tuning data for an intelligent agent, so as to solve the problem of short reasoning link in the prior art method of automatically generating fine-tuning data with a large language model, thereby improving the overall length and quality of automatically constructed fine-tuning data.

[0006] In a first aspect of the present application, a method for generating fine-tuning data for an intelligent agent is provided, comprising:

[0007] S1: obtain the first knowledge base;

[0008] S2: inputting the first knowledge base combined with the prompt information into a first large language model to generate questions related to the first knowledge base;

[0009] S3: inputting the question into the second largest language model to generate an answer; wherein the parameter magnitude of the first largest language model is greater than the parameter magnitude of the second largest language model;

[0010] S4: Determine whether the answer is a tool call type answer;

[0011] If not, go to S5;

[0012] If yes, calling the corresponding tool execution component to perform tool execution on the answer;

[0013] Determine whether the tool is executed successfully:

[0014] If the tool is successfully executed, the tool execution result information and the historical session record information are input into the second largest language model, and the question and answer are generated again, and the process returns to execute S4;

[0015] If the tool fails to execute, the error message and the historical session record information are input into the first language model, and the answer is generated again, and the process returns to execute S4;

[0016] S5: Input the answer and the historical conversation record information into a pre-fine-tuned large language model to obtain a confirmatory analysis result, wherein the confirmatory analysis result is used to determine the resolution status of the problem mentioned in the historical conversation record information;

[0017] S6: When the confirmatory analysis result indicates that the problem is solved, the complete historical session record information is entered into the intelligent agent fine-tuning dataset as valid data.

[0018] Optionally, after S5, the following steps are further included:

[0019] S7: When the confirmatory analysis result indicates that the problem is not solved, the confirmatory analysis result and the historical conversation record information are input into the first language model, an answer is generated again, and the process returns to S4.

[0020] Optionally, after the confirmatory analysis result indicates that the problem is unresolved, the method further includes:

[0021] Count the number of times S4 is returned to execute;

[0022] It is determined whether the number of times of returning to execute S4 exceeds a first threshold; if yes, the corresponding data is discarded or marked as requiring manual processing.

[0023] Optionally, after determining whether the tool is successfully executed, the following steps are further included:

[0024] After determining that the tool execution fails, the cumulative number of execution failures is counted;

[0025] Determine whether the accumulated number of execution failures exceeds a second threshold; if so, discard the corresponding data or mark it as requiring manual processing.

[0026] Optionally, the first threshold is 30 and the second threshold is 3.

[0027] Optionally, the fine-tuning training process of the pre-fine-tuned large language model includes:

[0028] Creating a confirmatory analysis alignment training data set, wherein the input of the confirmatory analysis alignment training data set is a conversation record, and the output is the problems mentioned in the conversation record and the resolution status of the problems;

[0029] Fine-tuning training is performed on the confirmatory analysis alignment training data set based on the large language model to obtain a large language model that has been pre-fine-tuned.

[0030] Optionally, the fine-tuning training based on the large language model on the confirmatory analysis alignment training data set to obtain the large language model that has been pre-fine-tuned includes:

[0031] Using any one of the LoRA fine-tuning method, the full parameter fine-tuning method, and the Prefix Tuning fine-tuning method, fine-tuning training is performed on the confirmatory analysis alignment training data set based on the large language model to obtain a large language model that has been pre-fine-tuned.

[0032] Optionally, the first large language model is GPT-4 or GLM-4, and the second large language model is Qwen2.5-7B or Qwen2.5-3B.

[0033] According to a second aspect of the present application, there is provided a device for generating fine-tuning data for an intelligent agent, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, a method for generating fine-tuning data for an intelligent agent is implemented according to any one of the above-described methods.

[0034] According to a third aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for generating fine-tuning data of an intelligent agent according to any one of the above-mentioned methods is implemented.

[0035] The method for generating fine-tuning data for intelligent agents provided in the present application first fine-tunes the large language model for the confirmatory analysis task, so that the large language model can predict the confirmatory analysis results based on the given multi-round interaction data, that is, evaluate whether the problems involved in the multi-round interaction have been solved. Then, two large language models, strong and weak, with different parameter magnitudes interact with each other for a certain problem, so that the large language model automatically generates complex long reasoning link intelligent agent fine-tuning data. During the generation process, the interaction between the strong and weak models enriches the diversity of the long reasoning links, and simulates some unexpected situations in the actual dialogue process to a certain extent. At the same time, data quality feedback is performed based on the confirmatory analysis results, and data with solved problems involved in the multi-round interaction are selected, thereby guiding the long link reasoning data generated by the large language model to have better quality. It can be seen that the present application solves the problem of short reasoning links in the method of automatically generating fine-tuning data based on large language models by introducing a strong and weak model self-interaction mechanism, thereby improving the overall length and quality of automatically constructed fine-tuning data.

[0036] In addition, the present application also provides a device and storage medium for generating intelligent agent fine-tuning data having the above-mentioned technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this specification. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0038] Figure 1 A flowchart of a specific implementation of the method for generating fine-tuning data for an intelligent agent provided in this application;

[0039] Figure 2 Schematic diagram of the fine-tuning training process of a large language model;

[0040] Figure 3 This is a structural block diagram of the device for generating the intelligent agent fine-tuning data provided in this application. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. It should be noted that, in the absence of conflict, the embodiments in the present disclosure and the features in the embodiments can be combined, separated, interchanged and / or rearranged with each other. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0042] The terms used here are for the purpose of describing specific embodiments, and are not intended to be restrictive. As used here, unless the context clearly indicates otherwise, the singular forms "one (kind, person)" and "said (the)" are also intended to include plural forms. In addition, when the terms "comprise" and / or "include" and their variations are used in this specification, it is explained that there are stated features, integral bodies, steps, operations, parts, assemblies and / or their groups, but it is not excluded that there are or add one or more other features, integral bodies, steps, operations, parts, assemblies and / or their groups. It should also be noted that, as used here, the terms "substantially", "approximately" and other similar terms are used as approximate terms and not as degree terms, so that they are used to explain the inherent deviations of the measured values, calculated values ​​and / or the values ​​provided that will be recognized by those of ordinary skill in the art.

[0043] Existing methods for automatically generating fine-tuning data based on large language models generally generate short data reasoning links, usually single-round knowledge questions and answers or fewer rounds of small talk, while intelligent agent fine-tuning tasks usually target more complex problem scenarios, often requiring more rounds of interaction or frequent use of external tools. Therefore, existing solutions often work well in common single-round knowledge questions and answers or small talk scenarios, but their effects are limited in complex long reasoning link multi-round interaction scenarios. Based on this, the present application provides a method for generating intelligent agent fine-tuning data to overcome the above-mentioned limitations of the prior art.

[0044] A flowchart of a specific implementation of the method for generating fine-tuning data for an intelligent agent provided in this application is as follows: Figure 1 As shown, the method specifically includes:

[0045] S1: Obtain the first knowledge base.

[0046] Specifically, a first knowledge base can be obtained from a data source (such as a database, document, API, etc.), denoted as K. This can be a knowledge set in a specific field, covering information related to the task, as the basis for generating questions.

[0047] S2: Inputting the first knowledge base into a first large language model in combination with prompt information to generate questions related to the first knowledge base.

[0048] The first knowledge base K is combined with the prompt word and input into the first language model to generate a question related to the first knowledge base, denoted as Q. The first language model can select a mainstream commercial powerful language model in the industry, such as GPT-4, GLM-4, etc. After receiving the knowledge base K and the prompt word, the first language model generates a question Q related to the knowledge base. These questions are usually based on reasoning or task generation of known information, which may involve extracting relevant answers from the knowledge base, reasoning, or constructing complex queries.

[0049] Powerful language models usually have strong reasoning and language generation capabilities, and can demonstrate higher intelligence in more complex tasks.

[0050] S3: Input the question into the second largest language model to generate an answer.

[0051] Input question Q into the second largest language model and generate an answer, recorded as A. The second largest language model selects the industry's mainstream open source lightweight weak language model, such as Qwen2.5-7B, Qwen2.5-3B, etc.

[0052] Among them, the parameter magnitude of the first largest language model is greater than the parameter magnitude of the second largest language model. That is, the first largest language model is a strong language model, and the second largest language model is a weak language model. A strong language model refers to a model that has been trained with large-scale data, can handle a wide range of tasks and exhibits strong adaptability, has strong reasoning ability and good task generalization ability. A weak language model refers to a model that performs poorly when handling complex tasks, has limited reasoning ability, and has relatively poor adaptability.

[0053] Understandably, a strong language model usually refers to a model that is pre-trained on a large-scale dataset and has strong generalization and task adaptability. This model is able to perform well in a variety of natural language processing (NLP) tasks, and even achieves good results when dealing with unseen tasks. A weak language model refers to a model with poor performance or generalization ability on a specific task. Compared to a strong language model, a weak language model has poor adaptability, reasoning ability, and processing ability on complex tasks, and may require more task-specific data to improve performance.

[0054] As a specific implementation, the first largest language model may be GPT-4 or GLM-4, and the second largest language model may be Qwen2.5-7B or Qwen2.5-3B.

[0055] S4: Determine whether the answer is a tool call type answer. If not, proceed to S5. If yes, call the corresponding tool execution component to perform tool execution on the answer. Determine whether the tool execution is successful: If the tool execution is successful, input the tool execution result information and historical session record information into the second language model, generate questions and answers again, and return to execute S4; if the tool execution fails, input the error message and historical session record information into the first language model, generate answers again, and return to execute S4.

[0056] Tool calls refer to the system calling external resources or performing specific operations to obtain information or perform processing when generating answers or completing tasks. External resources can be databases, API interfaces, web searches, real-time data sources, etc. For example, a weather query task may need to call a weather API to obtain real-time weather information. An online translation task may call a translation tool for language translation. A digital calculation task may need to call a calculation tool to obtain a calculation answer.

[0057] When the answer is not a tool call type, that is, it is an answer directly generated by a large language model, there is no need to rely on external tools or operations. The answer is completely generated based on the model's knowledge of pre-trained data.

[0058] Determine whether answer A is a tool call type, that is, whether the end reason returned by the model inference is "toolcalls". If not, proceed to the next step S5. If yes, input the tool call parameters in answer A into the tool execution component to perform the tool execution operation, wherein the tool execution component can be developed and implemented in languages ​​such as Python and Java.

[0059] As a specific implementation method, the following factors may be considered to determine whether answer A is a tool call type: Whether it is necessary to obtain external information in real time: for example, the answer needs to query through an API, retrieve information from a database, or obtain the latest data from the Internet. Whether it involves external operations or executions: for example, calling external computing tools, services, or controlling other devices to operate. Whether it involves complex reasoning: If the generation of the answer depends on an external reasoning process (such as search engine queries, complex data analysis, etc.), it may belong to the tool call type.

[0060] If the answer is a tool call type, the tool execution component is used to implement the actual tool call operation. The tool execution component usually calls external resources or interfaces, such as APIs, database queries, file system operations, etc. These tools can be: weather query APIs, mathematical calculation tools, data query tools, etc., depending on the requirements of the task. After the tool is called, it is necessary to determine whether the tool execution is successful, that is, whether the external tool returns valid data or operation results.

[0061] If the tool fails to execute, the error message and historical session record information are input into a powerful language model to generate answer A again. The powerful language model selects the mainstream commercial large model in the industry, such as GPT-4, GLM-4, etc. Return to execute S4.

[0062] In this step, after determining that the tool execution fails, the cumulative number of execution failures is counted; it is determined whether the cumulative number of execution failures exceeds the second threshold N; if so, the corresponding data is discarded or marked as requiring manual processing. Among them, the second threshold N is 3. That is, it is set to allow the tool to fail N times at most. If it exceeds N times, the data is discarded or marked as requiring manual processing.

[0063] If the tool is executed successfully, the tool execution result information and the historical session record information are input into the weak language model, answer A is generated again, and the process returns to S4. The weak language model selects the mainstream open source lightweight language model in the industry, such as Qwen2.5-7B, Qwen2.5-3B, etc.

[0064] S5: Input the answer and the historical conversation record information into a pre-fine-tuned large language model to obtain a confirmatory analysis result. The confirmatory analysis result is used to determine the solution status of the problem mentioned in the historical conversation record information.

[0065] Answer A and historical conversation record information are input into the large language model to obtain the confirmation analysis result, which is recorded as CR. The large language model is pre-fine-tuned and trained, and the model can be deployed on a cloud server. The pre-fine-tuned large language model can evaluate the effectiveness, accuracy, and completeness of the current answer. The confirmation analysis result is the result obtained by analyzing whether the generated answer solves the problem. Through this analysis, it can be determined whether the current answer solves the user's problem.

[0066] S6: When the confirmatory analysis result indicates that the problem is solved, the complete historical session record information is entered into the intelligent agent fine-tuning dataset as valid data.

[0067] S7: When the confirmatory analysis result indicates that the problem is not solved, the confirmatory analysis result and the historical conversation record information are input into the first language model, an answer is generated again, and the process returns to S4.

[0068] According to the confirmatory analysis result CR, confirm whether the problem has been solved. If not, input the confirmatory analysis result CR and the historical conversation record information into the powerful language model, generate the answer A again, and return to execute step S4. If the problem has been solved, the complete conversation record is used as valid data and entered into the long reasoning link dataset. This helps to collect high-quality fine-tuning data for subsequent training and optimization.

[0069] In step S7, the number of times of returning to execute S4 is counted; it is determined whether the number of times of returning to execute S4 exceeds a first threshold; if so, the corresponding data is discarded or marked as requiring manual processing. The first threshold M may be 30. If it exceeds M times, the data is discarded or marked as requiring manual processing.

[0070] The strong and weak large language model self-interaction mechanism introduced in the process of automatically generating fine-tuning data for intelligent agents based on large language models can be understood as a mechanism that uses two large language models with different ability levels (strong language models and weak large language models) to interact to improve or enhance the generation of model fine-tuning data. The strong language model and weak large language model here usually refer to two models that differ in reasoning ability, generation ability, accuracy or task adaptability.

[0071] The strong language model and weak language model of this application can complement each other. The weak language model can be used to handle some simple tasks or problems, while the strong language model focuses on tasks that require complex reasoning. In multi-round dialogue generation, the weak language model can first generate a preliminary answer, and the strong language model then performs complex reasoning or further development of the dialogue context. The content generated by the weak language model is relatively rough, and the strong language model can optimize or supplement the generated content based on its stronger reasoning ability. The strong language model may correct the generation results of the weak model to make it more accurate and in line with the expected task objectives. Through the self-interaction mechanism, the weak language model can learn from the strong language model how to perform more efficient reasoning, thereby forming a reinforcement learning process in the process of fine-tuning data generation.

[0072] The interaction between strong and weak large language models helps to generate diverse fine-tuning data. This diversity not only improves the richness of generated data, but also helps to improve the robustness of the model during fine-tuning. Since strong language models can generate high-quality complex data, while weak large language models can generate relatively simple but effective data, combining the results of the two can improve the breadth and depth of generated data.

[0073] Based on any of the above embodiments, Figure 2 As shown in the diagram of the fine-tuning training process of a large language model, the fine-tuning training process of a large language model that has been pre-fine-tuned specifically includes:

[0074] S21: Creating a confirmatory analysis alignment training data set, wherein the input of the confirmatory analysis alignment training data set is the conversation record, and the output is the problems mentioned in the conversation record and the resolution status of the problems.

[0075] The confirmatory analysis alignment training data set of the embodiment of the present invention uses the form of manual annotation, inputs a conversation record, and outputs the questions mentioned in the conversation record and the solution status. It is used to train the model on how to evaluate the solution status of the answer to ensure that the generated content is aligned with the expected goal.

[0076] The training data set format example of the embodiment of the present invention is as follows:

[0077] {

[0078] "prompt":"The following is a bar chart of national tourism travel data during the May Day holiday from 2018 to 2024, showing the trend of the number of tourists each year. You can click the following link to view the chart: [View bar chart](http: / / xxxxxx.com / xxxxxx_fig.png)",

[0079] "response":"[{"question":"Query the national tourism travel data for the May Day holiday every year from 2018 to 2024","status":"Solved"},{"question":"Draw the national tourism travel data for the May Day holiday every year from 2018 to 2024 into a bar chart to show the data trend","status":"Solved"}]",

[0080] "history":[

[0081] {

[0082] "user":"Hello",

[0083] "assistant":"Hello! Nice to meet you, is there anything I can help you with?"

[0084] },

[0085] {

[0086] "user":"Help me query the national tourism travel data for the May Day holiday every year from 2018 to 2024, and draw a bar chart to show the data trend.",

[0087] "assistant":"arguments="{\"from_year\":\"2018\",\"to_year\":\"2024\",\"type\":\"by_all\"}',name="get_tourist_data_by_year'"

[0088] },

[0089] {

[0090] "tool":"[100,100,200,200,300,400]",

[0091] "assistant":"import matplotlib.pyplot as plt\n\n#Data received fromthe API\nyears=["2018","2019","2020","2021","2022","2023","2024"]\ntourist_data=[100,100,200,200,300,400,500]#Assuming the data for 2023and 2024based on the trend\n\n#Creating a bar chart\nplt.figure(figsize=(10,6))\nplt.bar(years,tourist_data,color=\'skyblue\')\nplt.xlabel(\'Year\')\nplt.ylabel(\'Number of tourists\')\nplt.title(\'National tourism data trend during the May Day holiday from 2018 to 2024\')\nplt.grid(axis=\'y\')\n\n#Show the chart\nplt.show()"

[0092] },

[0093] {

[0094] "tool":"{\"type\":\"file\",\"file\":\"http: / / xxxxxx.com / xxxxxx_fig.png\"}",

[0095] }, ]

[0097] }

[0098] Among them, "prompt" represents the current round of model input, "response" represents the output confirmation analysis result, and "history" represents the historical session record information.

[0099] The above data shows the interaction records of a dialogue system in the process of processing queries, tool calls, and chart generation. Prompt represents the current user's input, that is, the user requires the system to query the national tourism travel data during the May Day holiday from 2018 to 2024, and display the data trend through a bar chart.

[0100] The specific content is: "The following is a bar chart of national tourism travel data during the May Day holiday from 2018 to 2024, showing the trend of the number of tourists each year. You can click the following link to view the chart: View bar chart." This shows that the user has obtained the query results and obtained the relevant chart link.

[0101] The response is the answer returned by the model, which summarizes the result of the request. In this case, the model returns a JSON array containing the query records. Here, "solved" means that the model has successfully processed and completed these two tasks: querying data and drawing a chart.

[0102] The history section shows the historical conversation records with the user, where each record represents an interaction between the user and the system.

[0103] User asked: "Hello"

[0104] The system responds: "Hello! Nice to meet you. Is there anything I can help you with?"

[0105] User question: "Please help me query the national tourism travel data for the May Day holiday every year from 2018 to 2024, and draw a bar chart to show the data trend." System tool call: The model constructs an API call request (arguments part) to obtain data from 2018 to 2024.

[0106] The tool returns data: [100,100,200,200,300,400] is the returned travel data. The system generates a bar chart code based on this data.

[0107] Chart generation: The system generates the chart and provides a download link for the chart (file part).

[0108] S22: Fine-tune the large language model on the confirmatory analysis alignment training data set to obtain a large language model that has been pre-fine-tuned.

[0109] The large language model uses existing models in the industry, such as ChatGLM, and uses the large language model to complete fine-tuning on the confirmatory analysis alignment training dataset created above, thereby enhancing the ability of the large language model to perform confirmatory analysis on questions in conversation records.

[0110] Among them, any one of the LoRA fine-tuning method, the full parameter fine-tuning method, and the Prefix Tuning fine-tuning method is adopted to perform fine-tuning training on the confirmatory analysis alignment training data set based on the large language model to obtain a large language model that has been pre-fine-tuned.

[0111] The method for generating fine-tuning data for intelligent agents provided in the present application first fine-tunes the large language model for the confirmatory analysis task, so that the large language model can predict the confirmatory analysis results based on the given multi-round interaction data, that is, evaluate whether the problems involved in the multi-round interaction have been solved. Then, based on the interaction between the two large language models, strong and weak, for a certain problem, the large language model automatically generates complex long reasoning link intelligent agent fine-tuning data. During the generation process, the interaction between the strong and weak models enriches the diversity of the long reasoning links, and simulates some unexpected situations in the actual dialogue process to a certain extent. At the same time, data quality feedback is performed based on the confirmatory analysis results, and data with solved problems involved in the multi-round interaction are selected, thereby guiding the long link reasoning data generated by the large language model to have better quality. It can be seen that the present application solves the problem of short reasoning links in the method of automatically generating fine-tuning data based on large language models by introducing a strong and weak model self-interaction mechanism, thereby improving the overall length and quality of automatically constructed fine-tuning data.

[0112] In addition, the present application also provides a device for generating intelligent agent fine-tuning data, such as Figure 3 As shown in the structural block diagram of the device for generating fine-tuning data for an intelligent agent provided in the present application, the device specifically includes a memory 31 and a processor 32, wherein the memory 31 stores a computer program, and when the computer program is executed by the processor 32, a method for generating fine-tuning data for an intelligent agent according to any one of the above-mentioned methods is implemented.

[0113] In addition, the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements any of the above-mentioned methods for generating intelligent agent fine-tuning data.

[0114] Computer readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0115] The professionals should also be further aware that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0116] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0117] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only the specific implementation method of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A method for generating fine-tuning data for an intelligent agent, characterized in that: include: S1: obtain the first knowledge base; S2: inputting the first knowledge base combined with the prompt information into a first large language model to generate questions related to the first knowledge base; S3: inputting the question into the second largest language model to generate an answer; wherein the parameter magnitude of the first largest language model is greater than the parameter magnitude of the second largest language model; S4: Determine whether the answer is a tool call type answer; If not, go to S5; If yes, calling the corresponding tool execution component to perform tool execution on the answer; Determine whether the tool is executed successfully: If the tool is successfully executed, the tool execution result information and the historical session record information are input into the second largest language model, and the question and answer are generated again, and the process returns to execute S4; If the tool fails to execute, the error message and the historical session record information are input into the first language model, and the answer is generated again, and the process returns to execute S4; S5: Input the answer and the historical conversation record information into a pre-fine-tuned large language model to obtain a confirmatory analysis result, wherein the confirmatory analysis result is used to determine the resolution status of the problem mentioned in the historical conversation record information; S6: When the confirmatory analysis result indicates that the problem is solved, the complete historical session record information is entered into the intelligent agent fine-tuning dataset as valid data.

2. The method for generating fine-tuning data of an intelligent agent according to claim 1, characterized in that: After S5, it also includes: S7: When the confirmatory analysis result indicates that the problem is not solved, the confirmatory analysis result and the historical conversation record information are input into the first language model, an answer is generated again, and the process returns to S4.

3. The method for generating fine-tuning data of an intelligent agent according to claim 2, characterized in that: After the confirmatory analysis result indicates that the resolution status of the problem is unresolved, the method further includes: Count the number of times S4 is returned to execute; It is determined whether the number of times of returning to execute S4 exceeds a first threshold; if yes, the corresponding data is discarded or marked as requiring manual processing.

4. The method for generating fine-tuning data of an intelligent agent according to claim 3, characterized in that: After judging whether the tool is executed successfully, the following steps are also included: After determining that the tool execution fails, the cumulative number of execution failures is counted; Determine whether the accumulated number of execution failures exceeds a second threshold; if so, discard the corresponding data or mark it as requiring manual processing.

5. The method for generating fine-tuning data of an intelligent agent according to claim 4, characterized in that: The first threshold is 30, and the second threshold is 3.

6. The method for generating fine-tuning data for an intelligent agent according to any one of claims 1 to 5, characterized in that: The fine-tuning training process of the pre-fine-tuned large language model includes: Creating a confirmatory analysis alignment training data set, wherein the input of the confirmatory analysis alignment training data set is a conversation record, and the output is the problems mentioned in the conversation record and the resolution status of the problems; Fine-tuning training is performed on the confirmatory analysis alignment training data set based on the large language model to obtain a large language model that has been pre-fine-tuned.

7. The method for generating fine-tuning data of an intelligent agent according to claim 6, characterized in that: The fine-tuning training is performed on the confirmatory analysis alignment training data set based on the large language model to obtain the large language model that has been pre-fine-tuned, including: Using any one of the LoRA fine-tuning method, the full parameter fine-tuning method, and the Prefix Tuning fine-tuning method, fine-tuning training is performed on the confirmatory analysis alignment training data set based on the large language model to obtain a large language model that has been pre-fine-tuned.

8. The method for generating fine-tuning data of an intelligent agent according to claim 6, characterized in that: The first largest language model is GPT-4 or GLM-4, and the second largest language model is Qwen2.5-7B or Qwen2.5-3B.

9. A device for generating fine-tuning data for an intelligent agent, characterized in that: It includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, it implements the method for generating intelligent agent fine-tuning data according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the method for generating intelligent agent fine-tuning data according to any one of claims 1-8 is implemented.

Images