Target model training data generation method, model training method and related equipment

Abstract

The invention discloses a target model training data generation method, a model training method and related equipment, and the method comprises the steps: generating a plurality of data generation instructions based on a data generation task; the multiple data generation instructions are subjected to multi-dimensional screening, multiple target instructions are obtained, multi-dimensional screening comprises at least two of instruction quality screening, diversity screening and necessity screening, the diversity screening is used for reserving one of multiple similar instructions in the multiple data generation instructions, and the necessity screening is used for reserving one of multiple similar instructions in the multiple data generation instructions; the necessity screening is used for reserving the data generation instructions which cannot be correctly executed in the plurality of data generation instructions; the multiple target instructions are input into the large language model to be executed, and execution results corresponding to the target instructions are obtained; and based on the plurality of target instructions and the execution results, constructing training data of the target model. According to the scheme, the quality of the training data can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a target model training data generation method and a model training method, and related equipment. BACKGROUND

[0002] The core of the intelligent agent is to accurately analyze the user's intention and correctly call the tool by using the large language model (LLM). However, in actual application, the user's question is diverse (for example, containing business terminology, short expression, biased colloquialization, fuzzy query intention, etc.), the tool business is strong (the tool description contains a large number of business terminologies), and the large language model lacks business background knowledge and has weak logical reasoning ability (especially in the vertical field), which leads to weak tool usage ability of the large language model. Model training can fundamentally improve the tool usage ability of the model, and thus improve the overall performance of the intelligent agent. However, model training depends on intelligent agent training data, especially high-quality training data. Therefore, how to construct high-quality intelligent agent training data has become a problem to be solved to improve the performance of the intelligent agent. SUMMARY

[0003] The present application at least provides a target model training data generation method and a model training method, and related equipment, which can improve the quality of the training data.

[0004] The first aspect of the present application provides a target model training data generation method, which comprises: generating a plurality of data generation instructions based on a data generation task; performing multi-dimensional screening on the plurality of data generation instructions to obtain a plurality of target instructions, wherein the multi-dimensional screening comprises at least two of instruction quality screening, diversity screening and necessity screening, the diversity screening is used to retain one instruction in a plurality of similar instructions in the plurality of data generation instructions, and the necessity screening is used to retain a data generation instruction that cannot be executed correctly in the plurality of data generation instructions; inputting the plurality of target instructions into a large language model for execution to obtain an execution result corresponding to the target instruction; and constructing training data for the target model based on the plurality of target instructions and the execution result.

[0005] Among them, based on the data generation task, a plurality of data generation instructions are generated, which comprises: performing format conversion based on the data generation task to obtain an instruction generation prompt; inputting the instruction generation prompt into a large language model for instruction generation to obtain a plurality of data generation instructions.

[0006] The method comprises the following steps: selecting a task example as a target task from a seed pool, wherein the seed pool comprises a plurality of task examples; obtaining corresponding target tool information from a tool set based on the target task, wherein the target tool information is information of a tool required to be called during execution of the data generation task; generating the data generation task based on the target task and the target tool information; and inputting the instruction generation prompt into the large language model to generate a plurality of data generation instructions, and then updating the demand task in the seed pool based on the data generation instructions.

[0007] The method comprises the following steps: performing instruction quality screening on the data generation instructions to obtain first screened instructions; performing diversity screening on the first screened instructions to obtain second screened instructions; performing necessity screening on the first screened instructions by using the second screened instructions to obtain third screened instructions; and performing deduplication on the second screened instructions and the third screened instructions to obtain target instructions.

[0008] The method comprises the following steps: performing instruction quality screening on the data generation instructions to obtain first screened instructions; performing diversity screening on the first screened instructions to obtain second screened instructions; performing necessity screening on the first screened instructions by using the second screened instructions to obtain third screened instructions; and performing deduplication on the second screened instructions and the third screened instructions to obtain target instructions.

[0009] The method comprises the following steps: performing instruction quality screening on the data generation instructions to obtain first screened instructions; performing diversity screening on the first screened instructions to obtain second screened instructions; performing necessity screening on the first screened instructions by using the second screened instructions to obtain third screened instructions; and performing deduplication on the second screened instructions and the third screened instructions to obtain target instructions.

[0010] The execution generation prompt is input into the large language model to obtain an execution result, including: inputting the execution generation prompt into the large language model to perform and obtain an initial execution result; verifying the initial execution result to obtain a verification result; and in response to the verification result being correct, taking the initial execution result corresponding to the correct verification as a final execution result and outputting the final execution result.

[0011] The second aspect of the present application provides a target model training method, including: performing format conversion on sample instructions in training data to obtain sample execution generation prompts, wherein the training data is obtained by the target model training data generation method in the first aspect; inputting the sample execution generation prompts into the target model to perform and obtain sample execution results; and adjusting network parameters of the target model based on differences between the sample execution results and labeled sample results in the training data until the target model converges.

[0012] The third aspect of the present application provides an electronic device, including a memory and a processor coupled to each other, the processor being configured to execute program instructions stored in the memory to implement the target model training data generation method in the first aspect or the target model training method in the second aspect.

[0013] The fourth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, the program instructions being executed by a processor to implement the target model training data generation method in the first aspect or the target model training method in the second aspect.

[0014] The above scheme uses data generation tasks to generate a plurality of data generation instructions, performs multi-dimensional screening (such as instruction quality screening, diversity screening, and necessity screening) on the plurality of data generation instructions to obtain a plurality of high-quality target instructions. Then, the plurality of high-quality target instructions are input into the large language model, so that the large language model operates according to the target instructions to obtain execution results corresponding to the target instructions, which can improve the quality of the execution results generated by the large language model according to the plurality of high-quality target instructions. Finally, the plurality of target instructions and the execution results are combined to generate training data for the target model, thereby improving the quality of the training data.

[0015] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, but not limiting the present application. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present application and, together with the specification, serve to explain the technical solutions of the present application.

[0017] Figure 1is a flowchart of an embodiment of a method for generating training data for a target model according to the present application; Figure 2 is a flowchart of an embodiment of step S130 of the method for generating training data for a target model according to the present application; Figure 3 is a flowchart of another embodiment of the method for generating training data for a target model according to the present application; Figure 4 is a flowchart of an embodiment of the method for training a target model according to the present application; Figure 5 is a framework diagram of an embodiment of the apparatus for generating training data for a target model according to the present application; Figure 6 is a framework diagram of an embodiment of the electronic device according to the present application; Figure 7 is a framework diagram of an embodiment of the computer-readable storage medium according to the present application. DETAILED DESCRIPTION

[0018] The schemes of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0019] In the following description, specific details such as specific system structures, interfaces, techniques, etc. are presented in order to provide a thorough understanding of the present application for the sake of explanation, rather than for the sake of limitation.

[0020] The term "and / or" herein merely describes the association relationship of the associated objects, and means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the character " / " herein generally represents that the front and rear associated objects are in an "or" relationship. In addition, "multiple" herein means two or more than two. In addition, the term "at least one" herein means any one of multiple or any combination of at least two of multiple, for example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0021] Agent: Based on a large language model (such as LLaMA (Large Language Model for Academics), QWEN (Thousand Questions), etc.), the agent understands user problems and external tools (such as API (Application Programming Interface), databases, etc.) using a large language model, and plans tasks for user problems, i.e., decomposes complex problems into solvable sub-problems and selects appropriate tools to solve them one by one, and finally summarizes the tool return results to obtain answers.

[0022] Large Language Model: A generative model trained on massive text data with a large number of parameters (e.g. 7B, 14B, 32B, 70B, etc.), such as LLaMA, QWEN, etc., with excellent language understanding and text generation capabilities. Large language models take prompts as input and generate answers to user questions through prompts.

[0023] Prompt: As the input of a large language model, it is a text form of description that guides the large language model to generate answers to user questions. For the agent application of this proposal, the main components of the prompt include: 1) instructions, which describe the task the model needs to handle; 2) tool description, which describes the function of the tool; 3) output format, which describes the model output in the specified format; 4) input question, which is the user input question.

[0024] Please refer to Figure 1 , Figure 1 is a flowchart of an embodiment of the data generation method of the target model of the present application. Specifically, it can include the following steps: Step S110: Based on the data generation task, generate a number of data generation instructions.

[0025] The present application is mainly applied in the field of artificial intelligence, especially in the fields of large language models, agents and data synthesis technology. Through the generation of high-quality training data for agents by large language models, the tool usage ability and generalization of large language models in agent applications are improved.

[0026] In the present application, first, based on the preset data generation task (such as a specified business scenario or tool set), a basic model (such as LLM) is used to generate a set of original instructions in batches. Then, a multi-dimensional automatic screening mechanism is used to strictly screen the original instructions from three key dimensions of instruction quality, diversity and necessity, eliminating low-quality, redundant or repetitive data. Finally, the high-quality instruction set after screening and optimization is input into the model again to automatically generate the corresponding tool calling process and results (including complete logic such as intent analysis, tool selection, parameter transmission, result response), and then construct structured training samples of specified formats (such as instructions, reasoning, actions, results).

[0027] In some embodiments, the role of the data generation task is to guide the large language model to generate the corresponding question (i.e., instruction), which requires first understanding the capability range of each tool in the tool set (i.e., supported functions), and then manually writing the data generation task based on the functions supported by the tool. The data generation task includes single-step tasks (tasks that can be solved by calling one tool) and multi-step tasks (tasks that need to call multiple tools together to solve). For example, a single-step task can be written as "find a car according to the conditions" according to the tool "search_car" (which supports finding the corresponding vehicle according to the conditions); a multi-step task can be written as "find a car according to the conditions and query its travel trajectory in the past week" by combining the tools "search_car" and "query_trajectory" (which supports querying the travel trajectory of a specified vehicle within a specified time period). For data generation tasks, in addition to manual writing, a prompt can also be designed to guide the large language model to generate, and then suitable data generation tasks are screened out. Specifically, an example of a data generation task (JSON (JavaScript Object Notation, lightweight data exchange format) format) is as follows: { "task": "find a bus according to the conditions and query its travel trajectory in the past week", "tool_calling": ["search_person", "query_trajectory"], "example": "query the travel trajectory of the bus in the past week" }.

[0028] In addition, a task example can be selected from a seed pool as a target task, wherein the seed pool includes a plurality of task examples, the task examples in the seed pool include pre-set task examples and previously executed tasks, and the task examples at least include a task description part, a tool calling part and an instruction example part. Then, according to the target task, corresponding target tool information is obtained from the tool set, wherein the target tool information is the information of the tool required to be called when the data generation task is executed. Finally, according to the target task and the target tool information, the data generation task is generated.

[0029] In some embodiments, a stronger large language model (e.g., Qwen2.5-72B-Instruct, ChatGPT, etc.) can be used to generate instructions (i.e., task-related questions) for the data generation task according to the constructed data generation task. The key to instruction generation is to design an instruction generation prompt template (Prompt Template), which is related to the quality of instruction generation. The instruction generation prompt template mainly includes four parts: task description, tool description, requirements and constraints, as shown below: <system prompt> You are an instruction generation assistant in a certain field. Please generate instructions that meet the constraints according to the requirements provided.

[0030] <tool description> {tool_description} <requirements> Task: {task} Tool call: {tool_calling} Instruction example: {example} <constraints> - The generated instructions must be within the capabilities of the called tool; - Generate instructions in a colloquial style; ... In the tool description section, the slot "tool_description" only needs to fill in the ability description of the tool related to the current data generation task, avoiding exceeding the tool usage range; in the requirements section, fill in the data generation task that needs to be instantiated (see the data generation task example in the previous example), including the task (fill in the slot "task", in the task description section, the large language model needs to be explained how to generate tool instructions, and the description requires accurate and unambiguous to make the large language model understand how to generate tool instructions), tool calling (fill in the slot "tool_calling") and instruction example (fill in the slot "example"), where the instruction example can deepen the model's understanding of the current task; in the constraints section, mainly guide the model to generate real and diverse instructions, including the number, length, complexity, and expression diversity of the instructions, such as: ensuring that the generated instructions are within the capabilities of the tool, the difficulty of the instructions is diverse (i.e., containing simple, medium and complex different difficulties), the expression of the instructions is diverse (active, passive, synonymous expression, etc.), the instructions are expressed in accordance with the actual expression of the user (i.e., colloquial).

[0031] Specifically, the data generation task can be format-converted according to the instruction generation prompt, that is, the data generation task is format-converted according to the instruction generation prompt template. Then the instruction generation prompt is input into the large language model to generate instructions.

[0032] The instruction generation prompt needs to be iteratively optimized, and the prompt words are constantly optimized by analyzing whether the generated instructions meet the expectations, so as to build a final available prompt. In addition, based on the data generation instructions, the demand task in the seed pool is updated as a new example of the data generation task, so as to iteratively enrich the data generation task examples and guide the large language model to generate diverse instructions.

[0033] Step S120: Multi-dimensional screening is performed on the data generation instructions to obtain target instructions.

[0034] The multi-dimensional screening includes at least two of instruction quality screening, diversity screening, and necessity screening. The diversity screening is used to retain one instruction in a plurality of similar instructions in the data generation instructions. The necessity screening is used to retain a data generation instruction that cannot be correctly executed in the data generation instructions.

[0035] Although the instruction set generated based on the large language model has certain quality and diversity, there may be problems such as the instruction not being within the tool range, the instruction expression not conforming to logic, the instruction being too similar, and the instruction expression being ambiguous. In order to solve this problem, the present application screens a high-quality instruction set from three dimensions of quality, diversity, and necessity. Please refer to steps S121 to S124.

[0036] Step S121: The data generation instructions are subjected to instruction quality screening to obtain first screened instructions.

[0037] The quality of the instruction directly affects the tool use performance of the target model. Low-quality instructions may introduce errors into the target model, causing the target model to have a deviation in understanding between the instruction and the tool. For example, if the instruction is inconsistent with the tool capability (that is, the instruction cannot be solved by calling the corresponding tool), such a deviated instruction, as a training sample, will introduce a deviation into the target model, causing the target model to tend to select the tool specified in the training data for such instructions in actual application, resulting in tool misadjustment.

[0038] To solve the above problems, the present proposal uses a strong language model (such as Qwen2.5-72B-Instruct, ChatGPT, etc.) to score the instruction set generated in the previous step. Compared with manual quality checking, the quality scoring method based on large language models can automatically score instructions based on multiple evaluation indicators, with advantages such as flexibility, speed, objectivity, and scalability. Specifically, the data generation instructions are de-duplicated to obtain remaining de-duplicated instructions, for example, using the MiniHash method for de-duplication to filter out some repeated instructions. Then, the remaining de-duplicated instructions are evaluated based on a strong language model to obtain instruction scores corresponding to the remaining de-duplicated instructions. If the instruction score is greater than a preset score, the remaining de-duplicated instruction corresponding to the instruction score is taken as the first screened instruction. The key to scoring instructions based on a strong language model is to design a prompt template for scoring. The instruction quality evaluation template designed in this embodiment scores each instruction from the aspects of instruction clarity, completeness, conciseness, task relevance, and instruction consistency with tools, with a higher score representing higher quality. By removing low-score instructions (such as instructions with a score of less than 7), a high-score instruction set can be screened. The instruction quality evaluation template is as follows: Please score the generated instructions according to the scoring criteria. The score range is 1 to 10, where 1 represents very poor and 10 represents very good.

[0039] <Scoring Criteria> Evaluation dimensions: Clarity: Whether the instruction is clearly expressed without ambiguity; Completeness: Whether the instruction provides sufficient information to complete the task; Conciseness: Whether the instruction is concise and avoids redundancy; Relevance: Whether the instruction is relevant to the corresponding task; Consistency: Whether the instruction can be solved by invoking the corresponding tool; … Scoring criteria: 1 point: Very poor, completely does not meet the standards of this dimension.

[0040] 2-3 points: Poor, with obvious shortcomings.

[0041] 4-6 points: Average, with some minor issues, but generally acceptable.

[0042] 7-8 points: Good, meets most standards, with a small amount of room for improvement.

[0043] 9-10 points: Excellent, completely meets the standards, with no obvious shortcomings.

[0044] <Constraints> According to the above scoring criteria, score the generated instructions and provide a brief scoring reason; Output in this format: {“reason”: scoring reason; “score”: score}; … <Generated instructions> Instruction: {instruction}; Instruction corresponding task: {task}; Tool calling by instruction: {tool_calling}; Tool description: {tool_description}.

[0045] Step S122: Diversity screening on the first screening instructions to obtain the second screening instructions.

[0046] In some embodiments, quality assessment can improve the quality of instructions and avoid introducing bias or errors into target model training by low-quality instructions. However, in the high-score instruction set (i.e. the first screening instructions) screened out, there are inevitably some similar instructions. These instructions can be considered as redundant instructions, and too many similar instructions may cause overfitting of the target model, reducing the generalization of the target model. In order to solve this problem, the core sampling based diversity screening method is used in this embodiment to screen out core instructions (i.e. representative instructions) from the high-score instruction set. These instructions can better represent the characteristics of the entire instruction data set, avoid too many redundant instructions in the training data, reduce the risk of overfitting of the target model, and improve the generalization ability of the target model.

[0047] Therefore, similarity calculation can be performed between the first screening instructions to obtain instruction similarity. If the instruction similarity meets the preset similarity condition, the first screening instruction corresponding to the instruction similarity is taken as the second screening instruction.

[0048] Specifically, K-Center-Greedy algorithm is used to sample the first screening instructions to select K representative instructions from them to construct a diversity instruction set (i.e. the second screening instructions). The algorithm implementation process is as follows: Input. The first screening instructions D, and the number of instructions K to be screened out.

[0049] Output. A diversity instruction set S containing K core instructions.

[0050] Step 1. Initialize an empty set S to store the sampled instructions.

[0051] Step 2. Randomly select an instruction from D as the first center point.

[0052] Step3. In the loop, the distance between each unselected point and the selected center point is calculated in each iteration, and the point with the largest distance is selected as the new center point.

[0053] Step4. Repeat the above steps until K center points are selected and stored in set S.

[0054] In addition, cosine similarity, Manhattan distance, Pearson correlation coefficient, etc. can also be used to calculate the similarity between the first screening instructions, which is not limited here.

[0055] Step S123: Use the second screening instruction to screen the necessity of the first screening instruction to obtain the third screening instruction.

[0056] In some embodiments, for a tool instruction, if the target model (i.e. a large language model that needs to be trained in the agent application) can select the correct tool, it indicates that the target model already has the ability to solve such instructions. Conversely, if the target model cannot select the correct tool for the instruction, it indicates that the target model lacks the ability to solve such instructions, so the instruction is necessary to enhance the tool usage ability of the target model, i.e. it is considered a necessary instruction. This essentially selects difficult samples for the target model, and increases difficult samples to improve the model's ability to solve complex instructions.

[0057] To this end, the second screening instruction can be used to train the target model (with the tool set and instruction as input, and the tool name called as output) to obtain a tool selection model that can predict the tool name required for an instruction. Then, the tool selection model is used to predict the first screening instruction to obtain a predicted tool, and if the predicted tool is different from the tool label information to be used, the corresponding first screening instruction is taken as the third screening instruction.

[0058] Step S124: De-duplicate the second screening instruction and the third screening instruction to obtain the target instruction.

[0059] After the three steps of instruction quality screening, diversity screening and necessity screening, a diverse instruction set and a necessary instruction set can be obtained from the first screening instruction set. Finally, the diverse instruction set and the necessary instruction set are merged and de-duplicated to obtain a final target instruction.

[0060] Step S130: Input several target instructions into the large language model for execution to obtain the execution result corresponding to the target instruction.

[0061] In some embodiments, for the target instruction set screened out, each sample contains an instruction, a task corresponding to the instruction, and a tool calling required by the instruction, and a training data set of tool usage can be constructed, i.e., taking the instruction as input and the tool as target. However, this training data set lacks the entire reasoning process from problem analysis to tool calling decision. Therefore, the target instruction can be format-converted to obtain an execution generation prompt. Then the execution generation prompt is input into a large language model for execution to obtain an execution result. In addition, the execution generation prompt is input into the large language model for execution to obtain an initial execution result, the initial execution result is checked to obtain a checking result, and if the checking result is correct, the initial execution result corresponding to the correct checking result is taken as the final execution result and output.

[0062] Please refer to Figure 2 In this embodiment, a large language model is used to generate an execution result of "Thought-Action-ActionInput-Observation-Thought-Final Answer" for an instruction.

[0063] In the execution result, Thought is the thinking process of the large language model for the instruction, including analyzing the user's intention, understanding the function of the tool, and associating to the most suitable tool according to the user's intention; Action is the tool name selected by the large language model after thinking; Action Input is the input parameter of the selected tool, which requires the large language model to understand the user's problem and extract the corresponding parameters; Observation is the return result after the tool is executed, which can provide feedback and context information for the next thinking and planning of the large language model; and Final Answer is the final execution result of the input instruction. The key to generating the execution result is to design an execution generation prompt to guide the large language model to generate a high-quality execution result for the instruction. An execution generation prompt template is as follows: Please generate the entire thinking process from the problem to the calling of the tool according to the input instruction and the tool calling sequence.

[0064] <Tool> {tool_description} <Example> {demonstrations} <Constraint> -Think and plan the problem according to the tool calling sequence in Tool_calling; -The Thought part needs to analyze the instruction intention and explain the reason for calling the specified tool; … <INPUT> Instruction: {instruction} Tool_calling: {tool_calling}.

[0065] Among them, the slot "tool_description" only needs to fill in the tool specified in Tool_calling; the slot "demonstrations" fills in the examples; the slot "instruction" fills in the instructions; the slot "tool_calling" fills in the list of tools called by the instructions. After filling in the corresponding content into the execution result generation prompt template, the execution result generation prompt can be obtained, and the prompt can be used to guide the large language model to generate the execution result, as shown in Figure 6 The execution result generation prompt provides the tools called by the instructions (tool_calling) as clues, and provides the functions, input parameters and output parameters of the tools for guiding the large language model to generate the entire thinking process (Thought) for the instructions from the question to the tool selection (Action), tool input parameters (Action Input), tool output results (Observation) and final execution result (Final Answer). Among them, the tool output result (Observation) is completely generated by the large language model, which can avoid the problems of complex dependency environment installation, slow return speed and other problems in the actual tool calling process. The Final Answer is based on the previous tool execution result, and the final execution result of the question is sorted out by the teacher large language model. In addition, the prompt also provides examples, which can deepen the understanding of the large language model for the generation of execution results, and guide the model to output according to the specified format. When the large language model generates the execution result, it needs to be checked for content and format, mainly checking whether the overall format of the execution result is complete, whether the tool name is correct, whether the tool input parameter format and content are correct, whether the tool return result can be parsed, etc. If the verification fails, it will be regenerated.

[0066] Specifically, the target instruction is: { "instruction": "Find a black small car with license plate Beijing A·XXXXX", "task": "Find a car according to the conditions", "tool_calling": ["search_car"], "score": 9, }.

[0067] The target instruction is converted into an execution generation prompt: Please generate the entire thought process from the question to the tool call based on the input instructions and the order of tool calls.

[0068] <Tool> ·search_car Function: Search for unknown vehicles according to one or more specified conditions. The conditions include vehicle attributes (such as brand, color, license plate, type (including sedan cars, SUVs, vans, trucks, buses, etc.))... Input parameters:... Output parameters:... ·... <Example> Instruction: Find a black sedan car; Tool_calling: ["search_car"]; Thought: The user wants to find a black sedan car. The search_car tool supports finding cars according to vehicle type and license plate. Therefore, choose search_car to solve this problem; Action: search_car; Action Input: {"conditions": {"type": "sedan car", "license plate": "Beijing A"}}; Observation: [{"license plate": "Beijing A", "color": "black", "type": "sedan car",...}...] Thought: I have obtained the answer and do not need to call the tool anymore; Final Answer: Have found the vehicle with Beijing license plate, type of sedan car and color of black... ... <Constraint> - Think and plan for the question according to the order of tool calls in Tool_calling; -... <Input> Instruction: Search for a vehicle that is black, type of sedan car and license plate is from Beijing; Tool_calling: ["search_car"].

[0069] Among them, SUV is Sport Utility Vehicle.

[0070] The execution generation prompt is input into the large language model for execution to obtain an initial execution result. The initial execution result is subjected to content and format verification to obtain a verification result. The verification result is judged. If the verification result is correct, the initial execution result corresponding to the correct verification result is taken as a final execution result and is output.

[0071] Step S140: based on the target instructions and the execution results, constructing training data for the target model.

[0072] In some embodiments, based on the generated target instructions and the corresponding execution results, training data for the target model is constructed.

[0073] The present application generates instruction sets by designing instruction generation prompts to guide the large language model to generate instructions, designs multiple quality evaluation dimensions (quality, diversity, and necessity) to screen high-quality instruction sets, and designs answer generation prompts to generate answers for the instructions, ultimately obtaining high-quality agent training sets. Based on the constructed agent training sets, the large language model can be fine-tuned to effectively improve the tool usage ability and generalization of the large language model in the agent application.

[0074] In addition, the agent training data constructed by the present application can be used to fine-tune language models of different sizes, especially to improve the tool selection performance of small-size language models, so that small-size language models can be deployed in agent applications, reducing deployment costs.

[0075] Please refer to Figure 3 , Figure 3 is a flowchart of another embodiment of the target model training data generation method of the present application. Specifically, it can include the following steps: Step S310: based on the data generation task, generating a plurality of data generation instructions.

[0076] This step is the same as step S110 described above, and will not be repeated here.

[0077] Step S320: multi-dimensional screening of the plurality of data generation instructions to obtain a plurality of target instructions.

[0078] This step is the same as step S120 described above, and will not be repeated here.

[0079] Step S330: inputting the plurality of target instructions into the large language model for execution to obtain execution results corresponding to the target instructions.

[0080] This step is the same as step S130 described above, and will not be repeated here.

[0081] Step S340: based on the target instructions and the execution results, constructing initial training data for the target model.

[0082] This step is the same as step S140 described above, and will not be described here.

[0083] Step S350: screening the initial training data to obtain final training data.

[0084] In some embodiments, the similarity between each initial training data can be calculated to obtain the training data similarity. The sum of the similarities between a training data and other training data is obtained, and the average similarity is obtained by averaging. The average similarity of the training data is compared with the similarity threshold value, if greater than or equal to the similarity threshold value, the training data is retained; if less than the similarity threshold value, the training data is discarded.

[0085] Please refer to Figure 4 , Figure 4 is a flowchart of an embodiment of the target model training method of the present application. Specifically, it can include the following steps: Step S410: format conversion of the sample instruction in the training data to obtain a sample execution generation prompt.

[0086] The training data is obtained by the above target model training data generation method.

[0087] In some embodiments, the sample instruction can be format converted according to the execution generation prompt template to obtain the sample execution generation prompt.

[0088] Step S420: inputting the sample execution generation prompt into the target model for execution to obtain a sample execution result.

[0089] Step S430: adjusting the network parameters of the target model based on the difference between the sample execution result and the labeled sample result in the training data until the target model converges.

[0090] In some embodiments, the difference between the sample execution result and the labeled sample result in the training data (i.e. the execution result in the above target model training data generation method) is weighted and summed to obtain a loss value. The network parameters of the target model are adjusted according to the loss value, and the iteration is performed until the target model converges.

[0091] Those skilled in the art can understand that in the above method of the specific embodiments, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0092] Please refer to Figure 5 , Figure 5is a schematic diagram of an embodiment of the target model training data generation apparatus 500. The target model training data generation apparatus 500 includes a generation module 510, a screening module 520, an execution module 530, and a data construction module 540. The generation module 510 performs a data generation task based on data generation instructions. The screening module 520 performs multi-dimensional screening on the data generation instructions to obtain target instructions, wherein the multi-dimensional screening includes at least two of instruction quality screening, diversity screening, and necessity screening, the diversity screening is used to retain one instruction in a plurality of similar instructions in the data generation instructions, and the necessity screening is used to retain data generation instructions that cannot be executed correctly. The execution module 530 executes the input of the target instructions into the large language model to obtain the execution results corresponding to the target instructions. The data construction module 540 constructs the training data for the target model based on the target instructions and the execution results.

[0093] In some embodiments, the generation module 510 performs a data generation task based on data generation instructions, including: performing format conversion based on the data generation task to obtain instruction generation prompts; inputting the instruction generation prompts into the large language model for instruction generation to obtain the data generation instructions.

[0094] In some embodiments, the generation module 510 performs format conversion based on the data generation task to obtain instruction generation prompts, including: selecting a task example from a seed pool as a target task, wherein the seed pool includes a plurality of task examples; based on the target task, obtaining corresponding target tool information from a tool set, wherein the target tool information is information of a tool required to be called when the data generation task is executed; based on the target task and the target tool information, generating the data generation task; after inputting the instruction generation prompts into the large language model for instruction generation to obtain the data generation instructions, including: updating the demand task in the seed pool based on the data generation instructions.

[0095] In some embodiments, the screening module 520 performs multi-dimensional screening on the data generation instructions to obtain target instructions, including: performing instruction quality screening on the data generation instructions to obtain first screening instructions; performing diversity screening on the first screening instructions to obtain second screening instructions; performing necessity screening on the first screening instructions using the second screening instructions to obtain third screening instructions; and performing deduplication on the second screening instructions and the third screening instructions to obtain the target instructions.

[0096] In some embodiments, the screening module 520 performs instruction quality screening on the data generation instructions to obtain first screening instructions, including: performing deduplication on the data generation instructions to obtain deduplication remaining instructions; performing evaluation on the deduplication remaining instructions to obtain instruction scores corresponding to the deduplication remaining instructions; and in response to the instruction scores being greater than a preset score, taking the deduplication remaining instructions corresponding to the instruction scores as the first screening instructions.

[0097] In some embodiments, the screening module 520 performs diversity screening on the first screening instructions to obtain second screening instructions, including: performing similarity calculation between the first screening instructions to obtain instruction similarities; and in response to the instruction similarities satisfying a preset similarity condition, taking the first screening instructions corresponding to the instruction similarities as the second screening instructions.

[0098] In some embodiments, the screening module 520 performs the second screening instructions including to-be-used tool tag information, performs necessity screening on the first screening instructions using the second screening instructions to obtain third screening instructions, including: training the target model using the second screening instructions to obtain a tool selection model; predicting the first screening instructions using the tool selection model to obtain predicted tools; and in response to the predicted tools being different from the to-be-used tool tag information, taking the corresponding first screening instructions as the third screening instructions.

[0099] In some embodiments, the execution module 530 performs inputting the target instructions into a large language model for execution to obtain an execution result corresponding to the target instructions, including: performing format conversion on the target instructions to obtain an execution generation prompt; and inputting the execution generation prompt into the large language model for execution to obtain the execution result.

[0100] In some embodiments, the execution module 530 performs inputting the execution generation prompt into the large language model to obtain the execution result, including: inputting the execution generation prompt into the large language model for execution to obtain an initial execution result; performing verification on the initial execution result to obtain a verification result; and in response to the verification result being correct, taking the initial execution result corresponding to the correct verification as a final execution result and outputting the final execution result.

[0101] Please refer to Figure 6 , Figure 6 is a frame schematic diagram of an embodiment of the electronic device 60. The electronic device 60 includes a memory 61 and a processor 62 coupled with each other. The processor 62 is configured to execute program instructions stored in the memory 61 to implement the steps of any of the above-described target model training data generation method embodiments, or to implement the steps in any of the above-described target model training method embodiments. In one specific implementation scenario, the electronic device 60 can include but is not limited to: a microcomputer, a server, in addition, the electronic device 60 can also include a notebook computer, a tablet computer, and the like mobile devices, which are not limited here.

[0102] In particular, the processor 62 is configured to control itself and the memory 61 to implement the steps of any of the above-described embodiments of the target model training data generation method, or implement the steps in any of the above-described embodiments of the target model training method. The processor 62 can also be referred to as a CPU (Central Processing Unit). The processor 62 can be an integrated circuit chip having a processing capability of signals. The processor 62 can also be a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. In addition, the processor 62 can be implemented by an integrated circuit chip together.

[0103] Please refer to Figure 7 , Figure 7 is a schematic diagram of an embodiment of the computer readable storage medium 70 of the present application. The computer readable storage medium 70 stores program instructions 701 capable of being executed by the processor, the program instructions 701 being configured to implement the steps of any of the above-described embodiments of the target model training data generation method, or implement the steps in any of the above-described embodiments of the target model training method.

[0104] In some embodiments, the apparatus provided by the embodiments of the present disclosure has functions or includes modules that can be used to perform the methods described in the above method embodiments, and the specific implementation can refer to the description of the above method embodiments. For brevity, it will not be described here.

[0105] The above description of each embodiment tends to emphasize the differences between each embodiment, and the same or similar parts can be mutually referred to. For brevity, they will not be described here.

[0106] In several embodiments provided in the present application, it should be understood that the disclosed methods and apparatuses can be implemented in other manners. For example, the division of the apparatus embodiments described above is merely a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0107] In addition, each function unit in the various embodiments of the present application can be integrated into a processing unit, or each unit can exist physically independently, or two or more units can be integrated into one unit. The integrated unit can be implemented in the form of hardware or in the form of a software function unit.

[0108] If the integrated unit is implemented in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the methods in the various embodiments of the present application. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, and various other media that can store program codes.

Claims

1. A method for generating training data of a target model, characterized in that, The method comprises the following steps: Based on the data generation task, generate a plurality of data generation instructions; Multi-dimensional screening is performed on the plurality of data generation instructions to obtain a plurality of target instructions, wherein the multi-dimensional screening includes at least two of instruction quality screening, diversity screening and necessity screening, the diversity screening is used to retain one instruction in a plurality of similar instructions in the plurality of data generation instructions, and the necessity screening is used to retain a data generation instruction that cannot be correctly executed in the plurality of data generation instructions; The plurality of target instructions are input into a large language model for execution to obtain an execution result corresponding to the target instruction; Based on the plurality of target instructions and the execution result, training data for a target model is constructed.

2. The method of claim 1, wherein, The method comprises the following steps: Based on the data generation task, generate a plurality of data generation instructions, comprising: Based on the data generation task, a format conversion is performed to obtain an instruction generation prompt; 3. The method of claim 2, wherein, The instruction generation prompt is input into the large language model for instruction generation to obtain a plurality of data generation instructions. Before the format conversion based on the data generation task is performed to obtain the instruction generation prompt, comprising: Select a task example from a seed pool as a target task, wherein the seed pool comprises a plurality of task examples; Based on the target task, corresponding target tool information is obtained from a tool set, wherein the target tool information is information of a tool required to be called when the data generation task is executed; Based on the target task and the target tool information, the data generation task is generated; After the instruction generation prompt is input into the large language model for instruction generation to obtain a plurality of data generation instructions, comprising:

4. The method of claim 1, wherein, Based on the data generation instruction, update the demand task in the seed pool. The method comprises the following steps: The data generation instructions are subjected to instruction quality screening to obtain first screening instructions; The first screening instructions are subjected to diversity screening to obtain second screening instructions; The necessity screening is performed on the first screening instructions by using the second screening instructions to obtain third screening instructions; 5. The method of claim 4, wherein, The second screening instructions and the third screening instructions are subjected to deduplication to obtain the target instructions. The method comprises the following steps: The data generation instructions are subjected to deduplication to obtain deduplication remaining instructions; The deduplication remaining instructions are evaluated to obtain instruction scores corresponding to the deduplication remaining instructions; In response to the instruction score being greater than a preset score value, the deduplication remaining instruction corresponding to the instruction score is taken as the first screening instruction; And / or, the method comprises the following steps: Similarity between the first screening instructions is calculated to obtain instruction similarity; In response to the instruction similarity satisfying a preset similarity condition, the first screening instruction corresponding to the instruction similarity is taken as a second screening instruction; And / or, the second screening instruction includes to-be-used tool label information, the necessity screening of the first screening instruction by the second screening instruction obtains a third screening instruction, including: Training the target model by the second screening instruction obtains a tool selection model; Predicting the first screening instruction by the tool selection model obtains a predicted tool; In response to the predicted tool being different from the to-be-used tool label information, corresponding to the first screening instruction is taken as the third screening instruction.

6. The method of claim 1, wherein, The target instruction is input into a large language model to obtain an execution result corresponding to the target instruction, including: Converting the format of the target instruction to obtain an execution generation prompt; Inputting the execution generation prompt into the large language model to obtain the execution result.

7. The method of claim 6, wherein, The execution generation prompt is input into the large language model to obtain the execution result, including: Inputting the execution generation prompt into the large language model to obtain an initial execution result; Verifying the initial execution result to obtain a verification result; In response to the verification result being correct, the initial execution result corresponding to the correct verification result is taken as the final execution result and output.

8. A target model training method characterized by comprising: Including: Converting the format of the sample instruction in the training data to obtain a sample execution generation prompt, wherein the training data is obtained by the target model training data generation method of any one of claims 1-7; Inputting the sample execution generation prompt into the target model to obtain a sample execution result; Based on the difference between the sample execution result and the labeled sample result in the training data, adjusting the network parameters of the target model until the target model converges.

9. An electronic device, comprising: The processor is used to execute the program instructions stored in the memory to realize the target model training data generation method of any one of claims 1-7, or realize the target model training method of claim 8.

10. A computer-readable storage medium having stored thereon program instructions, wherein, The program instructions are executed by the processor to realize the target model training data generation method of any one of claims 1-7, or realize the target model training method of claim 8.

Images