Training data generation method and device based on dynamic feedback, equipment and medium

Abstract

The invention provides a training data generation method and device based on dynamic feedback, equipment and a medium, and the method comprises the steps: carrying out the performance evaluation of a target model in a current environment, and determining a performance feedback state of the target model in the current training environment; the teacher model is controlled to make a data generation plan based on the performance feedback state, the data generation plan is converted based on a data generation engine, and training data of the target model is generated; and training the target model based on the training data, determining an updated target model, continuing to determine the training data of the updated target model, training the target model according to the new training data, and stopping training the target model until the performance feedback state meets a preset requirement. And training data is generated, and the target model is iteratively improved, so that the problems of low data generation efficiency, high cost, non-uniform data quality, lack of dynamic adaptability and universality and the like in the prior art are solved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, equipment and medium for generating training data based on dynamic feedback. Background Art

[0002] The performance improvement of artificial intelligence models depends largely on high-quality training data, which can greatly improve the generalization ability and adaptability of the model. Therefore, the means of obtaining training data have been continuously improved from a large number of manual annotations at the beginning. At present, the more mainstream and efficient methods of obtaining training data include synthetic data generation and active learning. However, the current data generation has technical problems such as high cost and low efficiency of manual intervention, lack of dynamic feedback mechanism, insufficient skill management and automation, and limited versatility and adaptability. Therefore, how to generate training data for the model has become a technical issue that cannot be underestimated. Summary of the invention

[0003] In view of this, the purpose of this application is to provide a training data generation method, device, equipment and medium based on dynamic feedback, dynamically adjust the generation strategy through the feedback mechanism, generate more accurate training data, significantly improve the model training effect, and avoid the problems of low data generation efficiency, high cost, uneven data quality, lack of dynamic adaptability and versatility in the prior art.

[0004] The embodiment of the present application provides a training data generation method based on dynamic feedback, and the training data generation method includes: Performing performance evaluation processing on the target model in the current environment to determine the performance feedback state of the target model in the current training environment; wherein the training environment includes an open training environment, a processing skill training environment, and a processing skill structure tree training environment; Controlling the teacher model to formulate a data generation plan based on the performance feedback state, and converting the data generation plan based on the data generation engine to generate training data for the target model; The target model is trained based on the training data to determine the updated target model, the training data of the updated target model is continued to be determined, and the target model is trained according to the new training data until the performance feedback state meets the preset requirements and the training of the target model is stopped.

[0005] In a possible implementation, for the open training environment, the control teacher model formulates a data generation plan based on the performance feedback state, including: The teacher model determines the probability distribution of error samples under each data type based on the error sample list in the performance feedback state; Based on the probability distribution of error samples under each data type, significant error data types in the error sample list are determined, and a data generation plan is formulated according to the significant error data types.

[0006] In a possible implementation, for the processing skill training environment, the control teacher model develops a data generation plan based on the performance feedback state, including: Based on the list of processing skills in the performance feedback state and the list of performance errors at the processing skill level, the teacher model determines the weak processing skills corresponding to the target model and formulates a data generation plan for the weak processing skills; wherein the processing skills are the skills of the target model for processing tasks.

[0007] In a possible implementation, for the processing skill structure tree training environment, the control teacher model formulates a data generation plan based on the performance feedback state, including: Based on the list of processing sub-skills in the performance feedback state and the list of performance errors at the processing sub-skill level, the teacher model determines the weak processing sub-skills corresponding to the target model, and formulates a data generation plan for the weak processing sub-skills; wherein the processing skill structure tree is constructed with the processing skill as the root node and the processing sub-skill as the leaf node, and the processing sub-skill is the hierarchical skill of the processing skill.

[0008] In a possible implementation, for the processing skill structure tree training environment, the control teacher model formulates a data generation plan based on the performance feedback state, including: The teacher model is controlled to perform expansion processing on the current processing sub-skills in the processing sub-skills list and dynamically adjust the priority of the corresponding processing tasks based on the skill performance results corresponding to the processing sub-skills list, so as to formulate the data generation plan.

[0009] In a possible implementation manner, controlling the teacher model to perform expansion processing on the current processing sub-skill in the processing sub-skill list includes: The sample training data of the processing skill is input into the teacher model, the sample training data is expanded into sub-skills, and new sub-skills and categories of new sub-skills are determined.

[0010] In a possible implementation manner, after inputting the sample training data of the processing skill into the teacher model, performing sub-skill expansion on the sample training data, and determining new sub-skills and new sub-skill categories, the training data generation method further includes: Detecting whether the subdivision degree of the processing sub-skills of the processing skill meets a preset threshold; If so, stop expanding the new processing sub-skills; If not, continue to expand on new processing sub-skills.

[0011] The embodiment of the present application further provides a training data generation device based on dynamic feedback, the training data generation device comprising: A model evaluation module is used to perform performance evaluation processing on the target model in the current environment to determine the performance feedback status of the target model in the current training environment; wherein the training environment includes an open training environment, a processing skill training environment, and a processing skill structure tree training environment; A data generation agent module is used to control the teacher model to formulate a data generation plan based on the performance feedback state, and to convert the data generation plan based on the data generation engine to generate training data for the target model; A model training module is used to train the target model based on the training data, determine the updated target model, continue to determine the training data of the updated target model, train the target model according to the new training data, and stop training the target model when the performance feedback status meets the preset requirements.

[0012] An embodiment of the present application also provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the training data generation method based on dynamic feedback as described above are performed.

[0013] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the training data generation method based on dynamic feedback as described above are executed.

[0014] The training data generation method, device, equipment and medium based on dynamic feedback provided by the embodiment of the present application, the training data generation method includes: performing performance evaluation processing on the target model in the current environment, and determining the performance feedback status of the target model in the current training environment; wherein the training environment includes an open training environment, a processing skill training environment and a processing skill structure tree training environment; controlling the teacher model to formulate a data generation plan based on the performance feedback status, converting the data generation plan based on the data generation engine, and generating training data for the target model; training the target model based on the training data, determining the updated target model, continuing to determine the updated training data of the target model, training the target model according to the new training data, and stopping the training of the target model until the performance feedback status meets the preset requirements. By dynamically adjusting the generation strategy through the feedback mechanism, the generated training data is more accurate, which significantly improves the model training effect, and avoids the problems of low data generation efficiency, high cost, uneven data quality, lack of dynamic adaptability and versatility in the prior art.

[0015] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A flowchart of a method for generating training data based on dynamic feedback provided in an embodiment of the present application; Figure 2 A schematic diagram of the expansion process of the processing skill structure tree provided in the embodiment of the present application; Figure 3 A schematic diagram of a training data generation method based on dynamic feedback provided in an embodiment of the present application; Figure 4 A schematic diagram of the structure of a training data generation device based on dynamic feedback provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.

[0019] First, the application scenarios to which the present application is applicable are introduced. The present application can be applied in the field of data processing technology.

[0020] Research has found that the performance improvement of artificial intelligence models depends to a large extent on high-quality training data, which can greatly improve the generalization ability and adaptability of the model. Therefore, the means of obtaining training data have been continuously improved from a large number of manual annotations at the beginning. At present, the more mainstream and efficient methods of obtaining training data include synthetic data generation and active learning. However, the current data generation has technical problems such as high cost and low efficiency of manual intervention, lack of dynamic feedback mechanism, insufficient skill management and automation, and limited versatility and adaptability. Therefore, how to generate training data for the model has become a technical issue that cannot be underestimated.

[0021] Based on this, the embodiments of the present application provide a training data generation method, device, equipment and medium based on dynamic feedback. The generation strategy is dynamically adjusted through the feedback mechanism, and the generated training data is more accurate, which significantly improves the model training effect and avoids the problems of low data generation efficiency, high cost, uneven data quality, lack of dynamic adaptability and versatility in the prior art.

[0022] See also Figure 1 , Figure 1 The following is a flow chart of a method for generating training data based on dynamic feedback provided in an embodiment of the present application. Figure 1 As shown in , the training data generation method provided in the embodiment of the present application includes: S101: Perform performance evaluation processing on the target model in the current environment to determine the performance feedback status of the target model in the current training environment; wherein the training environment includes an open training environment, a processing skill training environment, and a processing skill structure tree training environment.

[0023] In this step, a performance evaluation process is performed on the target model in the current environment to determine the performance feedback status of the target model in the current training environment.

[0024] Among them, the performance evaluation of the target model in the current environment includes: Error sample evaluation: At which specific data points the target model made mistakes, which can be classification errors, prediction bias, or specific examples where the model failed to give reasonable results. Skill weakness distribution evaluation: Clearly point out which skills or task modules the target model performs weakly, such as fraction operations in mathematics or recursive processing in programming. Performance indicators: Evaluation indicators such as accuracy, recall, F1 value, etc. are used to measure the overall performance of the target model in different tasks.

[0025] Among them, an open training environment is usually used to describe an environment without fixed rules or framework constraints. In such an environment, systems or individuals can freely explore, learn, and adapt to new situations without being restricted by strict preset rules. For example, in the field of artificial intelligence, an open environment may mean that the algorithm can make autonomous decisions and deal with unforeseen situations without explicit instructions. Processing skill training environment: If we take video games or simulation training systems as an example, a skill list environment may refer to an environment with specific task goals, in which users (or their agents) can achieve goals by performing a series of predefined actions (ie "skills"). Processing skills in a skill structure tree training environment abstracts skills into a structure tree representation.

[0026] S102: Control the teacher model to formulate a data generation plan based on the performance feedback state, and convert the data generation plan based on the data generation engine to generate training data for the target model.

[0027] In this step, the teacher model is controlled to formulate a data generation plan according to the performance feedback status, and the data generation plan is converted and processed according to the data generation engine to generate training data for the target model.

[0028] Here, according to the performance feedback status s t Develop a data generation strategy , specifically expressed as the conditional distribution π( p t | s t ) in the sample generation data generation plan p t . Sampling formula p t ~π( p t | s t) indicates that the data generation plan p t Based on the current status s t The result of the probability distribution generated by the engine function Engine Will p t Converted into training data d t 。

[0029] Among them, a large language model (such as GPT) can be used to generate question and answer pairs, with the data generation plan as the questions of the large language model and the answer pairs as training data.

[0030] In a possible implementation, for the open training environment, the control teacher model formulates a data generation plan based on the performance feedback state, including: A: The teacher model determines the probability distribution of error samples under each data type based on the error sample list in the performance feedback state.

[0031] Here, the teacher model determines the probability distribution of error samples under each data type based on the list of error samples in the performance feedback state.

[0032] The error sample list records the specific errors of each sample corresponding to the target model.

[0033] B: Based on the probability distribution of error samples under each data type, determine the significant error data types in the error sample list, and formulate a data generation plan according to the significant error data types.

[0034] Here, according to the probability distribution of error samples under each data type, the significant error data types in the error sample list are determined, and a data generation plan is formulated according to the significant error data types.

[0035] Among them, the significant error data type is the summer vacation type corresponding to the error samples whose probability distribution is greater than the preset probability distribution threshold, so that the teacher model can generate more targeted training data based on the feedback error samples to strengthen the weak skills of the target model.

[0036] In a possible implementation, for the processing skill training environment, the control teacher model develops a data generation plan based on the performance feedback state, including: Based on the list of processing skills in the performance feedback state and the list of performance errors at the processing skill level, the teacher model determines the weak processing skills corresponding to the target model and formulates a data generation plan for the weak processing skills; wherein the processing skills are the skills of the target model for processing tasks.

[0037] Here, the teacher model determines the weak processing skills corresponding to the target model based on the list of processing skills in the performance feedback state and the list of performance errors at the processing skill level, and formulates a data generation plan for the weak processing skills.

[0038] Among them, the weak processing skill is a processing skill in which the number of performance errors at the processing skill level is greater than the preset number of errors.

[0039] Here, in the Processing Skills Training Environment, the state includes a list of performance errors at the skill level for the target model and a list of skills, where the Processing Skills List represents the specific skills that need to be mastered in the target model. This approach first uses a large language model to automatically infer human-interpretable skills from the training data. This approach generates a more structured performance feedback state, making the agent's processing tasks more interpretable because it can obtain explicit feedback on which skills the target model performs weakly. Similar to the Open Training Environment, the Processing Skills Training Environment also requires the agent to generate data directly. By adding skills to the state representation, the Processing Skills Training Environment provides higher structure and interpretability to the processing task than the Open Environment.

[0040] In a possible implementation, for the processing skill structure tree training environment, the control teacher model formulates a data generation plan based on the performance feedback state, including: Based on the list of processing sub-skills in the performance feedback state and the list of performance errors at the processing sub-skill level, the teacher model determines the weak processing sub-skills corresponding to the target model, and formulates a data generation plan for the weak processing sub-skills; wherein the processing skill structure tree is constructed with the processing skill as the root node and the processing sub-skill as the leaf node, and the processing sub-skill is the hierarchical skill of the processing skill.

[0041] Here, the teacher model determines the weak processing sub-skills corresponding to the target model based on the list of processing sub-skills in the performance feedback state and the list of performance errors at the processing sub-skill level, and formulates a data generation plan for the weak processing sub-skills.

[0042] Among them, each processing skill corresponds to a processing skill structure tree.

[0043] Here, if the processing task of the target model is a mathematical processing task, the processing skill under the processing task is the "algebra" skill, and the sub-processing skills under the "algebra" skill are "linear equation" and "polynomial decomposition".

[0044] In a possible implementation, for the processing skill structure tree training environment, the control teacher model formulates a data generation plan based on the performance feedback state, including: The teacher model is controlled to perform expansion processing on the current processing sub-skills in the processing sub-skills list and dynamically adjust the priority of the corresponding processing tasks based on the skill performance results corresponding to the processing sub-skills list, so as to formulate the data generation plan.

[0045] Here, the control teacher model performs expansion processing on the current processing sub-skills in the processing sub-skills list and dynamically adjusts the priority of the corresponding processing tasks according to the skill performance results corresponding to the processing sub-skills list, and formulates the data generation plan.

[0046] Among them, if the skill performance result is poor, the priority of the processing task will be reduced, and the generation ratio of training data for the processing task will be adjusted.

[0047] In a possible implementation manner, controlling the teacher model to perform expansion processing on the current processing sub-skill in the processing sub-skill list includes: The sample training data of the processing skill is input into the teacher model, the sample training data is expanded into sub-skills, and new sub-skills and categories of new sub-skills are determined.

[0048] Here, new sub-skills are generated based on the input data of the processed skills through the large language network layer of the teacher model and classified into mutually exclusive categories.

[0049] In a possible implementation manner, after inputting the sample training data of the processing skill into the teacher model, performing sub-skill expansion on the sample training data, and determining new sub-skills and new sub-skill categories, the training data generation method further includes: Detect whether the subdivision degree of the processing sub-skills of the processing skill meets a preset threshold; if so, stop expanding the new processing sub-skills; if not, continue expanding the new processing sub-skills.

[0050] For further information, see Figure 2 , Figure 2 This is a schematic diagram of the expansion process of the processing skill structure tree provided in the embodiment of the present application. Figure 2As shown in the figure, the update of the processing skill structure tree of the skill "Algebra" in the math task in the processing skill structure tree test environment over time is taken as an example. Starting from an initial single node, the data generation strategy iteratively selects operations between exploration (expanding the processing skill structure tree) and development (rebalancing the processing skill structure tree). Then, sub-skills are dynamically added / removed as needed, and training data is reallocated for each sub-skill. Specifically, at time point t=0: the processing skill structure tree has only one root node "Algebra" in the initial state, indicating that the system only recognizes one high-level skill. The data allocation starts empty, and the processing skill structure tree is not assigned to a specific sub-skill. At time point t=1: two sub-skills "Linear Equations" and "Polynomial Decomposition" are added to the "Algebra" skill through the "Exploration" operation. The training data allocation for each sub-skill is 10, corresponding to the increment "(+10)" in the brackets, indicating that 10 data samples are added for training. At time point t=2: Explore again, and continue to add a new sub-skill for the "Algebra" skill: "Fractions". At this point, "Linear Equation" and "Polynomial Decomposition" still maintain the previous data allocation, while the newly added "Fractions" sub-skill obtains 15 training data, showing that the system further explores breadth. At time point t=3: the data allocation of existing sub-skills is rebalanced, and the amount of data for Linear Equations increases to 30 (+20), indicating that this sub-skill is more important to improving model performance, and Polynomial Decomposition maintains a data allocation of 10. The data allocation for the "Fractions" sub-skill is reduced to 0, indicating that the current training benefit of this skill is low.

[0051] In this application, a data generation plan is formulated based on the performance information of the target model. For example: Targeted data generation: The teacher model generates more targeted training data based on the feedback error samples and weak distribution to strengthen the weak skills of the target model. Optimize data ratio: If the performance of a skill or task is weak, targeted optimization can be performed by increasing the amount of related data generated. Dynamically adjust task priorities: Adjust the generation ratio of tasks according to the importance and current performance of different skills. For example, prioritize the generation of skill data that has the greatest impact on improving overall performance. Balance between exploration and development: Through feedback, decide whether to invest more resources in optimizing existing skills or expand into new skill areas.

[0052] S103: Train the target model based on the training data to determine the updated target model, continue to determine the updated training data of the target model, train the target model according to the new training data, and stop training the target model when the performance feedback status meets the preset requirements.

[0053] In this step, the target model is trained according to the training data to determine the updated target model, and the training data of the updated target model is further determined. The target model is trained according to the new training data until the performance feedback status meets the preset requirements and the training of the target model is stopped.

[0054] For further information, see Figure 3 , Figure 3 A schematic diagram of a training data generation method based on dynamic feedback provided in an embodiment of the present application. Figure 3 As shown, for the target model m t Evaluate the current performance of s t =Eval( m t ) to get the target model performance feedback s t , and then provide the performance status of the model to the data generation agent module s t , the data generation agent module formulates the data generation strategy π according to the feedback st , specifically expressed as the conditional distribution π( p t | s t ) in the sample generation data generation plan p t . Sampling formula p t ～ π( p t | s t ) indicates the data generation plan p t Based on the current status p t The result of the probability distribution generated by the engine function Engine Will p t Converted into training data d t , the training data generated d t Returned to the target model for training by the training function Train Update the target model to convert the current model m t Updated to better performance m t+1 The updated target model is then re-evaluated to generate the next round of states s t+1 .

[0055] In this application, the open training environment, processing skill training environment, and processing skill structure tree training environment are gradually increased with structural support, from a completely open environment to a processing skill training environment based on skill classification, and then to a hierarchical processing skill structure tree environment to help agents optimize the data generation process to varying degrees. At the same time, the skill-driven design shows higher pertinence and interpretability when solving specific problems of the model.

[0056] The embodiment of the present application provides a method for generating training data based on dynamic feedback, the method comprising: performing performance evaluation processing on the target model in the current environment, and determining the performance feedback status of the target model in the current training environment; wherein the training environment comprises an open training environment, a processing skill training environment, and a processing skill structure tree training environment; controlling the teacher model to formulate a data generation plan based on the performance feedback status, and converting the data generation plan based on the data generation engine to generate training data for the target model; training the target model based on the training data, determining the updated target model, continuing to determine the updated training data of the target model, and training the target model according to the new training data until the performance feedback status meets the preset requirements, and stopping the training of the target model. By dynamically adjusting the generation strategy through the feedback mechanism, the generated training data is more accurate, and the model training effect is significantly improved, avoiding the problems of low data generation efficiency, high cost, uneven data quality, lack of dynamic adaptability and versatility in the prior art.

[0057] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of a training data generation device based on dynamic feedback provided in an embodiment of the present application. Figure 4 As shown in , the training data generating device 400 includes: The model evaluation module 410 is used to perform performance evaluation processing on the target model in the current environment to determine the performance feedback status of the target model in the current training environment; wherein the training environment includes an open training environment, a processing skill training environment, and a processing skill structure tree training environment; A data generation agent module 420 is used to control the teacher model to formulate a data generation plan based on the performance feedback state, and to convert the data generation plan based on the data generation engine to generate training data for the target model; The model training module 430 is used to train the target model based on the training data, determine the updated target model, continue to determine the updated training data of the target model, train the target model according to the new training data, and stop training the target model when the performance feedback status meets the preset requirements.

[0058] Furthermore, when the data generation agent module 420 is used for the open training environment and the control teacher model formulates a data generation plan based on the performance feedback state, the data generation agent module 420 is specifically used to: The teacher model determines the probability distribution of error samples under each data type based on the error sample list in the performance feedback state; Based on the probability distribution of error samples under each data type, significant error data types in the error sample list are determined, and a data generation plan is formulated according to the significant error data types.

[0059] Furthermore, when the data generation agent module 420 is used for the processing skill training environment and the control teacher model formulates a data generation plan based on the performance feedback state, the data generation agent module 420 is specifically used to: Based on the list of processing skills in the performance feedback state and the list of performance errors at the processing skill level, the teacher model determines the weak processing skills corresponding to the target model and formulates a data generation plan for the weak processing skills; wherein the processing skills are the skills of the target model for processing tasks.

[0060] Further, when the data generation agent module 420 is used for the processing skill structure tree training environment and the control teacher model formulates a data generation plan based on the performance feedback state, the data generation agent module 420 is specifically used to: Based on the list of processing sub-skills in the performance feedback state and the list of performance errors at the processing sub-skill level, the teacher model determines the weak processing sub-skills corresponding to the target model, and formulates a data generation plan for the weak processing sub-skills; wherein the processing skill structure tree is constructed with the processing skill as the root node and the processing sub-skill as the leaf node, and the processing sub-skill is the hierarchical skill of the processing skill.

[0061] Further, when the data generation agent module 420 is used for the processing skill structure tree training environment and the control teacher model formulates a data generation plan based on the performance feedback state, the data generation agent module 420 is specifically used to: The teacher model is controlled to perform expansion processing on the current processing sub-skills in the processing sub-skills list and dynamically adjust the priority of the corresponding processing tasks based on the skill performance results corresponding to the processing sub-skills list, so as to formulate the data generation plan.

[0062] Further, when the data generation agent module 420 is used to control the teacher model to perform expansion processing on the current processing sub-skill in the processing sub-skill list, the data generation agent module 420 is specifically used to: The sample training data of the processing skill is input into the teacher model, the sample training data is expanded into sub-skills, and new sub-skills and categories of new sub-skills are determined.

[0063] Furthermore, the data generation agent module 420 is also used for: Detecting whether the subdivision degree of the processing sub-skills of the processing skill meets a preset threshold; If so, stop expanding the new processing sub-skills; If not, continue to expand on new processing sub-skills.

[0064] The embodiment of the present application provides a training data generation device based on dynamic feedback, the training data generation device includes: a model evaluation module, which is used to perform performance evaluation processing on the target model in the current environment, and determine the performance feedback status of the target model in the current training environment; wherein the training environment includes an open training environment, a processing skill training environment, and a processing skill structure tree training environment; a data generation agent module, which is used to control the teacher model to formulate a data generation plan based on the performance feedback status, convert the data generation plan based on the data generation engine, and generate training data for the target model; a model training module, which is used to train the target model based on the training data, determine the updated target model, continue to determine the updated training data of the target model, train the target model according to the new training data, and stop training the target model until the performance feedback status meets the preset requirements. By dynamically adjusting the generation strategy through the feedback mechanism, the generated training data is more accurate, the model training effect is significantly improved, and the problems of low data generation efficiency, high cost, uneven data quality, lack of dynamic adaptability and versatility in the prior art are avoided.

[0065] See also Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown in , the electronic device 500 includes a processor 510 , a memory 520 and a bus 530 .

[0066] The memory 520 stores machine-readable instructions executable by the processor 510. When the electronic device 500 is running, the processor 510 communicates with the memory 520 via the bus 530. When the machine-readable instructions are executed by the processor 510, the above-mentioned Figure 1 as well as Figure 3 The specific implementation of the steps of the method for generating training data based on dynamic feedback in the method embodiment shown can be found in the method embodiment, which will not be described in detail here.

[0067] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 as well as Figure 3 The specific implementation of the steps of the method for generating training data based on dynamic feedback in the method embodiment shown can be found in the method embodiment, which will not be described in detail here.

[0068] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0069] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0070] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0071] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0072] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0073] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A training data generation method based on dynamic feedback, characterized in that: The training data generation method comprises: Performing performance evaluation processing on the target model in the current environment to determine the performance feedback state of the target model in the current training environment; wherein the training environment includes an open training environment, a processing skill training environment, and a processing skill structure tree training environment; Controlling the teacher model to formulate a data generation plan based on the performance feedback state, and converting the data generation plan based on the data generation engine to generate training data for the target model; The target model is trained based on the training data to determine the updated target model, the training data of the updated target model is continued to be determined, and the target model is trained according to the new training data until the performance feedback state meets the preset requirements and the training of the target model is stopped.

2. The training data generation method according to claim 1, characterized in that: For the open training environment, the control teacher model formulates a data generation plan based on the performance feedback state, including: The teacher model determines the probability distribution of error samples under each data type based on the error sample list in the performance feedback state; Based on the probability distribution of error samples under each data type, significant error data types in the error sample list are determined, and a data generation plan is formulated according to the significant error data types.

3. The training data generation method according to claim 1, characterized in that: For the processing skill training environment, the control teacher model formulates a data generation plan based on the performance feedback state, including: Based on the list of processing skills in the performance feedback state and the list of performance errors at the processing skill level, the teacher model determines the weak processing skills corresponding to the target model and formulates a data generation plan for the weak processing skills; wherein the processing skills are the skills of the target model for processing tasks.

4. The training data generation method according to claim 1, characterized in that: For the processing skill structure tree training environment, the control teacher model formulates a data generation plan based on the performance feedback state, including: Based on the list of processing sub-skills in the performance feedback state and the list of performance errors at the processing sub-skill level, the teacher model determines the weak processing sub-skills corresponding to the target model and formulates a data generation plan for the weak processing sub-skills; wherein the processing skill structure tree is constructed with the processing skill as the root node and the processing sub-skill as the leaf node, and the processing sub-skill is the hierarchical skill of the processing skill.

5. The training data generation method according to claim 4, characterized in that: For the processing skill structure tree training environment, the control teacher model formulates a data generation plan based on the performance feedback state, including: The teacher model is controlled to perform expansion processing on the current processing sub-skills in the processing sub-skills list and dynamically adjust the priority of the corresponding processing tasks based on the skill performance results corresponding to the processing sub-skills list, so as to formulate the data generation plan.

6. The training data generation method according to claim 5, characterized in that: The controlling the teacher model to perform expansion processing on the current processing sub-skill in the processing sub-skill list includes: The sample training data of the processing skill is input into the teacher model, the sample training data is expanded into sub-skills, and new sub-skills and categories of new sub-skills are determined.

7. The training data generation method according to claim 6, characterized in that: After inputting the sample training data of the processing skill into the teacher model, expanding the sample training data into sub-skills, and determining new sub-skills and new sub-skill categories, the training data generation method further includes: Detecting whether the subdivision degree of the processing sub-skills of the processing skill meets a preset threshold; If so, stop expanding the new processing sub-skills; If not, continue to expand on new processing sub-skills.

8. A training data generation device based on dynamic feedback, characterized in that: The training data generating device comprises: A model evaluation module is used to perform performance evaluation processing on the target model in the current environment to determine the performance feedback status of the target model in the current training environment; wherein the training environment includes an open training environment, a processing skill training environment, and a processing skill structure tree training environment; A data generation agent module is used to control the teacher model to formulate a data generation plan based on the performance feedback state, and to convert the data generation plan based on the data generation engine to generate training data for the target model; A model training module is used to train the target model based on the training data, determine the updated target model, continue to determine the training data of the updated target model, train the target model according to the new training data, and stop training the target model when the performance feedback status meets the preset requirements.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the training data generation method based on dynamic feedback as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for generating training data based on dynamic feedback according to any one of claims 1 to 7 are executed.

Images