Training sample data generation method, device, readable storage medium and program product

Abstract

The present invention discloses a method, device, readable storage medium and program product for generating training sample data, and relates to the field of artificial intelligence technology. The method comprises inputting a seed question and a prompt word of a programming language processing task into a sample data generation model; using the prompt word to modify the seed question within the modification range boundary conditions based on the same algorithmic thinking, and generating a test case set covering the new question constraint conditions according to the new question type and use case generation conditions. New questions that have passed verification are answered, and answers verified by the test case set are output; the new question, the test case set and the answer are used as training sample data; the modification range boundary conditions and / or the prompt word and / or the use case generation conditions are adjusted, and new training sample data is generated using the new question. The present invention can solve the problem that the training data generated by the related technology cannot meet the model training requirements, and can efficiently generate high-quality training sample data for programming language processing tasks.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method for generating training sample data, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] NLP (Natural Language Processing) helps computers understand and process human language, enabling them to read and comprehend natural language. With the rapid development of artificial intelligence (AI), the accuracy requirements for language tasks performed using language models are increasing, and so is the demand for high-quality model training data. Related technologies use multi-agent automated code generation or comprehension to generate training data for these programming language processing tasks. However, the quality of these training data is poor, and the difficulty of the questions is poorly controlled, making it difficult to meet the training requirements of programming language processing models. Summary of the Invention

[0003] The present invention provides a training sample data generation method, electronic equipment, computer-readable storage medium and computer program product, which can efficiently generate high-quality training sample data for programming language processing tasks.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0005] In one aspect, the present invention provides a method for generating training sample data, comprising:

[0006] Input the seed questions and prompt words of the programming language processing task into the sample data generation model built based on the language model;

[0007] Using prompt words to generate a model from sample data, the seed question is modified within the modification range boundary conditions based on the same algorithmic thinking. A test case set that covers the constraints of the new question is generated based on the new question type and case generation conditions. The new question and test case set are verified, and the verified test case set is output. The new question that has passed the verification is solved, and the answer verified by the test case set is output. The new question, test case set, and answer are used as the first set of training sample data.

[0008] The new question is input into the sample data generation model, and the modification range boundary conditions and / or prompt words and / or use case generation conditions are adjusted to generate a second set of training sample data.

[0009] The present invention also provides an electronic device comprising a memory and a processor, wherein the processor is configured to implement the steps of any of the above-mentioned training sample data generation methods when executing a computer program stored in the memory.

[0010] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of any of the above-mentioned training sample data generation methods are implemented.

[0011] Finally, the present invention further provides a computer program product, comprising a computer program / instruction, which implements the steps of any of the above-mentioned training sample data generation methods when executed by a processor.

[0012] The advantage of the technical solution provided by the present invention is that it uses a language model to construct a sample data generation model that can automatically generate the training sample data required for programming processing tasks, thereby realizing the automation and intelligence of code-type training sample data generation, eliminating the need for manual labeling, reducing manual intervention, reducing labor costs, and effectively improving the efficiency of code-type training sample data generation. The newly generated questions are controlled to have the same algorithmic thinking as the original questions and are within the boundary conditions of the allowable modification range. This not only controls the difficulty of the newly generated questions, but also ensures the accuracy of the new questions, which is conducive to comprehensively covering various programming task scenarios and algorithm requirements, enriching the training data content of programming processing tasks, and improving the model training performance of executing programming processing tasks. Furthermore, by reviewing new questions and test cases, the reliability and accuracy of the generated data are effectively guaranteed. After the new questions are reviewed and approved, they are answered and the generated answers are verified to ensure the correctness and practicality of the answers and questions. Iterative evolution of the same question can generate training sample data of varying complexity, meeting the requirements for diverse data in different training phases and application scenarios for the language model corresponding to the programming language processing task. Structured training data at multiple levels of difficulty evolution, including question-test case-answer triples, can support supervised learning in the instruction fine-tuning phase and meet the reward calculation requirements in the reinforcement learning phase, enhancing the adaptability and generalization capabilities of the language model corresponding to the programming language processing task. Furthermore, the present invention also provides corresponding electronic devices, computer-readable storage media, and computer program products for the training sample data generation method, further making the method more practical. The electronic devices, computer-readable storage media, and computer program products have corresponding advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0014] Figure 1A schematic diagram of the hardware composition framework applicable to the training sample data generation method provided by the present invention;

[0015] Figure 2 A flow chart of a method for generating training sample data provided by the present invention;

[0016] Figure 3 A schematic diagram of a question evolution process provided by the present invention;

[0017] Figure 4 A schematic diagram of a test case generation process provided by the present invention;

[0018] Figure 5 A schematic diagram of a question review process provided by the present invention;

[0019] Figure 6 A schematic diagram of an answer generation process provided by the present invention;

[0020] Figure 7 A flow chart of another method for generating training sample data provided by the present invention;

[0021] Figure 8 This is a structural framework diagram of an exemplary embodiment of the training sample data generation device provided by the present invention. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. The terms "first," "second," "third," "fourth," etc. in the specification and the accompanying drawings are used to distinguish different objects rather than to describe a specific order. Furthermore, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. The term "exemplary" means "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as superior or better than other embodiments.

[0023] With the rapid development of artificial intelligence technology, the accuracy requirements for related language tasks performed using language models are becoming increasingly higher. For example, when using LLM (Large Language Model) to perform programming language processing tasks, such as code generation tasks and code comprehension tasks, the demand for high-quality model training sample data is also increasing.

[0024] Traditionally, training sample data is generated through manual annotation or screening from open-source code repositories. Manual annotation is not only costly but also difficult to ensure sample data diversity. Data obtained from open-source code repositories often suffer from uneven quality, inconsistent coding styles, and a lack of systematic difficulty grading. Furthermore, the proportion of "difficult problems" in open-source datasets is insufficient, leading to rapid model saturation due to simple problems, while difficult problems are few and far between, making it impossible to train high-performance models. To address the shortcomings of traditional training sample data generation, automatic generation is currently employed. However, this approach can only generate training sample data for simple scenarios. For programming problems that require deep reasoning and global understanding, such as programming competitions, the training sample data generated by this approach often suffers from quality issues such as logical errors and inconsistencies. For example, related technologies generate training data through code completion models. However, due to the lack of rigorous verification of the generated code, the data contains numerous logical errors or inefficient implementations, limiting the effectiveness of code-generated large-scale language model training. The related art Magicoder (large language model) uses OSS-INSTRUCT (a data generation method for generating low-bias and high-quality code from open source code snippets using large language models) to generate instruction data using LLMs. However, this relies on the quality and diversity of the open source code, making it difficult to generate high-quality optimization instructions. Furthermore, model training sample data generation methods typically use a single model for one-time generation, lacking a systematic problem evolution mechanism. This makes it difficult to cover the full difficulty spectrum from basic to advanced, and thus fails to meet the demand for high-quality training sample data.

[0025] Related technologies use multi-agents to automatically generate or understand code, generating training data for programming language processing tasks. For example, MetaGPT (an open-source framework for multi-agent collaboration) optimizes software development processes through multi-agent collaboration. However, it focuses on task decomposition rather than the generation and verification of code training data. It also lacks specialized design for key aspects such as the difficulty evolution of code questions, test case coverage, and answer verification. While AGENTVERSE (a multi-agent framework) utilizes multi-agent collaboration for processing, it targets general tasks rather than the structured generation of code data. CMAT (a multi-agent collaborative tuning framework) improves the collaborative capabilities of small language models through supervised fine-tuning, but it lacks dynamic evolution of code questions or test case generation and verification. Furthermore, multi-agent systems still have limitations in the domain of programming language processing tasks. For example, while a related technology optimizes communication efficiency between heterogeneous agents through information navigation, it addresses information asymmetry and cannot achieve iterative optimization of code data. During the test case generation process, the focus is often limited to the generation of the code itself, relying on simple input-output matching or random test data to generate test cases. This lack of in-depth analysis of boundary conditions, abnormal inputs, and algorithm coverage leads to a lack of systematic test case construction, resulting in the generated test cases being unable to effectively verify the robustness of the code, thus affecting the training effectiveness of the reinforcement learning phase. Furthermore, related technologies are often unable to simultaneously meet the different requirements of instruction fine-tuning and post-reinforcement learning training. The generated training sample data is single-purpose and difficult to adapt to the requirements of the different training stages of large-scale language models.

[0026] This demonstrates that related techniques struggle to systematically generate programming problems with a gradient of difficulty, resulting in insufficient training data to meet the algorithmic thinking training requirements of large-scale language models. Due to the lack of an effective difficulty assessment mechanism, the complexity of problems cannot be adjusted through iterative evolution. Consequently, simple problems generated fail to meet the requirements of advanced algorithmic thinking, such as dynamic programming and graph optimization. Non-simple problems are obscure and lack reasonable solution paths. Furthermore, related techniques ignore boundary conditions, extreme inputs, and algorithm coverage analysis when generating test cases for training sample data. This results in test cases being unable to effectively verify the robustness of the code, impacting the effectiveness of training during the reinforcement learning phase. Furthermore, related techniques struggle to perform multiple rounds of optimization on the same problem, unable to generate multiple variations of the same problem, such as dynamic programming problems under different constraints. Consequently, they are unable to replicate the characteristics of tasks with a clear progression in difficulty, such as programming competition problems. Consequently, the training sample data fails to meet the model's requirements for algorithmic thinking training, and thus cannot ensure the production of high-performance models capable of handling these programming language processing tasks.

[0027] In view of this, in order to be able to efficiently generate high-quality training sample data for programming language processing tasks, and at the same time meet the requirements of different training stages for training sample data such as instruction fine-tuning and reinforcement learning needs, the present invention performs multiple rounds of optimization on the same question, and forms a complete difficulty spectrum by generating multiple variants of the same question. It can effectively enrich the question data of programming language processing tasks that require deep reasoning and global understanding, such as competition-level programming, and adjust the complexity of the questions through iterative evolution to effectively control the quality and difficulty of the questions. It can not only meet the needs of high-level algorithmic thinking, but also have a reasonable problem-solving path. It not only improves the quality and diversity of training sample data, but also divides the data into "question-answer" pairs (for fine-tuning) and "question-test cases" (for reinforcement learning) through structured storage, meeting the needs of different training stages and solving the problem of the single use of training sample data. In combination with the specific application environment architecture or specific hardware architecture on which the execution of the training sample data generation method depends, the specific application environment architecture or specific hardware architecture is described here. The following is combined with Figure 1 Some possible application scenarios involved in the technical solution of the present invention are introduced by way of example, which may include the following:

[0028] The hardware framework may include a first electronic device 11 and a second electronic device 12, which are connected via a network 13. The first electronic device 11 has an interface for invoking multiple language models, such as an API (Application Programming Interface), and is equipped with a processor for executing the training sample data generation method described in any of the above embodiments. The second electronic device 12 is equipped with a user terminal for providing a human-computer interaction interface. The user enters a seed question and a prompt word through the second electronic device 12. The first electronic device 11 obtains the seed question and prompt word through the network 13, completes all or part of the steps for generating training sample data described in the above embodiments, and obtains multiple sets of training sample data corresponding to the seed question with a complete difficulty spectrum. The multiple sets of training sample data are then sent to the second electronic device 12, which stores the data in the form of "question-test case-answer" triples. After completing the generation of triple training sample data for a seed question, the second electronic device 12 can send the next seed question to the first electronic device 11. Of course, the second electronic device 12 can send the seed question library to the first electronic device 11. In order to improve efficiency and save computing resources and storage resources, the storage address of the seed question library can also be sent to the first electronic device 11. After the first electronic device 11 obtains the seed question library, in order to improve the efficiency of training sample data generation, if resources permit, it can construct multiple threads to simultaneously execute corresponding training sample data generation tasks for multiple seed questions. Similarly, the storage address of the training sample data of each seed question can also be sent to the second electronic device 12.

[0029] It should be noted that the above application scenarios are only provided to facilitate understanding of the concepts and principles of the present invention, and the embodiments of the present invention are not limited in this respect. On the contrary, the embodiments of the present invention can be applied to any applicable scenario.

[0030] After introducing the technical solution of the present invention, various non-limiting embodiments of the present invention are described in detail below with reference to the accompanying drawings and specific embodiments. Figure 2 , Figure 2 This is a flow chart of a method for generating training sample data provided in this embodiment. This embodiment may include the following contents:

[0031] S201: Inputting the seed title and prompt words of the programming language processing task into a sample data generation model constructed based on the language model.

[0032] In this step, the programming language processing task is, for example, a code generation task, a code comprehension task, a programming question generation-answering task, and an artificial intelligence programming education task. Of course, the training sample data generation method provided by the present invention is applicable to the generation of code training sample data, and can also be extended to the generation of training data in other fields, such as question-and-answer data for natural language processing, annotated data for image recognition, etc. The present invention does not impose any restrictions on this. The seed question is the original question of the programming language processing task. The question can be written manually or directly obtained from an existing database. This does not affect the implementation of the present invention. The seed question is the question corresponding to the programming language processing task, which can be text data, image data, audio and video data. Accordingly, the sample data generation model has a coding and decoding processing mechanism in a corresponding format. The prompt word is an instruction to prompt the sample data generation model on how to process the seed question. Similarly, it can be text data, image data, audio and video data. Those skilled in the art can choose any data format according to the actual scenario.

[0033] Among them, the sample data generation model can adopt a language model or a combination of multiple language models, which does not affect the implementation of the present invention. The language model can directly adopt any one of the related technologies that can help computers understand and process human language, so that computers can read and understand natural language and perform various language tasks. Those skilled in the art can directly adopt existing models that can conduct dialogues and can perform tasks with certain logical capabilities, such as intelligent chat robot models. Of course, pre-trained language models can also be used, and then fine-tuned to achieve the purpose of solving the training sample data generation. Pre-trained language models refer to designing language model training tasks based on large-scale corpora (including language training materials such as sentences and paragraphs), training large-scale neural network algorithm structures to learn and implement, and the large-scale neural network algorithm structure and parameters finally obtained are pre-trained language models. Subsequent tasks can be based on this model to perform feature extraction or task fine-tuning to achieve specific task purposes. The neural network algorithm structure for training pre-trained language models can be a convolutional neural network model, a long short-term memory network model, etc., or a model built on an attention network, such as Transformer (converter network model), BERT (Bidirectional Encoder Representations from Transformers, based on Transformer's bidirectional encoder representation), GPT (Generative Pre-trained Transformer), etc.

[0034] S202: Generate a model using sample data using prompt words, modify the seed question within the modification range boundary conditions based on the same algorithmic thinking, and generate a test case set that covers the constraints of the new question based on the new question type and case generation conditions; verify the new question and test case set, output the verified test case set, answer the verified new question, and output the answer verified by the test case set.

[0035] The same algorithmic thinking refers to the fact that the algorithm for the newly generated question, i.e., the new question in this embodiment, belongs to the same type of algorithm as the seed question, or is an algorithm that solves the same problem. For example, if the seed question uses a linear search method to search for a specific element, and its test point is the search algorithm, then the algorithm for the new question can be any search algorithm that can search for a specific element, such as a linear search method or a binary search. The modification range boundary condition is used to control the difficulty range of the newly generated question. It can control the complexity of the algorithm, the scale of the input data, and the type of data structure. In other words, the algorithm complexity and / or the scale of the input data and / or the type of data structure can be limited to control the modification range of the new question to ensure that the generated new question is both sufficiently challenging and within a reasonable difficulty range. For example, to control the difficulty of the new algorithm by algorithm complexity, the modification range boundary condition can be set to an algorithm difficulty level of no more than 2 levels. If the seed question uses a basic algorithm such as simple search, then the new algorithm can use an intermediate algorithm, such as binary search, or an advanced algorithm, such as a mathematical optimization method. However, combinations of advanced algorithms and advanced algorithms exceed this boundary and are not allowed. When modifying seed questions within the boundaries of the modified scope based on the same algorithmic thinking, the core algorithmic thinking must remain unchanged, while at the same time increasing the challenge of the questions through innovative question types. For example, for a traditional sorting algorithm question, the scenario can be expanded from simple numerical sorting to sorting objects with complex properties, thereby increasing the difficulty and novelty of the question.

[0036] After modifying a seed problem within the modified scope based on the same algorithmic thinking, a new problem is generated. The new problem type is determined based on the new problem's test focus and all constraints. For example, the new problem is to find a target value in a rotated sorted array. The test focus is identified as a variant application of the binary search algorithm, and the constraints include array element uniqueness, rotation point uniqueness, and time complexity requirements. The use case generation criteria include at least the use case composition type, which refers to the type of test case, such as base case, stress case, and edge case. The test case design plan is determined based on the new problem's test focus and all constraints. This test case design plan includes the planned test dimensions, the specific use case design ideas for each dimension, and the expected common error types to be discovered, thereby ensuring that all constraints of the new problem are covered. For example, for a new dynamic programming problem, the test case design plan may include: state transition correctness, boundary condition handling, and time efficiency verification. Each dimension may include at least the construction of special state transition paths, minimum / maximum input size, and invalid input detection. Expected error types include common problems such as state initialization errors, transition equation errors, and array out-of-bounds errors. The test case set is a collection of multiple test cases generated according to the solution of this step.

[0037] In this step, after generating new questions and test case sets, in order to ensure the model training performance and meet the training requirements of the programming language processing task, the new questions and test case sets need to be comprehensively evaluated, such as whether the questions are solvable, whether the difficulty is increased, whether error information or inconsistent information is introduced, and whether the test cases are appropriate, and the reasons are pointed out in detail. The quality of the generated new questions and test cases is ensured by reviewing and verifying the new questions and test cases. For example, when a test case in the test case set cannot cover all possible situations in the question, an inappropriate test case will be generated and an explanation of the omission will be given. For new questions or test case sets that do not pass the verification, the prompt words will be adjusted or the range boundary conditions or the case generation conditions will be modified to regenerate new questions and new test case sets until the verification passes. Of course, in order to avoid entering an infinite loop and ensure efficiency, a maximum number of regenerations can be set. When this limit is exceeded, a corresponding error prompt message can be generated. The new questions that pass the verification are answered, and the generated data of the new questions are the answers. Taking the code generation task as an example, the answer is a piece of code data generated. Taking the code understanding task as an example, the answer is the understanding content data that explains the given code. Taking the competition question setting task as an example, the answer is the answer to the competition question and the corresponding question. The answer can be text data, image data, audio and video data, etc., which does not affect the implementation of the present invention. In order to ensure the accuracy of the final answer, after the answer is generated, the generated answer will also be verified, such as whether the answer is accurate, whether it exceeds the constraints of the new question, and the answer needs to pass all or most test cases, such as 90%. Similarly, for questions that do not pass the verification, the prompt words will be adjusted or the range boundary conditions or the use case generation conditions will be modified to regenerate new questions until the verification is passed. Of course, in order to avoid entering an endless loop and ensure efficiency, a maximum number of regenerations can be set. When this limit is exceeded, a corresponding wrong prompt message can be generated. Finally, a correspondence is established between the generated new questions, test case sets, and answers as a set of training sample data, which is then stored in a structured manner, that is, data is stored according to the "question-test case-answer" triplet. This can support supervised learning in the instruction fine-tuning stage and meet the reward calculation requirements in the reinforcement learning stage. For example, "question-answer" can be applied to fine-tuning training needs, and "question-test case" can be used for reinforcement learning.

[0038] S203: Input the new topic into the sample data generation model, adjust the modification range boundary conditions and / or prompt words and / or use case generation conditions, and generate a second set of training sample data.

[0039] In this step, the modification range boundary conditions and / or prompt words and / or use case generation conditions can be adjusted according to actual needs. The new questions are used as seed questions in S201. If the prompt words are adjusted, the new prompt words and the new questions of the first set of training data are input as seed questions into the sample data generation model. When executing S202, if the modification range boundary conditions or the use case generation conditions are adjusted, the new modification range boundary conditions or the new use case generation conditions are used to replace the old modification range boundary conditions or the old use case generation conditions of S202, and the corresponding training sample data are generated according to the method of S202. For ease of distinction, the new questions, test case sets and answers generated in the above steps are used as the first set of training sample data, and the training sample data generated in this step is defined as the second set of training sample data. In order to improve the quality and diversity of the generated training sample data, this step will perform multiple rounds of iterative evolution on the same question. Those skilled in the art can determine the number of iterative evolution rounds according to actual needs. This embodiment takes one iteration as an example. If it is iterated twice, the new question of the second set of training sample data generated in S203 is used as the seed question of S201 to execute S201-S202, and the cycle continues until the number of iterative evolution rounds is reached.

[0040] In the technical solution provided in this embodiment, a language model is used to construct a sample data generation model that can automatically generate the training sample data required for programming processing tasks, thereby realizing the automation and intelligence of code-type training sample data generation, eliminating the need for manual labeling, reducing manual intervention, and lowering labor costs, effectively improving the efficiency of code-type training sample data generation. The newly generated questions are controlled to have the same algorithmic thinking as the original questions and are within the boundary conditions of the allowable modification range. This not only controls the difficulty of the newly generated questions, but also ensures the accuracy of the new questions, which is conducive to comprehensively covering various programming task scenarios and algorithm requirements, enriching the training data content of programming processing tasks, and improving the model training performance of executing programming processing tasks. Furthermore, by reviewing new questions and test cases, the reliability and accuracy of the generated data are effectively guaranteed. After the new questions are reviewed and approved, they are answered and the generated answers are verified to ensure the correctness and practicality of the answers and questions. Iterative evolution of the same question can generate training sample data of different complexities, meeting the needs of the language model corresponding to the programming language processing task for diversified data in different training stages and application scenarios. The structured training data with multiple difficulty evolution levels contains the "question-test case-answer" triple, which can not only support supervised learning in the instruction fine-tuning stage, but also meet the reward calculation needs in the reinforcement learning stage, thereby enhancing the adaptability and generalization ability of the language model corresponding to the programming language processing task.

[0041] In the above embodiment, there is no limitation on how to modify the seed topic within the modification range boundary conditions based on the same algorithmic thinking. This embodiment also provides multiple exemplary modification methods for the seed topic, which may include the following:

[0042] For example, the algorithm type and / or data structure type of the seed question can be determined by analyzing the test points of the seed question; based on the algorithm type and / or data structure type, multiple modification methods of the seed question can be determined; from the random combination of the seed question and each modification method, the modification method that meets the conditions of having the same test points as the seed question and adding at least one new technical point is selected as the optimal combination modification method; based on the optimal combination modification method, the seed question is modified accordingly to obtain a new question.

[0043] In this embodiment, the seed question is first algorithmically deconstructed. Using prompts, the core test points of the seed question are deeply analyzed, accurately extracting the algorithm types and data structure types involved. For example, if the seed question is about searching for a specific element, it is clear that the question is about the search algorithm, which may be linear search, binary search, etc., as well as the data structures involved, such as arrays and linked lists. Possible problem evolution directions are detailed based on the question type: For algorithmic questions, modifications can be made based on the algorithm's complexity, the size of the input data, and the type of data; for data structure questions, modifications can be made based on the organization of the data structure and the storage method of the elements. Once multiple modification methods are determined, they can be randomly combined, and the optimal modification combination can be selected from the numerous modification directions based on the modification range boundary conditions. In this embodiment, the modification range boundary conditions are modification methods that meet the conditions of sharing the same test points as the seed question and adding at least one new technical point. Finally, the final new question is generated based on the determined optimal combination of modification methods. For example, based on the previous analysis, a question is generated that requires using a specific algorithm to find nodes that meet complex conditions in a large-scale graph data structure. There are multiple optimal combination modification methods. Based on each optimal combination modification method, the seed topic can be modified accordingly to obtain multiple initial new topics; candidate new topics with no errors or omissions are selected from each initial new topic, and the algorithm type and / or data structure complexity of the candidate new topic and the seed topic are compared, and the candidate new topic with higher complexity than the seed topic is selected as the new topic.

[0044] For example, the present invention can also simulate the human question-setting process through prompt word engineering, and design reasonable prompt words in combination with language models to achieve the evolution of the original seed questions, such as Figure 3As shown, a high-performance large language model (such as DeepSeek V3) can be selected as the question-use case generation submodel. Specifically, the sample data generation model includes at least the question-use case generation submodel, which also serves as the role allocation. The question-use case generation submodel modifies the seed question through four steps: algorithm deconstruction, strategy transformation, strategy combination, and question generation. During the modification process, the core algorithmic thinking must be maintained while enhancing the challenge of the question through innovative question types. For example, if the seed question is a traditional sorting algorithm question, the scenario can be expanded from simple numerical sorting to sorting objects with complex attributes, thereby increasing the difficulty and novelty of the question. For example, during the algorithm deconstruction step, the question-use case generation submodel is prompted to deeply analyze the core test points of the original question and accurately extract the algorithm / data structure types involved. During the strategy transformation step, the question-use case generation submodel is prompted to list possible question modification directions in detail based on the question type. During the strategy combination step, the question-use case generation submodel is prompted to select the optimal modification combination from a variety of evolutionary directions and strictly constrain the modification range boundary conditions to ensure that the generated new question is both sufficiently challenging and within a reasonable difficulty range. For example, when increasing algorithm complexity, it's important to consider the feasibility and rationality of that complexity in real-world programming scenarios. During the problem generation phase, the problem use case generation sub-model is prompted to generate a new problem. For example, based on the previous analysis, a problem might be generated that requires using a specific algorithm to find nodes that meet complex conditions in a large-scale graph data structure.

[0045] In this embodiment, a question generation prompt template can be pre-constructed, and the question generation prompt template at least includes a test point analysis prompt word, an evolution prompt word, an optimal combination modification method prompt word, and a rewrite prompt word; the seed question is input into the question generation prompt template to obtain a question generation prompt word; the seed question and the question generation prompt word are input into a question use case generation sub-model, and the question use case generation sub-model analyzes the test points of the seed question under the prompt of the test point analysis prompt word to determine the algorithm type and / or data structure type of the seed question, and determines multiple modification methods of the seed question based on the algorithm type and / or data structure type under the prompt of the evolution prompt word; under the prompt of the optimal combination modification method prompt word, a modification method that meets the conditions of having the same test points as the seed question and adding at least one new technical point is selected from the random combination of the seed question and each modification method, as the optimal combination modification method; under the prompt of the rewrite prompt word, the seed question is modified accordingly based on the optimal combination modification method to obtain a new question. In the present invention, each prompt word in the prompt word template is data that does not include the content under the exemplary scenario. After the exemplary scenario content is input into the prompt word template, a prompt word template dedicated to the exemplary scenario is generated. In order to facilitate distinction, the template after the exemplary scenario data is input is defined accordingly. In this embodiment, the exemplary scenario data is the seed question of S201, which is input into the question generation prompt template, and the question generation prompt template containing the seed question is defined as the question generation prompt word. At this time, the question generation prompt word also includes the test point analysis prompt word, evolution prompt word, optimal combination modification method prompt word and rewrite prompt word.

[0046] In the process of modifying the seed question within the modification range boundary conditions using the question use case generation sub-model based on the same algorithmic thinking, the question use case generation sub-model can be controlled to output in multiple steps in sequence, or the thinking chain process of all steps can be generated at once. For example, the question generation prompt template can be:

[0047] As an expert in programming language processing tasks, rewrite / evolve a given #instruction# into a more complex version. The given instruction is the seed question.

[0048] Follow the steps below to rewrite a given "#instruction#" into a more complex version. The following conditions must be met: the new instruction has the same algorithmic core as the original instruction, the solution requires creative thinking rather than direct application, and it contains at least one important optimization breakthrough.

[0049] Step 1: Please read the "#instruction#" carefully, analyze the core test points of the original #instruction#, and extract the algorithm / data structure type.

[0050] Step 2: Please list three possible modification strategies for #instruction# based on the core test points of #instruction#.

[0051] Step 3: Based on the listed modification strategies, select the optimal evolutionary combination that meets the requirements of retaining the core test points of the seed questions and adding at least one new technical point.

[0052] Step 4: Please rewrite the original #instructions# according to the optimal combination of evolution strategies.

[0053] Step 5: Read the #rewrite instructions# carefully and identify any parts that don't make sense or are missing. Make sure the #rewrite instructions# is just a more complex version of the #instructions#. Provide only the #final rewrite instructions# without any explanation.

[0054] Please reply strictly according to the following format:

[0055] Step 1#Core test points#:

[0056] Step 2#Evolution Strategy#:

[0057] Step 3#Most Evolved Combination#:

[0058] Step 4#Rewrite instructions#:

[0059] Step 5#Final rewrite instructions#:

[0060] #Command#: {seed command}.

[0061] Replace the seed question into the above question generation prompt template to obtain the question generation prompt word. Input the question generation prompt word into the question use case generation sub-model. The question use case generation sub-model will generate corresponding data according to the format of the above template for output.

[0062] As can be seen from the above, this embodiment, through algorithmic deconstruction, transformation strategies, strategy combination selection, and variation boundary restrictions, generates new questions with diversity and challenge, comprehensively covering various programming scenarios and algorithmic requirements, and effectively enriching the content of training sample data for programming language processing tasks. Furthermore, through the use case generation sub-model for questions, algorithmic deconstruction and transformation of seed questions ensures that new questions enhance the challenge while maintaining the core algorithmic thinking, reducing manual intervention and labor costs, and effectively improving the efficiency of generating code training sample data.

[0063] In the above embodiment, there is no limitation on how to generate a test case set that covers the constraints of the new question based on the new question type and the test case composition type. This embodiment also provides multiple exemplary methods for generating a test case set, which may include the following:

[0064] Exemplarily, the test case composition type and the test case generation specification are obtained, and the test case generation plan is determined according to the algorithm test points, explicit constraints and implicit constraints of the new question; the test case generation plan includes at least the test coverage dimensions, the test ideas for each dimension and the expected error types to be found; according to the test case generation plan, multiple test cases that meet the test case composition type and the test case generation specification are generated.

[0065] Among them, the use case generation specification provides each test case with input and output correspondence and design intent annotation information to ensure that all question constraints are covered. This structured prompt can significantly improve the coverage and effectiveness of the test cases. After the algorithm test points and all constraints are determined, a three-dimensional test plan can be dynamically generated according to the characteristics of the question. According to the test case design plan, generate test cases that meet the use case composition type and use case generation specification requirements. For example, the prompt words include generating test cases for the question, which include at least: 3 basic use cases, 4 boundary use cases and 3 stress test cases. The basic use cases are used to verify the correctness of small-scale inputs, and the boundary use cases include extreme values, empty inputs, and special conditions. The stress test cases are used to verify the efficiency of the largest-scale inputs.

[0066] In order to further improve the practicality and accuracy of the test cases, based on the above embodiment, the test cases can also be standardized and the corresponding data can be formatted and output, such as Figure 4 As shown, accordingly, according to the test case generation scheme, the implementation process of generating multiple test cases that meet the test case composition type and the test case generation specification can be as follows: the prompt words include automatically supplementing the metadata of the test case and outputting it according to the output format, generating multiple test cases that meet the test case composition type and the test case generation specification according to the test case generation scheme, adding the test case type to each test case, and marking the key inspection points, and outputting according to the input parameter list, expected output, function name, basis / boundary / stress and test target description. Taking the binary search problem as an example, the generated test case output can be: {"inputs": [[4, 5, 6, 7, 0, 1, 2], 5], "outputs": [1], "fn_name": "search"}. Exemplarily, the output format can be expressed as:

[0067] {"inputs": [input parameter list],

[0068] "outputs": [expected output],

[0069] "fn_name": "Function name",

[0070] "test_type": "foundation / boundary / pressure",

[0071] "description": "Description of the purpose of the test"}.

[0072] As can be seen from the above, this embodiment not only facilitates automated testing by standardizing test cases, but also ensures that the test data generated by different questions have a consistent interface. By supplementing metadata, it can provide more refined reward signals in the reinforcement learning stage and improve the quality of training sample data.

[0073] In a parallel implementation to the above embodiment, during the test case generation phase, this embodiment may employ a structured prompt engineering guidance question case generation sub-model to generate a comprehensive test case set, which may include the following:

[0074] A use case generation prompt template is pre-built, and a new question is input into the use case generation prompt template to obtain a use case generation prompt word; the new question and the use case generation prompt word are input into the question use case generation sub-model, and the question use case generation sub-model obtains the test case composition type and the use case generation specification under the prompt of the analysis test point and extraction constraint condition prompt word, and the use case generation specification has input and output correspondence and design intention annotation information for each test case; under the prompt of the test case generation plan prompt word, the test case generation plan is determined according to the algorithm test points, explicit constraints and implicit constraints of the new question; the test case generation plan at least includes test coverage dimensions, test ideas for each dimension and expected error types; under the prompt of the use case generation prompt word, multiple test cases that meet the test case composition type and use case generation specification are generated according to the test case generation plan.

[0075] In this embodiment, in the process of generating a test case set using the question-use case generation sub-model, the question-use case generation sub-model can be controlled to output in sequence through multiple steps, or a thought chain process of all steps can be generated at one time. The use case generation prompt template includes at least prompt words for analyzing test points and extracting constraints, prompt words for test case generation solutions, and prompt words for use case generation. In this embodiment, the exemplary scenario data is a new question, which is input into the use case generation prompt template, and the use case generation prompt template containing the new question is defined as a use case generation prompt word. At this time, the use case generation prompt word also includes prompt words for analyzing test points and extracting constraints, prompt words for test case generation solutions, and prompt words for use case generation. Exemplarily, the use case generation prompt template can be expressed as:

[0076] As an expert in programming language processing tasks, please generate a comprehensive test case set for the provided topic. The requirements are as follows:

[0077] 1. Requirements for test case composition: Basic test cases (3 - 5): Small-scale typical inputs to verify the basic correctness of the algorithm; Boundary test cases (4 - 6): Including but not limited to the following types: Input scale boundaries (empty input, minimum / maximum scale), Numerical boundaries (extreme values, special values), Special condition boundaries (special constraints specified in the problem); Stress test cases (2 - 3): Maximum allowable scale inputs to verify time efficiency.

[0078] 2. Generation specifications: Each test case must include a clear input-output correspondence; Annotate the design intent of each test case; Ensure 100% coverage of all constraints described in the problem.

[0079] 3. Output format:

[0080] {"inputs": [list of input parameters],

[0081] "outputs": [expected output],

[0082] "fn_name": "function name",

[0083] "test_type": "basic / boundary / stress",

[0084] "description": "description of test purpose"}.

[0085] Step 1: Please carefully read the #problem#, analyze the core examination points of the #problem#, and extract all constraints described in the problem.

[0086] Step 2: Please generate a test case design plan according to the #core examination points and constraints# of the #problem#, and the plan should include: 1) Test dimensions planned to be covered 2) Specific test case design ideas for each dimension 3) Common error types expected to be discovered.

[0087] Step 3: Generate test cases that meet the requirements of test case composition, generation specifications, and output format according to the listed #test case design plan#.

[0088] Step 4: Provide the #final test case list# based on the #test cases#, without any explanation.

[0089] Please reply strictly in the following format:

[0090] Step 1#Core examination points and constraints#:

[0091] Step 2#Test case design plan#:

[0092] Step 3#Test cases#:

[0093] Step 4#Final test case list#:

[0094] #Title#: {new title}.

[0095] Replace the new question into the above use case generation prompt template to obtain the use case generation prompt word. Input the use case generation prompt word into the question use case generation sub-model. The question use case generation sub-model will generate corresponding data according to the format of the above template for output.

[0096] As can be seen from the above, this embodiment generates test case designs by analyzing core test points and constraints. This effectively exposes code issues in boundary condition handling, algorithm correctness, and execution efficiency, effectively improving the quality of training sample data. Furthermore, generating a test case set through the question-case generation sub-model reduces manual intervention and labor costs, effectively improving the efficiency of generating code-based training sample data.

[0097] In the above embodiment, there is no limitation on how to verify the new questions and test case sets. This embodiment also provides multiple exemplary verification methods for the new questions and test case sets, which may include the following:

[0098] Generate multiple problem-solving ideas for the new question, and select at least two target problem-solving ideas that correctly solve the new question; when the algorithmic difficulty of the new question is greater than the algorithmic difficulty of the seed question, the new question meets the preset error-free and unambiguous semantic conditions, and the new question matches the test case set, the new question passes verification; if the test case set matches the new question, and the test case set contains at least basic test cases, boundary test cases, and stress test cases, the test case set passes verification; when there are no at least two target problem-solving ideas, or the algorithmic difficulty of the new question is less than or equal to the algorithmic difficulty of the seed question, or the semantic conditions are not met, generate question modification suggestions; when the new question does not match the test case set, generate question and case modification suggestions; when the test case set does not contain at least one of the basic test cases, boundary test cases, and stress test cases, generate case modification suggestions.

[0099] In this embodiment, the questions can be verified by simulating different problem-solving strategies and scenarios to ensure that the generated questions are solvable, have a higher difficulty level than the seed questions, do not introduce errors or inconsistencies, and match the test case set. If the new question is found to not meet the above conditions during verification, a statement indicating that the new question is unsuitable and reasons for its inappropriateness can be generated. Algorithmic difficulty can be categorized as basic, intermediate, advanced, and combinatorial. Basic algorithms, such as simulation, greedy, and simple search, are suitable for introductory questions. Intermediate algorithms, such as dynamic programming, classic graph theory algorithms, and binary search, require a certain level of algorithmic expertise. Advanced algorithms, such as network flows, computational geometry, advanced string processing, and mathematical optimization, are generally more difficult. Combinatorial algorithms require the combination of multiple algorithms or data structures, such as dynamic programming plus segment trees. The test case set must match the new question and cover all possible scenarios of the new question. If a test case is found to not cover all possible scenarios in the question, feedback can be generated indicating that the test case is unsuitable and explaining the omissions.

[0100] In a parallel implementation to the above embodiment, a high-performance large language model (such as GPT-4) can be used as a data verification sub-model. Specifically, the sample data generation model at least includes generating a data verification sub-model, which executes the verification process for questions and test cases. Structured prompts guide the optimization agent to conduct a comprehensive assessment of the questions. The prompt words include at least verification analysis prompt words and modification suggestion prompt words. Under the prompt of the verification analysis prompt words, the data verification sub-model generates multiple solution ideas for the new question and selects at least two target solution ideas to correctly solve the new question; when the algorithmic difficulty of the new question is greater than the algorithmic difficulty of the seed question, the new question meets the preset error-free and unambiguous semantic conditions, and the new question matches the test case set, then the new question passes verification; if the test case set matches the new question and the test case set contains at least basic test cases, boundary test cases, and stress test cases, then the test case set passes verification and outputs no modification suggestions; under the prompt of the modification suggestion prompt words, when there are no at least two target solution ideas, or the algorithmic difficulty of the new question is less than or equal to the algorithmic difficulty of the seed question, or the semantic conditions are not met, the data verification sub-model generates and outputs problem modification suggestions; when the new question does not match the test case set, it generates and outputs problem and case modification suggestions; when the test case set does not contain at least one of the basic test cases, boundary test cases, and stress test cases, it generates and outputs case modification suggestions.

[0101] Furthermore, to improve verification accuracy, this embodiment uses a multi-round verification and dynamic feedback mechanism to ensure the quality of generated questions and test cases, which may include the following:

[0102] A review prompt template is pre-built, and the seed questions, new questions, and test case set are input into the review prompt template to obtain single-round review prompt words. The single-round review prompt words, seed questions, new questions, and test case set are input into the data verification sub-model multiple times to obtain multiple independent single-round review results. Each single-round review result is voted on. If the voting result does not meet the preset voting pass conditions, each single-round review result is input into the review prompt template to obtain review result prompt words. The review result prompt words and each single-round review result are input into the data verification sub-model. Under the prompt of the modification prompt words, the data verification sub-model modifies the new questions or test cases according to the modification suggestions of each single-round review result, and outputs the modified new questions or modified test cases under the prompt of the audit result output prompt words.

[0103] In this embodiment, if Figure 5 As shown, the verification process may include three steps: single-round question verification, multiple rounds of voting, and question modification. Single-round question verification can be achieved through the above-mentioned embodiment, that is, the data verification sub-model generates multiple problem-solving ideas for the new question under the prompt of the verification analysis prompt word, and selects at least two target problem-solving ideas to correctly solve the new question; when the algorithmic difficulty of the new question is greater than the algorithmic difficulty of the seed question, the new question meets the preset error-free and unambiguous semantic conditions, and the new question matches the test case set, then the new question passes the verification; if the test case set matches the new question, and the test case set contains at least basic test cases, boundary test cases, and stress test cases, then the test case set passes the verification and outputs no modification suggestions. ; Under the prompt of the modification suggestion word, the data verification sub-model generates and outputs modification suggestions for the question when there are no at least two target problem-solving ideas, or the algorithmic difficulty of the new question is less than or equal to the algorithmic difficulty of the seed question, or the semantic conditions are not met; when the new question does not match the test case set, it generates and outputs modification suggestions for the question and use case; when the test case set does not contain at least one of the basic test case, boundary test case, and stress test case, it generates and outputs use case modification suggestions. The single-round review result will output no modification suggestions or modification suggestions for the question, modification suggestions for the question and use case, or use case modification suggestions. After the single-round review results are generated, this embodiment adopts a multi-round sampling voting mechanism to summarize the independent single-round review results generated in each round through the voting mechanism. For example, 5 single-round review results are generated for the same question and test case. Only when more than 80% (4 times) of the verifications pass, the new question and test case set are recognized. Otherwise, the modification suggestions are summarized as question modification prompts and sent to the question and use case generation sub-model to modify the new question or test case set. For example, if the data validation sub-model points out that there is an unclear condition in the question, the question use case generation sub-model will rephrase the question based on the prompt to make it clearer and more accurate.

[0104] In this embodiment, in the process of verifying the generated data using the data verification sub-model, the data verification sub-model can be controlled to output in sequence through multiple steps, or a thought chain process of all steps can be generated at one time. The audit prompt template includes at least a single-round verification prompt word and a result output prompt word. The single-round verification prompt word includes at least a verification analysis prompt word and a modification opinion prompt word; the result output prompt word includes at least a modification prompt word and an audit result output prompt word. The audit prompt template of this embodiment includes two parts, namely, a single-round verification prompt word and a result output prompt word. For the template corresponding to the single-round verification prompt word in the first part, the exemplary scenario data at this time is a seed question, a new question, and a test case set, which are input into the audit prompt template, and the audit prompt template containing these input data is defined as a single-round audit prompt word. At this time, the single-round audit prompt word also includes a verification analysis prompt word and a modification opinion prompt word. For the template corresponding to the result output prompt words in the latter part, the exemplary scenario data at this time is the results of each single round of review, which are input into the review prompt template, and the review prompt template containing these input data is defined as the review result prompt words. At this time, the single round review prompt words also include the modification prompt words and the review result output prompt words. For example, the review prompt template can be expressed as:

[0105] As an expert in programming language processing tasks, please conduct multi-dimensional verification of the #new topic# and provide modification suggestions.

[0106] 1. Note that #new questions# are #seed questions# that have been upgraded to more complex versions through instruction evolution technology.

[0107] 2. #Test case# is a test case for solving the #new question#.

[0108] 3. Verification dimensions include: 1) Solvability analysis: generate three different problem-solving approaches and evaluate whether at least two of them can correctly solve the problem; 2) Difficulty assessment: compare with the original problem to determine whether the algorithm complexity has been reasonably increased; 3) Consistency check: ensure that the problem description is unambiguous and matches the test case; 4) Test case coverage: verify whether basic, boundary, and stress tests are included.

[0109] 4. Generate #Modification Suggestions# based on the #Verification Analysis# content. If all verification dimensions meet the requirements, output "No Modification Suggestions."

[0110] Please reply strictly according to the following format:

[0111] Step 1#Verification Analysis#:

[0112] Step 2#Modify comments#:

[0113] Among them, #seed topic#: {seed topic}; #new topic#: {new topic}; #test case#: {test case}.

[0114] As a programming language processing expert, please strictly follow the #modifications# list to modify the #new topic# and #test case#. Output the modified topic and test case separately. Please reply strictly according to the following format:

[0115] Step 1#Modified title#:

[0116] Step 2#Modified test case#:

[0117] Among them, #modified content#: {modified content}, #new title#: {new title}, #test case#: {test case}.

[0118] As can be seen above, this embodiment, through multiple verifications and voting mechanisms, can complete rigorous question and test case review, further ensuring the validity of new question and test case generation, and effectively guaranteeing the reliability and accuracy of generated data. Furthermore, by using the data validation sub-model to review and verify test cases and new questions, manual intervention and labor costs are reduced, effectively improving the efficiency of generating code training sample data.

[0119] In the above embodiment, there is no limitation on how to answer the new question that has been verified. This embodiment also provides multiple implementation methods for verifying the answers to the new questions, such as Figure 6 As shown, it may include the following:

[0120] Determine the time complexity and space complexity based on the algorithm test points of the verified new questions; determine the corresponding problem-solving process and algorithm step annotation information based on the time complexity and space complexity; answer the verified new questions according to the problem-solving process, annotate the corresponding steps of the answer according to the algorithm step annotation information, and identify whether the answer covers all constraints; input the test case set and the answer that covers all constraints into the sandbox verification environment for answer verification, and if the answer verification fails, adjust the answer according to the error information during the answer verification process.

[0121] The time and space complexity can be calculated using any relevant technology, and the algorithm step annotation information can be used to annotate user-specified steps or steps that are critical to the accuracy of the answer. This does not affect the implementation of the present invention. Answer verification can be performed using a containerized sandbox environment. The sandbox environment provides a secure and isolated runtime environment, preventing the runtime of the answer from impacting external systems while ensuring the accuracy and reliability of the test. When testing answers in the sandbox environment, string segmentation can be used to extract answer data fragments, such as the code fragment "```python[code fragment]```". The extracted data fragments and the test case for the problem are then input into the sandbox environment for execution. Memory usage, execution time, and other metrics are monitored in real time, and solutions that exceed the problem constraints are directly deemed failed. Verification success can be measured by the pass rate of the test cases in the sandbox environment. For example, if the pass rate of the answer across the test case set is greater than or equal to 90%, the answer is considered verified. If the answer fails verification, that is, if the pass rate of the answer across the test case set is less than 90%, a prompt word can be added to the "answer + error message" to regenerate a new answer and verify it again according to the above method. To avoid infinite loops, the process can be repeated a maximum number of times, such as three times. For example, in the code generation task, the answer is the generated code data. If the first generated code data encounters an array out-of-bounds error during runtime, the error message and the original code data are added to the prompt. Based on this information, the code generation logic is adjusted and new code is generated for verification.

[0122] In a parallel implementation to the above-mentioned embodiment, a large language model (such as Kimi) can be selected as the answer sub-model. Specifically, the sample data generation model includes at least the answer sub-model, which generates solutions and answers corresponding to new questions based on the question. The answer sub-model first analyzes the question requirements, determines the appropriate algorithm and data structure, and then gradually builds the code logic, ultimately generating a complete code implementation. For example, if the new question requires implementing a specific encryption algorithm, the answer sub-model will explain the encryption algorithm's principles and steps in detail, and then write the corresponding code based on the algorithm. After the answer sub-model generates the code, it executes test cases in a sandbox environment to verify the code's correctness, and optimizes answer generation through an error feedback loop. By adding "Generate answer + error message" to the answer sub-model's prompt, the answer sub-model will regenerate the answer and verify it again.

[0123] In this embodiment, an answer prompt template can be constructed in advance, and the new question that has passed the verification is input into the answer prompt template to obtain an answer generation prompt word; the new question that has passed the verification and the answer generation prompt word are input into the answer sub-model, and the answer sub-model determines the time complexity and space complexity according to the algorithm test points of the new question that has passed the verification under the prompt of the answer generation prompt word; according to the time complexity and space complexity, the corresponding problem-solving idea flow and algorithm step annotation information are determined; the new question that has passed the verification is answered according to the problem-solving idea flow, and the corresponding steps of the answer are annotated according to the algorithm step annotation information, and it is identified whether the answer covers all constraints; the answer that covers all constraints and the error information in the answer verification process are input into the answer prompt template to obtain an answer correction prompt word; the answer correction prompt word is input into the answer sub-model, and the answer sub-model regenerates a new answer according to the error information under the prompt of the answer regeneration prompt word.

[0124] In this embodiment, when answering a new question using the answer sub-model, the answer sub-model can be controlled to output the answer sequentially through multiple steps, or a thought chain process for all steps can be generated at once. The answer prompt template includes at least an answer prompt word and an answer regeneration prompt word. The answer prompt template of this embodiment includes two parts: an answer prompt word and an answer regeneration prompt word. For the template corresponding to the answer prompt word in the first part, the exemplary scenario data at this time is a new question that has passed verification. This is input into the answer prompt template, and the answer prompt template containing this input data is defined as an answer generation prompt word. In this case, the answer generation prompt word includes the answer prompt word in the answer prompt template. For the template corresponding to the answer regeneration prompt word in the second part, the exemplary scenario data at this time is an answer that covers all constraints and error information during the answer verification process. This is input into the answer prompt template, and the answer prompt template containing this input data is defined as an answer correction prompt word. In this case, the answer correction prompt word includes the answer regeneration prompt word. Exemplarily, the answer prompt template can be expressed as:

[0125] As an expert in programming language processing tasks, please follow these steps to generate a solution to the problem:

[0126] 1. Analyze the core algorithm requirements of the question and clarify the time and space complexity targets;

[0127] 2. Design the problem-solving process and mark the key algorithm steps;

[0128] 3. Write code in the specified programming language, including detailed comments;

[0129] 4. Self-verify whether the code covers all the constraints of the problem;

[0130] #Title#: {Title}.

[0131] Please regenerate the solution for the #new question# based on the #error description# and #submit answer#.

[0132] #Submit answer#: {Answer that did not pass verification};

[0133] This answer has the following problems:

[0134] #Error Description#

[0135] {Error description}.

[0136] Please reply strictly according to the following format:

[0137] Step 1#Error Analysis#:

[0138] Step 2#Modify the code#:

[0139] Step 3 #Self-check#:

[0140] #Title#: {Title}.

[0141] Once the answer is verified, the generated "question-test case-answer" can be formatted and saved in the database as a complete data entry for subsequent training of the language model corresponding to the programming language processing task.

[0142] As can be seen from the above, this embodiment uses a sandbox to verify the answers, and repeats the answers for those that fail verification, ensuring the correctness and practicality of the answers. By adopting a circular verification and solution mechanism, the generated answer solutions both meet the requirements of the questions and have engineering reliability. Furthermore, by using the solution sub-model to answer and verify newly generated questions, manual intervention and labor costs are reduced, effectively improving the efficiency of generating code training sample data.

[0143] Based on the above embodiment, the present invention further provides an implementation method for generating training sample data with different difficulty gradients, which may include the following:

[0144] Adjust the modification range boundary conditions of the new question to increase the difficulty of the current new question; input the new question and prompt words into the sample data generation model, modify the new question within the new modification range boundary conditions based on the same algorithmic thinking, and generate a new test case set covering the constraints of the current new question according to the current new question type and use case generation conditions; verify the current new question and the new test case set, output the verified new test case set, answer the verified current new question, and output the new answer verified by the new test case set; use the current new question, the new test case set and the new answer as the second set of training sample data to generate training sample data with different difficulty levels.

[0145] In this embodiment, in each iteration process, the new question generated in the last iteration is used as the seed question for the current iteration. The feedback data in the last generation process is referred to, such as the difficulty assessment feedback when the new question is generated, the constraint point coverage feedback when the test case is generated, and the error information during the verification of the new question and the test case, to make fine adjustments to the question modification strategy and parameters. For example, in the first iteration, a relatively simple question prototype can be generated based on the basic settings; in the second iteration, the same prompt words are used, and based on the feedback on the difficulty assessment, knowledge point coverage, etc. of the first generated question, the complexity of the question is increased again, such as adjusting the conditions for algorithm application, adding restrictive factors, etc. In this cycle, a series of questions about specific algorithm applications from simple basics to complex and advanced levels can be gradually generated, and the corresponding answers and test cases are simultaneously provided, fully meeting the needs of large models for diversified training data in different training stages such as pre-training and fine-tuning, as well as in different application scenarios such as academic research and industrial applications.

[0146] From the above, it can be seen that this embodiment can generate multiple difficulty variants for the same question by performing multiple rounds of iterative evolution on the same question. It can efficiently collect question types of multiple difficulty levels and form a complete difficulty spectrum to meet the needs of diversified data in different training stages and application scenarios, and enhance the adaptability and generalization ability of the programming language processing task model.

[0147] Finally, the present invention also provides a method for generating training sample data based on multi-agent collaboration. In this embodiment, multiple agents collaborate to complete the process of question evolution, test generation, question review, answer generation and iterative evolution. Figure 7 As shown, it may include the following:

[0148] Different types of language models are pre-used as the question-use case generation sub-model, the generated data verification sub-model, and the answer sub-model. These sub-models are then combined into a sample data generation model. Prompt words are obtained, including at least prompt words for question generation, prompt words for use case generation, prompt words for single-round review, prompt words for review results, prompt words for answer generation, and prompt words for answer correction. Different LLMs are used to respectively undertake the roles of question evolution, test generation, question review, and answer generation: the question use case generation sub-model generates prompt words according to the question, modifies the seed question within the modification range boundary conditions based on the same algorithmic thinking, and generates prompt words according to the use case, and generates a test case set covering the new question constraint conditions according to the new question type and use case generation conditions; the data verification sub-model verifies the new question and test case set multiple times according to the single-round review prompt words, and if the results of multiple rounds of voting do not meet the preset voting pass conditions, the new question or test case set is modified accordingly according to the review result prompt words and the results of each single-round review; here, "or" means: if the new question fails to pass the verification, the new question is modified accordingly according to the review result prompt words and the results of each single-round review; if the test case set fails to pass the verification, the test case set is modified accordingly according to the review result prompt words and the results of each single-round review; the answer sub-model answers the verified new question according to the answer generation prompt words, and if the answer does not pass the answer verification process, the answer is modified according to the answer correction prompt words.

[0149] This example implements the automated generation of high-quality programming training data at multiple difficulty levels by constructing a closed-loop process: "Question evolution - test generation - question review - answer generation." The question use case generation sub-model implements algorithm deconstruction and question evolution, ensuring the assessment of core algorithmic thinking. The data validation sub-model ensures question quality through voting and feedback mechanisms. The solution sub-model generates code solutions and performs rigorous testing in a sandbox environment.

[0150] From the above, it can be seen that this embodiment can generate training sample data with a complete difficulty spectrum through dynamic difficulty control, multi-dimensional test case generation and iterative evolution, and ultimately generate structured training data with multiple difficulty evolution levels. It can not only support supervised learning in the instruction fine-tuning stage, but also meet the reward calculation requirements in the reinforcement learning stage, effectively improving the performance of the language model in code comprehension and generation tasks, and enhancing its ability to solve complex algorithmic problems.

[0151] The present invention also provides a corresponding apparatus for the training sample data generation method, further enhancing the practicality of the method. The apparatus can be described from the perspective of functional modules and hardware. The training sample data generation apparatus described below and the training sample data generation method described above can be used in conjunction with each other.

[0152] From the perspective of functional modules, see Figure 8 , Figure 8 This is a structural diagram of a training sample data generation device provided in this embodiment under a specific implementation. The device may include:

[0153] The data input module 801 is used to input the seed title and prompt words of the programming language processing task into the sample data generation model constructed based on the language model.

[0154] Data generation module 802 is used to generate a model using sample data using prompt words. Based on the same algorithmic thinking, the seed question is modified within the modification range boundary conditions. A test case set is generated that covers the constraints of the new question based on the new question type and use case generation conditions. The module verifies the new question and test case set, outputs the verified test case set, solves the verified new question, and outputs the answer verified by the test case set. The new question, test case set, and answer serve as the first set of training sample data. The module then inputs the new question into the sample data generation model, adjusts the modification range boundary conditions and / or prompt words and / or use case generation conditions, and generates a second set of training sample data.

[0155] Illustratively, in some implementations of this embodiment, the above-mentioned data generation module 802 can also be used to: determine the algorithm type and / or data structure type of the seed question by analyzing the test points of the seed question; determine multiple modification methods of the seed question based on the algorithm type and / or data structure type; from the random combination of the seed question and each modification method, select the modification method that meets the conditions of having the same test points as the seed question and adding at least one new technical point as the optimal combination modification method; modify the seed question accordingly based on the optimal combination modification method to obtain a new question.

[0156] As an exemplary implementation method of the above embodiment, the above data generation module 802 can be further used to: modify the seed topics accordingly based on each optimal combination modification method to obtain multiple initial new topics; screen candidate new topics with no errors and no omissions from each initial new topic, compare the algorithm type and / or data structure complexity of the candidate new topics and the seed topics, and select the candidate new topics with higher complexity than the seed topics as new topics.

[0157] Exemplarily, in some other implementations of this embodiment, the above-mentioned data generation module 802 can also be used to: obtain the test case composition type and the test case generation specification, the test case generation specification is for each test case to have input and output correspondence and design intention annotation information; determine the test case generation plan according to the algorithm test points, explicit constraints and implicit constraints of the new question; the test case generation plan includes at least test coverage dimensions, test ideas for each dimension and expected error types; according to the test case generation plan, generate multiple test cases that meet the test case composition type and the test case generation specification.

[0158] Illustratively, in some other implementations of this embodiment, the sample data generation model includes at least a question use case generation sub-model, and the data generation module 802 may be further configured to: pre-construct a question generation prompt template, the question generation prompt template including at least a test point analysis prompt word, an evolution prompt word, an optimal combination modification method prompt word, and a rewrite prompt word; input the seed question into the question generation prompt template to obtain a question generation prompt word; input the seed question and the question generation prompt word into the question use case generation sub-model, the question use case generation sub-model analyzing the test points of the seed question under the prompt of the test point analysis prompt word to determine the algorithm type and / or data structure type of the seed question; and determining multiple modification methods for the seed question based on the algorithm type and / or data structure type under the prompt of the evolution prompt word; selecting, from a random combination of the seed question and each modification method, a modification method that meets the conditions of having the same test points as the seed question and adding at least one new technical point under the prompt of the optimal combination modification method; and modifying the seed question accordingly based on the optimal combination modification method under the prompt of the rewrite prompt word to obtain a new question.

[0159] Illustratively, in some other implementations of this embodiment, the sample data generation model includes at least a question-use case generation sub-model, and the data generation module 802 may be further configured to: pre-build a use case generation prompt template, the use case generation prompt template including at least an analysis test point and constraint extraction prompt word, a test case generation plan prompt word, and a use case generation prompt word; input a new question into the use case generation prompt template to obtain a use case generation prompt word; input the new question and the use case generation prompt word into the question-use case generation sub-model, the question-use case generation sub-model obtaining a test case composition type and a use case generation specification under the prompt of the analysis test point and constraint extraction prompt word, the use case generation specification providing input and output correspondence and design intent annotation information for each test case; determine a test case generation plan based on the algorithm test points, explicit constraints, and implicit constraints of the new question under the prompt of the test case generation plan prompt word; the test case generation plan including at least test coverage dimensions, test ideas for each dimension, and expected error types; and generate multiple test cases that meet the test case composition type and use case generation specification according to the test case generation plan under the prompt of the use case generation prompt word.

[0160] Exemplarily, in some other implementations of this embodiment, the above-mentioned data generation module 802 can also be used to: generate multiple problem-solving ideas for a new question, and select at least two target problem-solving ideas that correctly solve the new question; when the algorithmic difficulty of the new question is greater than the algorithmic difficulty of the seed question, the new question meets the preset error-free and unambiguous semantic conditions, and the new question matches the test case set, then the new question passes the verification; if the test case set matches the new question, and the test case set contains at least basic test cases, boundary test cases and stress test cases, then the test case set passes the verification; when there are no at least two target problem-solving ideas, or the algorithmic difficulty of the new question is less than or equal to the algorithmic difficulty of the seed question, or the semantic conditions are not met, then a question modification opinion is generated; when the new question does not match the test case set, then a question and case modification opinion is generated; when the test case set does not contain at least one of the basic test cases, boundary test cases and stress test cases, then a case modification opinion is generated.

[0161] Illustratively, in some other implementations of this embodiment, the sample data generation model at least includes generating a data verification sub-model, and the data generation module 802 can be further used to: pre-build an audit prompt template, the audit prompt template at least includes a single-round verification prompt word and a result output prompt word, the single-round verification prompt word at least includes a verification analysis prompt word and a modification opinion prompt word; the result output prompt word at least includes a modification prompt word and an audit result output prompt word; input the seed question, new question and test case set into the audit prompt template to obtain a single-round audit prompt word; input the single-round audit prompt word, ... The project and test case set are input into the data verification sub-model multiple times to obtain multiple independent single-round review results; each single-round review result is voted, and when the voting result does not meet the preset voting pass conditions, each single-round review result is input into the review prompt template to obtain the review result prompt word, and the review result prompt word and each single-round review result are input into the data verification sub-model; wherein, the data verification sub-model modifies the new question or test case according to the modification opinions of each single-round review result under the prompt of the modification prompt word, and outputs the modified new question or modified test case under the prompt of the audit result output prompt word.

[0162] Exemplarily, in some other implementations of this embodiment, the above-mentioned data generation module 802 can also be used to: determine the time complexity and space complexity based on the algorithm test points of the verified new questions; determine the corresponding problem-solving idea flow and algorithm step annotation information based on the time complexity and space complexity; answer the verified new questions according to the problem-solving idea flow, annotate the corresponding steps of the answer according to the algorithm step annotation information, and identify whether the answer covers all constraints; input the test case set and the answer covering all constraints into the sandbox verification environment for answer verification, and when the answer verification fails, adjust the answer according to the error information in the answer verification process.

[0163] Exemplarily, in some other implementations of this embodiment, the above-mentioned sample data generation model includes at least an answer sub-model, and the data generation module 802 can be further used to: pre-construct an answer prompt template, the answer prompt template including at least an answer prompt word and an answer regeneration prompt word; input the new question that has passed the verification into the answer prompt template to obtain an answer generation prompt word; input the new question that has passed the verification and the answer generation prompt word into the answer sub-model, and the answer sub-model determines the time complexity and space complexity according to the algorithm test points of the new question that has passed the verification under the prompt of the answer generation prompt word; determine the corresponding problem-solving idea flow and algorithm step annotation information based on the time complexity and space complexity; answer the new question that has passed the verification according to the problem-solving idea flow, annotate the corresponding steps of the answer according to the algorithm step annotation information, and identify whether the answer covers all constraints; input the answer that covers all constraints and the error information in the answer verification process into the answer prompt template to obtain an answer correction prompt word; input the answer correction prompt word into the answer sub-model, and the answer sub-model regenerates a new answer based on the error information under the prompt of the answer regeneration prompt word.

[0164] Exemplarily, in some other implementations of this embodiment, the above-mentioned data generation module 802 can be further used to: adjust the modification range boundary conditions of the new question to increase the difficulty of the current new question; input the new question and prompt words into the sample data generation model, modify the new question within the new modification range boundary conditions based on the same algorithmic thinking, and generate a new test case set covering the current new question constraint conditions according to the current new question type and case generation conditions; verify the current new question and the new test case set, output the verified new test case set, answer the verified current new question, and output the new answer verified by the new test case set; use the current new question, the new test case set and the new answer as the second set of training sample data to generate training sample data with different difficulty levels.

[0165] Exemplarily, in some other implementations of this embodiment, the above-mentioned data generation module 802 can be further used to: pre-adopt different types of language models as question case generation sub-models, generate data verification sub-models and answer sub-models, and combine the question case generation sub-model, the generated data verification sub-model and the answer sub-model into a sample data generation model; the question case generation sub-model modifies the seed question within the modification range boundary conditions based on the same algorithmic thinking according to the question generation prompt words, and generates a test case set covering the new question constraint conditions according to the new question type and the use case generation conditions according to the use case generation prompt words; the data verification sub-model verifies the new question and the test case set multiple times according to the single-round review prompt words, and if the results of multiple rounds of voting do not meet the preset voting pass conditions, the new question and the test case set are modified accordingly according to the review result prompt words; the answer sub-model answers the new question that has passed the verification according to the answer generation prompt words, and modifies the answer according to the answer correction prompt words when the answer does not pass the answer verification process.

[0166] For the description of the features in the embodiment corresponding to the training sample data generating apparatus, reference can be made to the relevant description of the embodiment corresponding to the training sample data generating method, which will not be repeated here.

[0167] The training sample data generating device mentioned above is described from the perspective of functional modules. Furthermore, the present invention also provides an electronic device, which is described from the perspective of hardware. The electronic device includes a memory and a processor, and the memory stores a computer program. The processor is configured to run the computer program to execute the steps in any one of the above-mentioned training sample data generating method embodiments.

[0168] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any of the above-mentioned training sample data generation method embodiments when running.

[0169] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0170] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above-mentioned training sample data generation method embodiments are implemented.

[0171] An embodiment of the present application further provides another computer program product, comprising a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned training sample data generation method embodiments are implemented.

[0172] The above is a detailed introduction to a training sample data generation method, electronic device, computer-readable storage medium and computer program product provided by the present invention. The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. Whether the units and algorithm steps of each example described in each disclosed embodiment are executed in electronic hardware or computer software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, and such implementation should not be considered to exceed the scope of the present invention. Without departing from the principles of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A method for generating training sample data, characterized in that: include: Input the seed questions and prompt words of the programming language processing task into the sample data generation model built based on the language model; Generate a model using the sample data using the prompt words, modify the seed question within the modification range boundary conditions based on the same algorithmic thinking, and generate a test case set that covers the constraints of the new question according to the new question type and case generation conditions; verify the new question and the test case set, output the verified test case set, answer the verified new question, and output the answer verified by the test case set; Using the new question, the test case set and the answer as a first set of training sample data; Inputting the new topic into the sample data generation model, adjusting the modification range boundary condition and / or the prompt word and / or the use case generation condition, and generating a second set of training sample data; The first set of training sample data and the second set of training sample data are one or any combination of text data, image data, audio and video data; Wherein, based on the same algorithmic thinking, the seed question is modified within the modification range boundary conditions, including: By analyzing the test points of the seed question, the algorithm type and / or data structure type of the seed question is determined; according to the algorithm type and / or data structure type, multiple modification methods of the seed question are determined; from the random combination of the seed question and each modification method, a modification method that meets the conditions of having the same test points as the seed question and adding at least one new technical point is selected as the optimal combination modification method; based on the optimal combination modification method, the seed question is modified accordingly to obtain a new question.

2. The method for generating training sample data according to claim 1, wherein: There are multiple optimal combination modification methods. Modifying the seed topic accordingly based on the optimal combination modification method to obtain a new topic includes: Modify the seed questions accordingly based on each optimal combination modification method to obtain multiple initial new questions; Select candidate new topics without errors or omissions from each initial new topic, compare the algorithm type and / or data structure complexity of the candidate new topic with that of the seed topic, and select the candidate new topic with higher complexity than the seed topic as the new topic.

3. The method for generating training sample data according to claim 1, wherein: Generate a test case set that covers the constraints of the new topic based on the new topic type and test case composition type, including: Obtaining a test case composition type and a test case generation specification, wherein the test case generation specification includes at least input and output correspondence and design intent annotation information for each test case; Determine a test case generation plan based on the algorithm test points, explicit constraints, and implicit constraints of the new question; the test case generation plan at least includes test coverage dimensions, test ideas for each dimension, and expected error types; According to the test case generation scheme, a plurality of test cases that meet the test case composition type and the test case generation specification are generated.

4. The method for generating training sample data according to claim 1, wherein: The sample data generation model includes at least a topic use case generation sub-model, which modifies the seed topic within the modification range boundary conditions based on the same algorithmic thinking, including: Pre-building a prompt template for question generation, wherein the prompt template at least includes prompt words for test point analysis, prompt words for evolution, prompt words for optimal combination modification method, and prompt words for rewriting; Inputting the seed topic into the topic generation prompt template to obtain a topic generation prompt word; The seed question and the question generation prompt word are input into the question use case generation sub-model. The question use case generation sub-model analyzes the test points of the seed question under the prompt of the test point analysis prompt word, determines the algorithm type and / or data structure type of the seed question, and determines multiple modification methods of the seed question based on the algorithm type and / or data structure type under the prompt of the evolution prompt word; under the prompt of the optimal combination modification method prompt word, selects a modification method that meets the conditions of having the same test points as the seed question and adding at least one new technical point from the random combination of the seed question and each modification method as the optimal combination modification method; under the prompt of the rewrite prompt word, the seed question is modified accordingly based on the optimal combination modification method to obtain a new question.

5. The method for generating training sample data according to claim 1, wherein: The sample data generation model includes at least a question and use case generation sub-model, which generates a test case set covering the constraints of the new question according to the new question type and use case generation conditions, including: Pre-build a use case generation prompt template, the use case generation prompt template at least including prompt words for analyzing test points and extracting constraints, prompt words for test case generation plan, and prompt words for use case generation; Inputting the new topic into the use case generation prompt template to obtain a use case generation prompt word; The new question and the use case generation prompt words are input into the question use case generation sub-model. The question use case generation sub-model obtains the test case composition type and the use case generation specification under the prompt of the analysis test point and extraction constraint condition prompt words. The use case generation specification includes at least the input and output correspondence relationship and design intention annotation information for each test case; under the prompt of the test case generation plan prompt words, the test case generation plan is determined according to the algorithm test points, explicit constraints and implicit constraints of the new question; the test case generation plan includes at least test coverage dimensions, test ideas for each dimension and expected error types; under the prompt of the use case generation prompt words, multiple test cases that meet the test case composition type and the use case generation specification are generated according to the test case generation plan.

6. The method for generating training sample data according to claim 1, wherein: Verify the new topic and the test case set, including: Generate multiple solution ideas for the new problem, and select at least two target solution ideas that correctly solve the new problem; if the algorithmic difficulty of the new problem is greater than the algorithmic difficulty of the seed problem, the new problem meets the preset error-free and unambiguous semantic conditions, and the new problem matches the test case set, then the new problem passes the verification; If the test case set matches the new question and the test case set includes at least basic test cases, boundary test cases and stress test cases, then the test case set passes the verification; When there are no at least two target problem-solving ideas, or the algorithmic difficulty of the new problem is less than or equal to the algorithmic difficulty of the seed problem, or the semantic condition is not met, then generating a problem modification suggestion; When the new topic does not match the test case set, generating topic and case modification suggestions; When the test case set does not include at least one of the basic test case, the boundary test case and the stress test case, a case modification suggestion is generated.

7. The method for generating training sample data according to claim 1, wherein: The sample data generation model at least includes a generation data verification sub-model to verify the new question and the test case set, including: Pre-constructing an audit prompt template, the audit prompt template including at least a single-round verification prompt word and a result output prompt word, the single-round verification prompt word including at least a verification analysis prompt word and a modification suggestion prompt word; the result output prompt word including at least a modification prompt word and an audit result output prompt word; Inputting the seed question, the new question and the test case set into the review prompt template to obtain a single-round review prompt word; Inputting the single-round review prompt words, the seed questions, the new questions, and the test case set into the data verification sub-model multiple times to obtain multiple independent single-round review results; Voting on the results of each single round of review; if the voting results do not meet the preset voting pass conditions, inputting the results of each single round of review into the review prompt template to obtain a review result prompt word; inputting the review result prompt word and the results of each single round of review into the data verification sub-model; Among them, the data verification sub-model modifies the new question or the test case according to the modification suggestions of each single round of review results under the prompt of the modification prompt word, and outputs the modified new question or the modified test case under the prompt of the review result output prompt word.

8. The method for generating training sample data according to claim 1, wherein: Solve the new questions that have passed the verification and output the answers that are verified by the test case set, including: Determine the time complexity and space complexity based on the algorithm test points of the verified new questions; Determine the corresponding problem-solving process and algorithm step annotation information based on the time complexity and the space complexity; Solve the verified new questions according to the problem-solving process, annotate the corresponding steps of the answer according to the algorithm step annotation information, and identify whether the answer covers all constraints; The test case set and the answers covering all constraints are input into the sandbox verification environment for answer verification. If the answer verification fails, the answer is adjusted according to the error information during the answer verification process.

9. The method for generating training sample data according to claim 1, wherein: The sample data generation model includes at least an answer sub-model, which answers new questions that have passed verification and outputs answers that have passed the test case set, including: Pre-building an answer prompt template, wherein the answer prompt template at least includes an answer prompt word and an answer regeneration prompt word; Inputting the verified new question into the answer prompt template to obtain the answer and generate prompt words; The verified new question and the answer generation prompt word are input into the solution sub-model. The solution sub-model determines the time complexity and space complexity according to the algorithm test points of the verified new question under the prompt of the answer generation prompt word; determines the corresponding problem-solving process and algorithm step annotation information based on the time complexity and space complexity; solves the verified new question according to the problem-solving process, annotates the corresponding steps of the answer according to the algorithm step annotation information, and identifies whether the answer covers all constraints; Inputting the answers covering all constraints and the error information in the answer verification process into the answer prompt template to obtain answer correction prompt words; The answer correction prompt word is input into the solution sub-model, and the solution sub-model regenerates a new answer according to the error information under the prompt of the answer regeneration prompt word.

10. The method for generating training sample data according to any one of claims 1 to 9, characterized in that: Inputting the new topic into the sample data generation model, adjusting the modification range boundary condition and / or the prompt word and / or the use case generation condition, and generating a second set of training sample data, including: Adjusting the modification range boundary conditions of the new question to increase the difficulty of the current new question; The new question and the prompt word are input into the sample data generation model, the new question is modified within the new modification range boundary conditions based on the same algorithmic thinking, and a new test case set covering the current new question constraint conditions is generated according to the current new question type and the case generation conditions; the current new question and the new test case set are verified, the new test case set that has passed the verification is output, the current new question that has passed the verification is answered, and a new answer verified by the new test case set is output; the current new question, the new test case set and the new answer are used as the second set of training sample data to generate training sample data with different difficulty levels.

11. The method for generating training sample data according to any one of claims 1 to 9, characterized in that: The prompt words include prompt words for question generation, prompt words for use case generation, prompt words for single round review, prompt words for review result, prompt words for answer generation and prompt words for answer correction; Generating a model using the prompt word and the sample data includes: Different types of language models are pre-adopted as a question case generation sub-model, a generated data verification sub-model, and an answer sub-model, and the question case generation sub-model, the generated data verification sub-model, and the answer sub-model are combined into a sample data generation model; The question use case generation sub-model generates prompt words according to the question, modifies the seed question within the modification range boundary conditions based on the same algorithmic thinking, and generates prompt words according to the use case, generates a test case set covering the new question constraint conditions according to the new question type and use case generation conditions; the generated data verification sub-model verifies the new question and the test case set multiple times according to the single-round review prompt words, and if the results of multiple rounds of voting do not meet the preset voting pass conditions, the new question or the test case set is modified accordingly according to the review result prompt words and the results of each single-round review; the answer sub-model answers the verified new question according to the answer generation prompt words, and when the answer does not pass the answer verification process, modifies the answer according to the answer correction prompt words.

12. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the method for generating training sample data according to any one of claims 1 to 11 when executing the computer program.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the training sample data generation method according to any one of claims 1 to 11 are implemented.

14. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the method for generating training sample data according to any one of claims 1 to 11 are implemented.

Images