Model training method, test case generation method, equipment, medium and product

Abstract

The invention relates to the technical field of artificial intelligence, and particularly provides a model training method, a test case generation method, equipment, a medium and a product. According to the method, multiple natural supervision signals of software source codes are extracted, and multi-task instruction fine tuning training of a large-scale language model is realized based on the multiple natural supervision signals of the source codes, so that the large-scale language model can learn semantic features complementary with code features or code features complementary with the semantic features; therefore, when the test case is generated by utilizing the large-scale language model subjected to fine tuning training, the large-scale language model can accurately extract complementary semantic features or code features no matter whether the original data input into the large-scale language model is a code block or a demand description text; the purpose of jointly generating the test case in combination with the semantic features and the code features is achieved, the accuracy of the generated test case is improved, and the problem of feature missing in related technologies is solved.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a model training method, a test case generation method, an apparatus, a medium, and a product. Background Technology

[0002] Before a software project goes live, it requires extensive testing to improve its quality. Therefore, software testing is an indispensable part of the software development cycle. Consequently, there is a pressing need for a solution that can automatically generate test cases for software. Summary of the Invention

[0003] This disclosure is made in view of the above-mentioned problems. This disclosure provides a model training method, a test case generation method, an apparatus, a medium, and a product.

[0004] According to a first aspect of this disclosure, a model training method is provided, comprising: Obtain the software source code, including the requirements description document and test plan and test case document of the software source code; Based on the requirements description document, the software source code, and the test plan and test case document, a variety of natural supervision signals are extracted. These natural supervision signals are used to characterize the code features and the true human intent of the code. Based on the aforementioned multiple natural supervision signals, multiple training tasks are constructed; Based on the aforementioned multiple training tasks and various natural supervision signals, a large-scale language model is fine-tuned and trained.

[0005] Furthermore, according to the model training method of the first aspect of this disclosure, based on the requirements description document, the software source code, and the test plan and test case document, various natural supervision signals are extracted, including: The large-scale language model and pre-set summary prompts are invoked to extract a summary from the requirement description document, thereby obtaining the requirement description text. Based on the requirement description text, business function code blocks and unit test code blocks corresponding to the business function code blocks are obtained from the software source code; and, the large-scale language model and pre-set search prompts are invoked to search the test plan and test case document to obtain test case paragraphs that match the requirement description text. Obtain the serialized text of the abstract syntax tree of the business function code block and the serialized text of the abstract syntax tree of the unit test code block; The requirement description text, the test case paragraph, the unit test code block, the business function code block, the abstract syntax tree serialized text of the unit test code block, and the abstract syntax tree serialized text of the business function code block are used as various natural supervision signals.

[0006] Furthermore, according to the model training method of the first aspect of this disclosure, the multiple training tasks include: Training task based on the business function code block and / or the unit test code block to predict abstract syntax tree serialized text; A training task based on predicting requirement description text using code block serialized text combinations is performed. The code block serialized text combinations include a first combination and / or a second combination. The first combination includes the business function code block and the corresponding abstract syntax tree serialized text. The second combination includes the unit test code block and the corresponding abstract syntax tree serialized text. Training task based on predicting unit test code blocks from functional code blocks.

[0007] Furthermore, according to the model training method of the first aspect of this disclosure, the multiple training tasks also include: The training task is based on the text prediction unit test code block described in the above requirements; and The training task for predicting test cases is based on the requirement description text, the functional code block, the abstract syntax tree serialized text corresponding to the functional code block, the unit test code block, and the abstract syntax tree serialized text corresponding to the unit test block.

[0008] Furthermore, according to the model training method of the first aspect of this disclosure, the training task based on the required text prediction unit test code block includes: The first, second, third, and fourth subtasks are executed in parallel. The first subtask is used to predict the total abstract syntax tree sequence text based on the requirement description text; the second subtask is used to predict the total code block based on the total abstract syntax tree sequence text; the third subtask is used to predict the unit test code serialization text based on the total code block and the total abstract syntax tree sequence text; and the fourth subtask is used to predict the unit test code based on the unit test code serialization text.

[0009] According to a second aspect of this disclosure, a test case generation method is provided, comprising: Obtain the raw input data, which is the code block under test or the requirement description text; The large-scale language model trained based on the first aspect is invoked to perform multi-hop feature enhancement on the original input data to obtain enhanced data; wherein, when the original input data is a code block, the large-scale language model performs semantic feature enhancement on the original input data, and when the original input data is a requirement description text, the large-scale language model performs code feature enhancement on the original input data. Based on the original input data and the enhanced data, the large-scale language model is invoked to generate test cases.

[0010] According to a third aspect of this disclosure, a model training apparatus is provided, comprising: The first acquisition module is used to acquire the software source code, including the requirements description document and test plan and test case document of the software source code. The extraction module is used to extract various natural supervision signals based on the requirements description document, the software source code, and the test plan and test case document. The natural supervision signals are used to characterize the code features and the true human intent of the code. A construction module is used to construct multiple training tasks based on the aforementioned multiple natural supervision signals; The training module is used to fine-tune the training of a large-scale language model based on the multiple training tasks and the multiple natural supervision signals.

[0011] According to a fourth aspect of this disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, the processor executing the computer program to implement the steps of the method described in the first aspect. According to a fifth aspect of this disclosure, a computer-readable storage medium is provided having a computer program / instructions stored thereon that, when executed by a processor, implements the steps of the method described in the first aspect. According to a sixth aspect of this disclosure, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the method described in the first aspect.

[0012] As will be described in detail below, the method according to embodiments of this disclosure extracts multiple natural supervision signals from the software source code and performs multi-task instruction fine-tuning training of a large-scale language model based on these signals. This allows the large-scale language model to learn semantic features that complement code features, or code features that complement semantic features. Thus, when generating test cases using the fine-tuned large-scale language model, regardless of whether the original data input to the large-scale language model is a code block or a requirement description text, the large-scale language model can accurately extract complementary semantic features or code features. This achieves the goal of combining semantic features and code features to generate test cases, improving the accuracy of the generated test cases and solving the feature loss problem existing in related technologies.

[0013] It should be understood that both the foregoing general description and the following detailed description are exemplary and intended to provide further illustration of the claimed technology. Attached Figure Description

[0014] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0015] Figure 1 This is a flowchart illustrating a model training method according to an embodiment of the present disclosure.

[0016] Figure 2 This is a flowchart illustrating a test case generation method according to an embodiment of the present disclosure.

[0017] Figure 3 This is a flowchart illustrating a test case generation method according to an embodiment of the present disclosure.

[0018] Figure 4 This is a flowchart illustrating yet another method for generating test cases according to an embodiment of the present disclosure.

[0019] Figure 5 This is a flowchart illustrating yet another method for generating test cases according to an embodiment of the present disclosure.

[0020] Figure 6 This is a structural diagram illustrating a model training apparatus according to an embodiment of the present disclosure.

[0021] Figure 7 This is a structural diagram of a test case generation apparatus according to an embodiment of the present disclosure.

[0022] Figure 8 This is a hardware block diagram illustrating an electronic device according to an embodiment of the present disclosure.

[0023] Figure 9 This is a schematic diagram illustrating a computer-readable program according to an embodiment of the present disclosure. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0025] From the perspective of model input, test case automatic generation techniques based on large-scale language models can be broadly categorized into: test case automatic generation based on requirements description documents and test case automatic generation based on source code parsing. Among these: The test case automatic generation technology based on requirements description documents performs intent analysis or test type classification on the requirements description text / document, dynamically generates prompts and instructions by a large-scale language model, or matches relevant context slices in the knowledge base through vector retrieval, splices them together and then feeds them back into the large-scale language model to generate test cases.

[0026] The test case generation technology based on source code parsing performs structured parsing of the source code. Through methods such as code compilation and trial execution, parsing the test types of classes or methods, and extracting the code abstract syntax tree, it obtains natural supervision signals related to the code as additional contextual features. The prompts are then dynamically generated by a large-scale language model, or relevant context slices in the knowledge base are matched through vector retrieval, and then concatenated and fed back into the large-scale language model to generate test cases.

[0027] The drawback of related technologies lies in the lack of features in the initial input content, namely: Test case generation technology based on requirements documentation suffers from missing code features. This technology cannot obtain specific contextual information related to the functional logic of the source code; therefore, the accuracy and usability of the generated test cases depend heavily on the level of detail in the requirements documentation.

[0028] While related technologies can attempt to introduce code features by extracting operation keywords and matching code segments using vector retrieval, problems such as inaccurate code segments and the inability to incrementally update the vector knowledge base exist, resulting in low usability of the generated test cases. This is especially true in test case generation scenarios such as UI automation testing, where code segments need to be generated.

[0029] Automatic test case generation technology based on source code parsing suffers from a lack of semantic features. Because this technology only parses source code features, it fails to capture the semantic information of code requirements and input / output descriptions. Test cases generated through instruction fine-tuning or few-sample learning on small datasets exhibit several problems: inability to correctly extract the test type of the code under test, insufficient accuracy and scalability of the generated test cases, and insufficient ability to capture true human intent due to the lack of requirement description text. Existing technologies attempt to improve the quality of generated test cases by extracting code comments, associating unit tests, and variable substitution. However, these methods are limited by issues such as comment quality, semantic differences between code and text, and the loss of code features due to variable name substitution. They cannot fully compensate for the impact of missing semantic information such as human intent on the quality of generated test cases, especially in scenarios such as interface testing and manual testing where text paragraphs need to be generated, resulting in unsatisfactory final results.

[0030] To alleviate the technical problems existing in related technologies, this disclosure provides a model training method and a test case generation method. This method extracts multiple natural supervision signals from software source code and uses these signals to fine-tune the training of a large-scale language model for multi-task instructions. This allows the large-scale language model to learn semantic features complementary to code features, or code features complementary to semantic features. Thus, when generating test cases using the fine-tuned large-scale language model, regardless of whether the original data input to the model is a code block or requirement description text, the model can accurately extract complementary semantic or code features. This achieves the goal of combining semantic and code features to generate test cases, improving the accuracy of the generated test cases and solving the feature loss problem existing in related technologies.

[0031] To facilitate understanding of this embodiment, a model training method disclosed in this disclosure will first be described in detail. The execution subject of the model training method provided in this disclosure is generally an electronic device with certain computing power, such as a terminal device, a server, or other processing device. In some possible implementations, the model training method can be implemented by a processor calling computer-readable instructions stored in memory.

[0032] See Figure 1The diagram shows a flowchart of a model training method provided in this embodiment of the present disclosure. The method includes the following steps: Step 101: Obtain the software source code, including the requirements description document and test plan and test case documents.

[0033] A requirements document is a collection of business requirements, user requirements, functional requirements, and non-functional requirements. It describes in a structured way the conditions and capabilities that a software product must meet. In the early stages of software development, a requirements document needs to be created, and the software source code is written based on it.

[0034] Test plans and test case documents are documents needed in the later stages of software development. These documents define the content of software testing.

[0035] Step 102: Based on the requirements description document, software source code, and test plan and test case documents, extract various natural supervision signals. Natural supervision signals are used to characterize the code features and the true human intent of the code.

[0036] In an optional embodiment, step 102 may include the following steps: By calling a large-scale language model and pre-set summary prompts, a summary extraction is performed on the requirements description document to obtain the requirements description text. Based on the requirements description text, obtain the business function code blocks and the corresponding unit test code blocks from the software source code; and, by calling a large-scale language model and pre-set search prompts, retrieve test plans and test case documents to obtain test case paragraphs that match the requirements description text. Obtain the serialized text of the abstract syntax tree of the business function code block and the serialized text of the abstract syntax tree of the unit test code block; The requirement description text, test case paragraphs, unit test code blocks, business function code blocks, the abstract syntax tree sequence of unit test code blocks, and the abstract syntax tree sequence of business function code blocks are used as various natural supervision signals.

[0037] The requirements description text may include functional descriptions of various business interfaces, request methods, method name definitions, URL definitions, parameter definitions, and / or return value content. Using a large language model and based on pre-built summary prompts, this descriptive information can be extracted from the requirements description document, and this descriptive information constitutes the requirements description text in this embodiment.

[0038] By using regular expression matching and programming language-specific boundary end character positioning methods to process the descriptive information in the requirement description text, we can obtain the business function code block corresponding to the requirement description text, as well as the unit test code block corresponding to the business function code block.

[0039] By using pre-built search suggestions and calling a large-scale language model, test plans and test case documents can be retrieved, and test case paragraphs that match the requirement description text can be extracted.

[0040] For the abstract syntax trees corresponding to the business function code blocks and unit test code blocks, the N-ary tree serialization algorithm is used to convert the abstract syntax trees into serialized text that can be read and modeled by large-scale language models, thus obtaining the serialized text of the abstract syntax trees of the business function code blocks and the serialized text of the abstract syntax trees of the unit test code blocks.

[0041] By using pre-designed instruction prompt templates and invoking a large-scale language model, mappings can be established between software source code and requirement description text, test case paragraphs, unit test code blocks, business function code blocks, and the serialized text of the abstract syntax tree of unit test code blocks and the serialized text of the abstract syntax tree of business function code blocks. Multiple such mappings can be obtained for each software source code, each mapping serving as a training data point. To ensure the effectiveness of subsequent training of the large-scale language model, the total size of this training data can be controlled to be over 100,000 points.

[0042] There are no strict requirements for defining the instruction prompt template. Clear and concise prompt text should be written based on the specific subtask type and its inputs and outputs. Optionally, for abstract syntax tree serialized text, special identifiers in the sequence text can be customized as tokens in the vocabulary to enhance the modeling effect of large-scale language models on sequence text.

[0043] Step 103: Construct multiple training tasks based on various natural supervision signals.

[0044] By constructing a multi-task instruction dataset, and based on a large-scale language model, fine-tuning training of instructions for multiple tasks can be performed simultaneously.

[0045] In one alternative embodiment, multiple training tasks may include: Training task based on predicting abstract syntax tree serialized text using business function code blocks and / or unit test code blocks; The training task is based on predicting requirement description text by combining code block serialized text. The code block serialized text combination includes a first combination and / or a second combination. The first combination includes business function code blocks and the corresponding abstract syntax tree serialized text. The second combination includes unit test code blocks and the corresponding abstract syntax tree serialized text. Training task based on predicting unit test code blocks from functional code blocks.

[0046] Based on functional code blocks or unit test code blocks, and concatenated with corresponding prompt word templates, a large-scale language model predicts and generates the abstract syntax tree (AST) serialized text corresponding to the functional code blocks and the unit test code blocks. By transforming the functional logic, input / output content, and variable data types of the code into explicit text sequences that can be modeled by the model, the semantic features of the original input are enhanced.

[0047] The task involves generating corresponding requirement description text based on functional code blocks, unit test code blocks, and the serialized text of the abstract syntax trees (ASTs) corresponding to the functional code blocks and unit test code blocks. The serialized text of the ASTs corresponding to the code blocks serves as a transition between the code segments and the requirement description text. By accurately converting the functional logic, input / output, variable types, and defect assertions contained in the functional code and unit test code into serialized text that can be explicitly modeled by large-scale language models, the accuracy of the generated requirement description text can be improved. Furthermore, by sub-tasks that generate requirement text from code blocks, the semantic features of the original input can be further enhanced.

[0048] Training tasks based on code block prediction of abstract syntax tree serialized text and training tasks based on code block serialized text combination to predict requirement description text enable large-scale language models to accurately capture the true human intent of the code under test, understand the specific testing purpose, and thus generate more accurate test cases.

[0049] Based on the serialized text of functional code blocks and their corresponding abstract syntax trees, and with the help of corresponding instruction prompt templates, the training task of generating corresponding unit test code blocks can be achieved. This allows large-scale language models to model the potential relationship between the functional logic of the code and the content to be tested. By generating unit test code, additional code features can be introduced into the subsequent test case generation task.

[0050] In some embodiments, multiple training tasks may further include: Training task based on the requirement description text prediction unit test code block; and The training task is to predict test cases based on requirement description text, functional code blocks, serialized text of the abstract syntax tree corresponding to the functional code blocks, unit test code blocks, and serialized text of the abstract syntax tree corresponding to the unit test blocks.

[0051] The training task for the unit test code block based on the requirement description text prediction includes: The first, second, third, and fourth subtasks are executed in parallel. The first subtask is used to predict the total abstract syntax tree sequence text based on the requirement description text; the second subtask is used to predict the total code block based on the total abstract syntax tree sequence text; the third subtask is used to predict the unit test code serialization text based on the total code block and the total abstract syntax tree sequence text; and the fourth subtask is used to predict the unit test code based on the unit test code serialization text.

[0052] Based on the requirement description text and the corresponding multi-segment instruction prompt template, 1) generate the abstract syntax tree serialization text of the corresponding function, 2) generate its corresponding code block, then 3) generate the abstract syntax tree serialization text of the unit test code block based on the above results, and 4) generate the task of the corresponding unit test code. Due to the need for parallel training, the serialization generation task consisting of the above four steps needs to be masked multiple times, splitting it into multiple training sample sets: for the generation task of the i-th step, the results of the (i+1)-4th generation tasks need to be masked, and the prompt text concatenated from the known inputs of the 1st to (i-1)th tasks is used to predict the result of the current i-th step. In this process, converting the requirement description text into abstract syntax tree serialized text can improve the accuracy of the generated code block's functional logic and variable / method types, better helping large-scale language models model the relationship between the requirement description text and code variables, syntax, functional logic, etc.

[0053] Based on the requirement description text, functional code blocks and their corresponding abstract syntax tree serialized text, unit test code blocks and their corresponding abstract syntax tree serialized text, the task of generating final test cases is performed by concatenating them with the corresponding instruction prompt templates and then feeding them into a large-scale language model. The requirement description text and abstract syntax tree serialized text enhance semantic feature information; the functional code blocks and unit test blocks enhance code feature information; and the corresponding abstract syntax tree serialized text displays the relationship between the modeled requirement description text and the code functional logic, thereby improving the accuracy of the generated test cases.

[0054] The five tasks described above form two multi-hop generation chains: "requirement description text -> code and abstract syntax tree serialization text -> unit test code and abstract syntax tree serialization text" and "code and abstract syntax tree serialization text -> unit test code and abstract syntax tree serialization text -> requirement description text," corresponding to two different subsequent input scenarios. By fine-tuning the instructions on multiple closely related subtasks, the overall performance of all subtasks can be improved, thereby enhancing the accuracy of the test case generation subtasks.

[0055] In addition to optimizing the embedding of instruction prompts in multi-task fine-tuning through instruction fine-tuning techniques, alternatively, techniques such as LORA and Adapter fine-tuning can be used to retain the model's pre-training knowledge while improving the model's performance on new tasks by adding small low-rank matrices or Adapter layers.

[0056] Step 104: Fine-tune the large-scale language model based on multiple training tasks and various natural supervision signals.

[0057] The technical solution provided in this embodiment extracts multiple natural supervision signals from the software source code and uses these signals to fine-tune the training of a large-scale language model for multi-task instructions. This allows the large-scale language model to learn semantic features that complement code features, or code features that complement semantic features. Thus, when generating test cases using the fine-tuned large-scale language model, regardless of whether the original data input to the model is a code block or a requirement description text, the model can accurately extract complementary semantic or code features. This achieves the goal of combining semantic and code features to generate test cases, improving the accuracy of the generated test cases and solving the feature loss problem in related technologies.

[0058] This disclosure also provides a test case generation method, such as... Figure 2 As shown, the method may include: Step 201: Obtain the raw input data, which is the code block under test or the requirement description text; Step 202: Call the large-scale language model trained based on the model training method to perform multi-hop feature enhancement on the input raw data to obtain enhanced data; wherein, when the input raw data is a code block, the large-scale language model performs semantic feature enhancement on the input raw data, and when the input raw data is a requirement description text, the large-scale language model performs code feature enhancement on the input raw data. Step 203: Based on the input raw data and augmented data, call the large-scale language model to generate test cases.

[0059] Based on the large-scale language model after fine-tuning training, and the multi-hop generation link formed between training tasks, when the input raw data is the code block under test or the requirement description text, additional semantic features or code features are introduced into the input raw data to perform corresponding feature enhancement.

[0060] As an example, the automatic generation process of test cases is explained when the input raw data is the code block under test and the requirement description text, respectively. Please refer to [link / reference]. Figure 3 , Figure 3This illustrates the process of generating test cases when the input raw data is the code block under test. For example... Figure 3 As shown, based on the preset instruction prompt template, the large-scale language model generates the abstract syntax tree serialized text corresponding to the code block under test, generates unit test code and the abstract syntax tree serialized text corresponding to the unit test code based on the abstract syntax tree serialized text corresponding to the unit test code, and generates requirement description text based on the unit test code and the abstract syntax tree serialized text corresponding to the unit test code, thereby enhancing semantic features.

[0061] Please refer to Figure 4 , Figure 4 This illustrates the process of generating test cases when the input raw data is a requirement description text. For example... Figure 4 As shown, based on the preset instruction prompt template, the large-scale language model generates code blocks and the corresponding abstract syntax tree serialized text, unit test code and its corresponding abstract syntax tree serialized text in sequence to enhance code features.

[0062] Finally, by performing multi-hop feature enhancement on the original input data, and based on the corresponding prompt word template, the generated intermediate content is concatenated with the original input to generate the corresponding test cases.

[0063] The solution in this embodiment improves the quality of the generated test cases by introducing code features or semantic features that are missing from the original input data during the test case generation process.

[0064] This disclosure also provides a test case generation method, such as... Figure 5 As shown, the method may include: Step 501: Constructing a multi-task instruction dataset.

[0065] The raw data for constructing the dataset comes from the project source code, requirement description documents, test plans and test case documents of actual business. Based on this, it is necessary to complete the relationship mapping between the code and the requirement description text, test cases, unit test code, and abstract syntax tree serialized text, so as to concatenate it with the instruction template and construct a multi-task instruction dataset.

[0066] First, using a large-scale language model, a document summary prompt template is constructed to extract the functional descriptions, request methods, method name definitions, URL definitions, parameter definitions, and return value content of various business interfaces in the requirements / design documents, and to construct the requirements description text.

[0067] The system uses regular expression matching and programming language-specific boundary end character positioning methods to match the description information such as method names and request methods in the requirement description text, thereby obtaining the business function code block corresponding to the requirement description text, as well as the unit test code block corresponding to the business function code block.

[0068] Subsequently, by constructing document retrieval prompt word templates and leveraging a large-scale language model, test case paragraphs in test plans and test case documents that match the requirement description text were retrieved.

[0069] Finally, the abstract syntax trees corresponding to the business function code blocks and unit test code blocks are converted into serialized text that can be read and modeled by large-scale language models using an N-ary tree serialization algorithm.

[0070] Step 502: Multi-task instruction fine-tuning training.

[0071] By constructing a multi-task instruction dataset, and based on a large-scale language model, fine-tuning training of instructions for multiple tasks can be performed simultaneously. The code-to-sequenced AST tree generation task refers to concatenating functional code blocks or unit test code blocks with corresponding prompt word templates to enable a large model to predict and generate the corresponding AST tree sequence.

[0072] The task of generating requirement description text from serialized AST tree refers to the task of generating corresponding requirement description text based on functional code blocks, unit test code blocks, and the corresponding serialized AST tree.

[0073] The code -> unit test code generation task refers to the task of generating corresponding unit test code blocks based on functional code blocks and their corresponding serialized AST trees, with the help of corresponding instruction prompt templates.

[0074] The requirement description text -> code generation task refers to the task of generating the code serialization AST tree of the corresponding function based on the requirement description text and the corresponding multi-segment instruction prompt template, 1) generating the code serialization AST tree of the corresponding function, 2) generating its corresponding code block, 3) generating the corresponding unit test code serialization AST tree based on the above results, and 4) generating the corresponding unit test code.

[0075] The test case generation task refers to the task of generating final test cases by concatenating the requirement description text, functional code blocks and their serialized AST trees, and unit test code blocks and their serialized AST trees with the corresponding instruction prompt templates and then feeding them into a large-scale language model.

[0076] Step 503: Test cases are automatically generated.

[0077] This disclosure also provides a model training apparatus for performing the model training method provided in any of the above embodiments. Figure 6 As shown, the device includes: The first acquisition module 61 is used to acquire software source code, including the requirements description document and test plan and test case document of the software source code. Extraction module 62 is used to extract various natural supervision signals based on the requirements description document, the software source code, and the test plan and test case document. The natural supervision signals are used to characterize the code features and the true human intent of the code. Construction module 63 is used to construct multiple training tasks based on the multiple natural supervision signals; Training module 64 is used to fine-tune the training of a large-scale language model based on the multiple training tasks and the multiple natural supervision signals.

[0078] In an optional embodiment, the extraction module 62 is used for: The large-scale language model and pre-set summary prompts are invoked to extract a summary from the requirement description document, thereby obtaining the requirement description text. Based on the requirement description text, business function code blocks and unit test code blocks corresponding to the business function code blocks are obtained from the software source code; and, the large-scale language model and pre-set search prompts are invoked to search the test plan and test case document to obtain test case paragraphs that match the requirement description text. Obtain the serialized text of the abstract syntax tree of the business function code block and the serialized text of the abstract syntax tree of the unit test code block; The requirement description text, the test case paragraph, the unit test code block, the business function code block, the abstract syntax tree serialized text of the unit test code block, and the abstract syntax tree serialized text of the business function code block are used as various natural supervision signals.

[0079] In one optional embodiment, the multiple training tasks include: Training task based on the business function code block and / or the unit test code block to predict abstract syntax tree serialized text; A training task based on predicting requirement description text using code block serialized text combinations is performed. The code block serialized text combinations include a first combination and / or a second combination. The first combination includes the business function code block and the corresponding abstract syntax tree serialized text. The second combination includes the unit test code block and the corresponding abstract syntax tree serialized text. Training task based on predicting unit test code blocks from functional code blocks.

[0080] In an optional embodiment, the multiple training tasks further include: The training task is based on the text prediction unit test code block described in the above requirements; and The training task for predicting test cases is based on the requirement description text, the functional code block, the abstract syntax tree serialized text corresponding to the functional code block, the unit test code block, and the abstract syntax tree serialized text corresponding to the unit test block.

[0081] In an optional embodiment, the training task for the text prediction unit test code block based on the requirements description includes: The first, second, third, and fourth subtasks are executed in parallel. The first subtask is used to predict the total abstract syntax tree sequence text based on the requirement description text; the second subtask is used to predict the total code block based on the total abstract syntax tree sequence text; the third subtask is used to predict the unit test code serialization text based on the total code block and the total abstract syntax tree sequence text; and the fourth subtask is used to predict the unit test code based on the unit test code serialization text.

[0082] The model training apparatus and the model training method provided in this disclosure are based on the same inventive concept and have the same beneficial effects as the methods they employ, operate, or implement.

[0083] This disclosure also provides a test case generation apparatus for executing the test case generation method provided in any of the above embodiments. Figure 7 As shown, the device includes: The second acquisition module 71 is used to acquire input raw data, wherein the input raw data is the code block under test or the requirement description text. The module 72 is used to call the large-scale language model trained by the model training method of the aforementioned embodiment to perform multi-hop feature enhancement on the input raw data to obtain enhanced data; wherein, when the input raw data is a code block, the large-scale language model performs semantic feature enhancement on the input raw data, and when the input raw data is a requirement description text, the large-scale language model performs code feature enhancement on the input raw data. The generation module 73 is used to generate test cases by calling the large-scale language model based on the input raw data and the enhanced data.

[0084] This disclosure also provides an electronic device for executing the above-described model training method or test case generation method. Please refer to... Figure 8 It illustrates a schematic diagram of an electronic device provided by some embodiments of this disclosure. For example... Figure 8As shown, the electronic device 8 includes: a processor 800, a memory 801, a bus 802, and a communication interface 803. The processor 800, the communication interface 803, and the memory 801 are connected via the bus 802. The memory 801 stores a computer program that can run on the processor 800. When the processor 800 runs the computer program, it executes the model training method or test case generation method provided in any of the foregoing embodiments of this disclosure.

[0085] The memory 801 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this device network element and at least one other network element is achieved through at least one communication interface 803 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0086] Bus 802 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 801 is used to store programs. After receiving an execution instruction, the processor 800 executes the program. The model training method or test case generation method disclosed in any of the foregoing embodiments of this disclosure can be applied to the processor 800, or implemented by the processor 800.

[0087] The processor 800 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 800 or by instructions in software form. The processor 800 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this disclosure can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 801. Processor 800 reads the information in memory 801 and, in conjunction with its hardware, completes the steps of the above method.

[0088] This disclosure also provides a computer-readable storage medium corresponding to the model training method or test case generation method provided in the foregoing embodiments. The computer-readable storage medium is an optical disc, on which a computer program (i.e., a computer program product) is stored. When the computer program is run by a processor, it executes the model training method or test case generation method provided in any of the foregoing embodiments.

[0089] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0090] The computer-readable storage medium provided in the above embodiments of this disclosure is based on the same inventive concept as the model training method or test case generation method provided in the embodiments of this disclosure, and has the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0091] This disclosure also provides a computer program product; please refer to [reference needed]. Figure 9The computer program product 900 carries program code, namely computer program 901. The instructions included in the computer program 901 can be used to execute the steps of the model training method or test case generation method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0092] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0093] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0094] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0095] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0096] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0097] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0098] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0099] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A model training method, characterized in that, The method comprises: obtaining software source code, requirement description documents and test plan and use case documents of the software source code; extracting multiple natural supervision signals based on the requirement description documents, the software source code and the test plan and use case documents, the natural supervision signals being used to represent code features of the code and human true intentions of the code; constructing multiple training tasks based on the multiple natural supervision signals; fine-tuning training a large-scale language model based on the multiple training tasks and the multiple natural supervision signals.

2. The method of claim 1, wherein, Extracting multiple natural supervision signals based on the requirement description documents, the software source code and the test plan and use case documents comprises: calling the large-scale language model and pre-set summary prompt words to perform summary extraction on the requirement description documents to obtain requirement description texts; obtaining business function code blocks and unit test code blocks corresponding to the business function code blocks from the software source code based on the requirement description texts; and calling the large-scale language model and pre-set retrieval prompt words to retrieve the test plan and use case documents to obtain test case paragraphs matched with the requirement description texts; obtaining abstract syntax tree serialized texts of the business function code blocks and abstract syntax tree serialized texts of the unit test code blocks; taking the requirement description texts, the test case paragraphs, the unit test code blocks, the business function code blocks, the abstract syntax tree serialized texts of the unit test code blocks and the abstract syntax tree serialized texts of the business function code blocks as the multiple natural supervision signals.

3. The method of claim 2, wherein, The multiple training tasks comprise: a training task of predicting abstract syntax tree serialized texts based on the business function code blocks and / or the unit test code blocks; a training task of predicting requirement description texts based on code block serialized text combinations, the code block serialized text combinations comprising a first combination and / or a second combination, the first combination comprising the business function code blocks and abstract syntax tree serialized texts corresponding to the business function code blocks, and the second combination comprising the unit test code blocks and abstract syntax tree serialized texts corresponding to the unit test code blocks; and a training task of predicting unit test code blocks based on function code blocks.

4. The method of claim 3, wherein, The multiple training tasks further comprise: a training task of predicting unit test code blocks based on the requirement description texts; and a training task of predicting test cases based on the requirement description texts, the function code blocks, abstract syntax tree serialized texts corresponding to the function code blocks, the unit test code blocks and abstract syntax tree serialized texts corresponding to the unit test code blocks.

5. The method of claim 4, wherein, The training task of predicting unit test code blocks based on the requirement description texts comprises: first, second, third and fourth subtasks executed in parallel; The first subtask is configured to predict a total abstract syntax tree sequence text based on the requirement description text, the second subtask is configured to predict a total code block based on the total abstract syntax tree sequence text, the third subtask is configured to predict a unit test code serialization text based on the total code block and the total abstract syntax tree sequence text, and the fourth subtask is configured to predict a unit test code based on the unit test code serialization text.

6. A test case generation method characterized by, Comprise: Obtaining input raw data, the input raw data being a code block or a requirement description text; Calling a large-scale language model trained based on the method of any one of claims 1-5 to perform multi-hop generation feature enhancement on the input raw data to obtain enhanced data; wherein, in the case that the input raw data is a code block, the large-scale language model performs semantic feature enhancement on the input raw data, and in the case that the input raw data is a requirement description text, the large-scale language model performs code feature enhancement on the input raw data; Based on the input raw data and the enhanced data, calling the large-scale language model to generate test cases.

7. A model training apparatus characterized by comprising: Comprise: A first obtaining module is configured to obtain software source code, requirement description documents and test plan and use case documents of the software source code; An extraction module is configured to extract a plurality of natural supervision signals based on the requirement description documents, the software source code and the test plan and use case documents, the natural supervision signals being used to represent code features of the code and human real intentions of the code; A construction module is configured to construct a plurality of training tasks based on the plurality of natural supervision signals; A training module is configured to fine-tune train a large-scale language model based on the plurality of training tasks and the plurality of natural supervision signals.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program, when executed by the processor, is arranged to perform the method of any one of claims 1 to 7. The processor executes the computer program to implement the steps of the method of any one of claims 1-5.

9. A computer readable storage medium having stored thereon computer programs / instructions, characterized in that, The computer program / instructions are executed by the processor to implement the steps of the method of any one of claims 1-5.

10. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instructions are executed by the processor to implement the steps of the method of any one of claims 1-5.

Images