Method and device for generating test case based on large model and electronic equipment

Abstract

The invention provides a method and device for generating a test case based on a large model and electronic equipment, and relates to the technical field of artificial intelligence such as natural language processing, large language models and deep learning. According to the specific implementation scheme, when a test case is executed to test a tested code, assertion error reporting information is obtained; generating a first prompt word based on at least one of the test case, the tested code and the assertion error report information; inputting the first cue word into the large model to obtain a first output result of the large model; the first output result comprises repair scheme information, and the large model is used for outputting the repair scheme information matched with the first cue word based on the first cue word; and performing multiple rounds of interactive dialogues with the large model based on the first output result to generate a test case of the tested code. By implementing the embodiment of the invention, the accuracy and the code coverage rate of the test case can be improved, the problem can be quickly positioned, the repair suggestion can be automatically generated, the manual intervention is reduced, and the test efficiency is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to the field of artificial intelligence technologies such as natural language processing, large language models, and deep learning, and specifically to a method, device, and electronic device for generating test cases based on a large model. Background Art

[0002] In the related art, unit test generation tools can usually generate test cases. However, when faced with assertion errors, they can only prompt the error location and error information, which requires developers to manually analyze and repair, increasing time cost and complexity. Summary of the invention

[0003] The present disclosure provides a method for generating test cases based on a large model.

[0004] According to a first aspect of the present disclosure, a method for generating test cases based on a large model is provided, comprising:

[0005] When executing test cases to test the code under test, obtain assertion error information;

[0006] Generate a first prompt word based on at least one of the test case, the tested code, and the assertion error information;

[0007] Inputting the first prompt word into the large model to obtain a first output result of the large model; the first output result includes repair solution information, and the large model is used to output the repair solution information matching the first prompt word based on the first prompt word;

[0008] Conduct multiple rounds of interactive dialogues with the large model based on the first output result to generate test cases for the code under test.

[0009] According to a second aspect of the present disclosure, there is provided a device for generating test cases based on a large model, comprising:

[0010] The first acquisition module is used to obtain assertion error information when executing the test case to test the tested code;

[0011] A first generating module, configured to generate a first prompt word based on at least one of the test case, the tested code, and the assertion error information;

[0012] A second acquisition module is used to input the first prompt word into a large model to obtain a first output result of the large model; the first output result includes repair solution information, and the large model is used to output the repair solution information matching the first prompt word based on the first prompt word;

[0013] The second generation module is used to conduct multiple rounds of interactive dialogues with the large model based on the first output result to generate test cases for the code under test.

[0014] According to a third aspect of the present disclosure, there is provided an intelligent agent, comprising:

[0015] An input module, used for receiving input information;

[0016] A processing module, configured to determine a target task based on the input information received by the input module, determine a large model based on the target task, and obtain output information by calling the large model to execute the method described in the first aspect;

[0017] An output module is used to output the output information obtained by the processing module.

[0018] According to a fourth aspect of the present disclosure, there is provided an electronic device, including:

[0019] at least one processor; and

[0020] a memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for generating test cases based on a large model as described in the first aspect above.

[0022] According to a fifth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is proposed, wherein the computer instructions are used to enable the computer to execute the method for generating test cases based on a large model as described in the first aspect above.

[0023] According to a sixth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of the method for generating test cases based on a large model according to the first aspect described above.

[0024] According to the technology disclosed in the present invention, not only can test cases be automatically generated, but also the source of errors can be automatically identified and repaired when assertions go wrong, without the need for developers to manually analyze and modify them. This greatly improves test efficiency, reduces test costs and complexity, and improves test case accuracy and code coverage.

[0025] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0027] Figure 1 It is a flowchart of a method for generating test cases based on a large model provided by an embodiment of the present disclosure;

[0028] Figure 2 It is a flowchart of a method for generating test cases based on a large model provided by an embodiment of the present disclosure;

[0029] Figure 3 It is a flowchart of a method for generating test cases based on a large model provided by an embodiment of the present disclosure;

[0030] Figure 4 is a flowchart of a method for generating test cases based on a large model provided by an embodiment of the present disclosure;

[0031] Figure 5 is a block diagram of a test case generating device provided by an embodiment of the present disclosure;

[0032] Figure 6 is a block diagram of a test case generating device provided by an embodiment of the present disclosure;

[0033] Figure 7 is a block diagram of a test case generating device provided by an embodiment of the present disclosure;

[0034] Figure 8 is a block diagram of a test case generating device provided by an embodiment of the present disclosure;

[0035] Fig. 9 is a block diagram of an electronic device for implementing an embodiment of the present disclosure.

[0036] Fig.10 It is a block diagram of the intelligent agent provided by the embodiment of the present disclosure. DETAILED DESCRIPTION

[0037] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0038] The embodiments of the present disclosure relate to artificial intelligence technology fields such as natural language processing, large models, and deep learning.

[0039] Artificial Intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence.

[0040] Natural Language Processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between humans and computers using natural language. It is a discipline that uses computer technology to analyze, understand and process natural language, that is, it uses computers as a powerful tool for language research, conducts quantitative research on language information with the support of computers, and provides language descriptions that can be used by humans and computers.

[0041] Large Language Model (LLM) refers to a deep learning model trained with a large amount of text data, which can generate natural language text or understand the meaning of language text. Large language models can handle a variety of natural language tasks, such as text classification, question answering, and dialogue, and are an important path to artificial intelligence.

[0042] Deep learning is the process of learning the inherent laws and representation levels of sample data. The information obtained in the learning process is of great help in interpreting data such as text, images, and sounds. The ultimate goal of deep learning is to enable machines to have analytical learning capabilities like humans and to recognize data such as text, images, and sounds.

[0043] An agent is an agent that can perceive the environment and take actions to achieve specific goals. It can be software, hardware or a system with autonomy, adaptability and interaction. The agent perceives changes in the environment (such as through sensors or data input), makes judgments and decisions based on the knowledge and algorithms it has learned, and then performs actions to affect the environment or achieve predetermined goals.

[0044] It should be noted that in the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0045] It should also be noted that in the embodiments of the present disclosure, certain software, components, models and other existing solutions in the industry may be mentioned, which should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present disclosure, but it does not mean that the applicant has or will necessarily use the solution.

[0046] In the related art, unit test generation tools can usually generate test cases. However, when faced with assertion errors, they can only prompt the error location and error information, which requires developers to manually analyze and repair, increasing time cost and complexity.

[0047] Based on this, the embodiments of the present disclosure provide a method, device, electronic device and storage medium for generating test cases based on a large model. By combining the tested code, test cases and assertion error information, an automatic repair mechanism can be provided to help developers quickly locate problems and automatically generate repair suggestions, thereby reducing manual intervention, improving test efficiency, and improving the accuracy of test cases and code coverage.

[0048] It should be noted that the execution subject of the method for generating test cases based on a large model in an embodiment of the present disclosure may be a test case generation device, which may be implemented by software and / or hardware, and which may be configured in an electronic device, which may include but is not limited to a terminal, a server, etc.

[0049] It is worth noting that the method for generating test cases based on a large model in the embodiment of the present disclosure can be implemented by an autonomous agent (AutoAgent) based on a large language model (LLM). The autonomous agent based on the large language model can automatically identify assertion errors and attempt to repair them through a multi-round feedback mechanism, thereby realizing test case generation and assertion repair.

[0050] The following describes the method, device and electronic device for generating test cases based on a large model according to an embodiment of the present disclosure with reference to the accompanying drawings.

[0051] Figure 1 FIG. 1 is a flow chart of a method for generating test cases based on a large model provided by an embodiment of the present disclosure. Figure 1 As shown, the method for generating test cases based on a large model may include but is not limited to the following steps.

[0052] In step 101, when executing a test case to test the code under test, assertion error information is obtained.

[0053] In some embodiments, the test case can be a unit test (also called module test) case, but is not limited to this. For example, the test case can also be an integration test (also called assembly test or joint test) case, etc. The present disclosure does not limit this and will not elaborate on it.

[0054] In some embodiments, the test case can be a test case automatically generated by an agent based on a large language model. Taking the test case as a unit test case as an example, the agent based on a large language model can automatically generate a unit test case. Exemplarily, in the unit test case generation process, the agent can locate the tested code segment and the tested file through code analysis, and further dig out the associated code call or attribute definition to reveal the interactive relationship and potential dependence inside the business code. For specific scenarios, domain knowledge can be combined so that the generated test case can be more business-oriented and accurate. The agent can also analyze the structure and pattern of existing use cases, and integrate effective test strategies (such as effective test cases) into the newly generated unit test, thereby enhancing the coverage and practicality of the generated test case.

[0055] In some embodiments, the code under test may be tested using a test case. When the test case is executed to test the code under test, a test execution result may be obtained, and assertion error information may be obtained from the test execution result.

[0056] In some embodiments, the assertion error information may include, but is not limited to, detailed information about the assertion failure, such as the difference between the expected value and the actual value. Optionally, the assertion error information may also include the error location, etc., to help automatically identify the source of the assertion error (or the problem) and repair it, which is not limited in this disclosure and will not be described in detail.

[0057] In step 102, a first prompt word is generated based on at least one of a test case, a tested code, and an assertion error message.

[0058] In some embodiments, when executing a test case to test the code under test, when an assertion error occurs, the assertion error information, the test case, and the code under test may be integrated to generate a first prompt word.

[0059] In some embodiments, the assertion error information, test case, and tested code can be rendered by template based on the prompt word template to obtain the first prompt word. Exemplarily, taking the test case as a unit test case as an example, when executing the unit test case to test the tested code, if the assertion error information is obtained, the assertion error information, the unit test case, and the tested code can be rendered into the prompt word template based on the prompt word template to obtain the first prompt word.

[0060] In some embodiments, the first prompt word may be used to guide the large language model to analyze the cause of the assertion error (also called assertion failure), and guide the large language model to output repair solution information matching the first prompt word.

[0061] In step 103, the first prompt word is input into the large model to obtain a first output result of the large model.

[0062] In some embodiments, the large model may be a large language model. Exemplarily, the large model is a deep learning model that has been trained, so that the large model has learned the ability to analyze the cause (or source) of assertion errors (also called assertion failures) and provide corresponding repair solution information.

[0063] In some embodiments, the first output result may include repair solution information, and the large model is used to output the repair solution information matching the first prompt word based on the first prompt word.

[0064] Exemplarily, the first prompt word may be input into the large model, and the first prompt word may be used to guide the large model to analyze the cause of the assertion error (also called assertion failure), and guide the large model to output repair solution information matching the first prompt word.

[0065] In step 104, multiple rounds of interactive dialogues are conducted with the large model based on the first output result to generate test cases for the code under test.

[0066] In some embodiments, the first prompt word can guide the large model to determine the source of the assertion error, and guide the large model to output the repair solution information matching the first prompt word. If the source of the assertion error is an assertion content error, the first output result can include the test case repaired by the large model, and the test case repaired by the large model can be executed, and the above step 101 can be returned to execute, that is, the step of obtaining the assertion error information when executing the test case to test the tested code is returned to execute the above step, so as to conduct the next round of interactive dialogue with the large model, and the cycle is repeated until the repaired test case passes or the tested code defect is determined.

[0067] Exemplarily, after the test case is repaired for the first time through the large model, the repaired test case can be executed again. If the assertion error information can still be obtained, a new prompt word is generated according to at least one of the new assertion error information, the test case, and the code under test, and a second round of analysis and repair is performed based on the newly generated prompt word. This cycle continues until the repaired test case passes or the defect in the code under test is determined.

[0068] In the above embodiment, by introducing a multi-round feedback verification mechanism, when an assertion failure (also called an assertion error) occurs, by reorganizing at least one of the test case, the code under test, and the assertion error information into prompt words, the large model is guided to analyze the cause of the assertion error and provide corresponding repair plan information. Not only can the test case be automatically generated, but also the source of the error can be automatically identified and repaired when an assertion error occurs, without the need for developers to manually analyze and modify, which greatly improves the test efficiency, reduces the test cost and complexity, and improves the accuracy of the test cases and the code coverage.

[0069] Figure 2 FIG. 1 is a flow chart of a method for generating test cases based on a large model provided by an embodiment of the present disclosure. Figure 2 As shown, the method for generating test cases based on a large model may include but is not limited to the following steps.

[0070] In step 201, when executing the test case to test the code under test, assertion error information is obtained.

[0071] In the embodiments of the present disclosure, step 201 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0072] In step 202, based on the assertion error information, a use case at a corresponding position is determined from the test cases as a use case to be verified.

[0073] Exemplarily, the assertion error information may include an error location, etc., and based on the assertion error information, a use case at a corresponding location may be determined from the test cases as a use case to be verified.

[0074] In step 203, based on the assertion error information, the code at the corresponding position is determined from the tested code as the code to be verified.

[0075] Exemplarily, the assertion error information may include an error location, etc., and based on the assertion error information, the code at the corresponding location in the tested code may be determined as the code to be verified.

[0076] It should be noted that, in some embodiments, step 202 and step 203 may be performed in an exchanged order or simultaneously. For example, step 202 may be performed first and then step 203, or step 203 may be performed first and then step 202, or step 202 and step 203 may be performed simultaneously.

[0077] In step 204, based on the prompt word template, the use case to be verified, the code to be verified and the assertion error information are rendered by template to obtain a first prompt word.

[0078] Exemplarily, based on the prompt word template, the use case to be verified, the code to be verified and the assertion error information can be rendered into the prompt word template to obtain the first prompt word. Exemplarily, the above first prompt word can be used to guide the large language model to analyze the cause of the assertion error (also called assertion failure) and guide the large language model to output the repair solution information matching the first prompt word.

[0079] In step 205, the first prompt word is input into the large model to obtain a first output result of the large model; the first output result includes repair solution information, and the large model is used to output the repair solution information matching the first prompt word based on the first prompt word.

[0080] In the embodiments of the present disclosure, step 205 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0081] In step 206, multiple rounds of interactive dialogues are conducted with the large model based on the first output result to generate test cases for the code under test.

[0082] In the embodiments of the present disclosure, step 206 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0083] In the above embodiment, based on the assertion error information, the use case at the corresponding position can be determined from the test case as the use case to be verified, and the code at the corresponding position can be determined from the tested code as the code to be verified. Based on the prompt word template, the use case to be verified, the code to be verified and the assertion error information are template rendered to obtain a more accurate first prompt word. The first prompt word is used to guide the large model to analyze the cause of the assertion error (also called assertion failure), and guide the large language model to output the repair solution information matching the first prompt word, which can further improve the accuracy of the test case and the code coverage.

[0084] Figure 3 FIG. 1 is a flow chart of a method for generating test cases based on a large model provided by an embodiment of the present disclosure. Figure 3 As shown, the method for generating test cases based on a large model may include but is not limited to the following steps.

[0085] In step 301, when executing the test case to test the code under test, assertion error information is obtained.

[0086] In the embodiments of the present disclosure, step 301 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0087] In step 302, a first prompt word is generated based on at least one of a test case, a tested code, and an assertion error message.

[0088] In the embodiments of the present disclosure, step 302 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0089] In step 303, the first prompt word is input into the large model, and the large model is used to compare the context of the tested code to determine the source of the assertion error.

[0090] Exemplarily, the first prompt word may be input into a trained large model, and the large model may compare the context of the tested code to analyze the source of the assertion error and distinguish whether it is caused by a defect in the tested code or an error in the assertion content.

[0091] In some embodiments, the input of the large model is a prompt word. If the large model identifies that the source of the assertion error message is a defect in the tested code, the output of the large model is a detailed description of the code defect and repair suggestions. If the large model identifies that the source of the assertion error message is an assertion error, the output of the large model is the assertion content in the automatically repaired test case.

[0092] In step 304, based on the first prompt word, the large model generates repair solution information corresponding to the source of the assertion error, and the first output result includes the repair solution information. Exemplarily, the generated repair solution information is used as the first output result of the large model.

[0093] In some embodiments, when the source of the assertion error is a defect in the tested code, the code defect description information and repair suggestion information can be generated through the large model based on the defect in the tested code, and the code defect description information and repair suggestion information are used as the repair solution information corresponding to the defect in the tested code. Exemplarily, if the source of the assertion error is determined to be a defect in the tested code through the large model, the large model can generate detailed code defect description information and repair suggestion information to facilitate the auxiliary developer to repair the tested code based on the code defect description information and repair suggestion information.

[0094] In some embodiments, when the source of the assertion error is a defect in the tested code, the test case can be stored on disk as a valid test case. Exemplarily, if the source of the assertion error is a defect in the tested code, it can be considered that the test case has no problem (or it can be considered that the test case is correct), and the test case can be stored on disk as a valid test case. In this case, the storage on disk can refer to storing the test case, such as persisting the test case.

[0095] In some embodiments, when the source of the assertion error is an assertion content error, the assertion content in the test case can be repaired through the big model based on the tested code and the test case, and the repair information of the assertion content by the big model is used as the repair solution information corresponding to the assertion content error. Exemplarily, if the source of the assertion error is determined to be an assertion content error through the big model, the big model automatically repairs the assertion content in the test case to achieve the repair of the test case.

[0096] In step 305, multiple rounds of interactive dialogues are conducted with the large model based on the first output result to generate test cases for the code under test.

[0097] Exemplarily, when the source of the assertion error is an error in the assertion content, that is, when the test case fails, the assertion content in the test case is automatically repaired through the big model, and the test case repaired by the big model is executed, and the process returns to execute the above step 301, that is, returns to execute the above step of obtaining the assertion error information when executing the test case to test the code under test, so as to have the next round of interactive dialogue with the big model, and repeats this cycle until the repaired test case passes or the defect of the code under test is determined.

[0098] In some embodiments, during the process of executing the repaired test case to test the tested code, the test case that passed the test can be stored on the disk as a valid test case. Exemplarily, if the test case passes, it can be considered that the test case is correct and is a valid test case, and the valid test case can be stored on the disk.

[0099] In some embodiments, during multiple rounds of interactive dialogue with the big model, the repair solution information provided by the big model is obtained; the big model is trained based on the repair solution information, thereby optimizing the big model and improving the recognition and repair efficiency of future assertion errors.

[0100] For example, Figure 4 As shown, taking the test case as a single test case as an example, the method for generating test cases based on a large model involved in the embodiment of the present disclosure can be divided into the following steps:

[0101] Step 1) Error acquisition and prompt word generation:

[0102] When executing the automatically generated unit test, the assertion error information is obtained, and the assertion error information, the unit test case and the tested code are integrated to generate a prompt word.

[0103] Step 2) Error judgment and repair:

[0104] The prompt word is input into a trained large model (such as LLM), which analyzes the cause of the assertion error and distinguishes whether it is a defect (bug) in the tested code or an error in the assertion content. If the cause of the assertion error is a defect in the tested code (bug), the large model generates a detailed description of the code defect and repair suggestions, and stores the test case on disk. Exemplarily, in this case, the output of the large model may include the cause of the assertion error being a defect in the tested code (bug), as well as a description of the code defect and repair suggestions. If the cause of the assertion error is an error in the assertion content, the assertion content in the test case can be automatically repaired by the large model. In this case, the output of the large model may include the cause of the assertion error being an error in the assertion content, and the modified test case code. After the test case repair is completed, the test can be executed again.

[0105] Step 3) Multiple rounds of feedback verification:

[0106] After the first repair, the repaired test case can be executed again. If there is still an assertion error message, a new prompt word is generated based on the tested code, test case, and new assertion error message, and a second round of analysis and repair is performed. This cycle continues until the test case passes or the specific defect (bug) is identified.

[0107] Step 4) Model upgrade and self-learning:

[0108] Repair solution information can be collected during multiple rounds of feedback, and this data can be used to further optimize the large model and improve the efficiency of identifying and repairing future assertion errors.

[0109] In the above embodiment, by introducing the assertion repair function, the manual intervention of the developer is greatly reduced, the test generation tool is extended from simple use case generation to assertion repair, and the quality of the generated test is improved through multiple rounds of feedback. By reducing the manual repair time of developers, the present disclosure can reduce development costs and shorten the test cycle, and is particularly suitable for testing scenarios of large-scale complex projects. In addition, multiple rounds of feedback and automatic repair mechanisms can greatly improve the coverage and accuracy of unit tests, helping the development team to discover and fix potential code problems more quickly.

[0110] Figure 5 is a block diagram of a test case generation device provided by an embodiment of the present disclosure. Figure 5 As shown, the test case generating device may include but is not limited to: a first acquisition module 501 , a first generation module 502 , a second acquisition module 503 and a second generation module 504 .

[0111] The first acquisition module 501 is used to acquire assertion error information when executing a test case to test the code under test.

[0112] The first generating module 502 is used to generate a first prompt word based on at least one of a test case, a tested code, and an assertion error message.

[0113] The second acquisition module 503 is used to input the first prompt word into the large model to obtain a first output result of the large model; the first output result includes repair solution information, and the large model is used to output the repair solution information matching the first prompt word based on the first prompt word.

[0114] The second generating module 504 is used to conduct multiple rounds of interactive dialogues with the large model based on the first output result to generate test cases for the tested code.

[0115] In some embodiments, the first generation module 502 is used to: perform template rendering on the test case, the tested code and the assertion error information based on the prompt word template to obtain the first prompt word.

[0116] In some embodiments, the first generation module 502 is used to: based on the assertion error information, determine the use case at the corresponding position from the test case as the use case to be verified; based on the assertion error information, determine the code at the corresponding position from the tested code as the code to be verified; based on the prompt word template, perform template rendering on the use case to be verified, the code to be verified and the assertion error information to obtain the first prompt word.

[0117] In some embodiments, the second acquisition module 503 is used to: input the first prompt word into the large model, and compare the context of the tested code through the large model to determine the source of the assertion error; based on the first prompt word, generate repair solution information corresponding to the source of the assertion error through the large model, and the first output result includes the generated repair solution information.

[0118] In some embodiments, the second acquisition module 503 is used to: assert that the source of the error is a defect in the code under test, and based on the defect in the code under test, generate code defect description information and repair suggestion information through a large model; the repair solution information includes code defect description information and repair suggestion information repair solution information.

[0119] In some embodiments, Figure 6 As shown, the test case generation device may further include a first storage module 605. The first storage module 605 is used to assert that the source of the error is a defect in the tested code and store the test case as a valid test case. Figure 6 601-604 and Figure 7 501-504 have the same function and structure.

[0120] In some embodiments, the second acquisition module 603 is used to: the source of the assertion error is an error in the assertion content, and based on the tested code and the test case, the assertion content in the test case is repaired by the big model; the repair solution information includes the repair information of the big model for the assertion content.

[0121] In some embodiments, the second generation module 604 is used to: if the test case fails, execute the test case that has been repaired by the large model, and return to the step of obtaining assertion error information when executing the test case to test the code under test, so as to conduct the next round of interactive dialogue with the large model.

[0122] In some embodiments, Figure 7 As shown, the test case generation device may also include: a second storage module 706. The second storage module 706 is used to store the passed test cases as valid test cases in the process of executing the repaired test cases to test the tested code. Optionally, in some embodiments, the second storage module 706 and the first storage module 705 can be independent storage modules respectively; or, the second storage module 706 and the first storage module 705 can be the same storage module; or, the second storage module 706 and the first storage module 705 can be concentrated on the same storage module. Figure 7 701-705 and Figure 6 601-605 have the same function and structure.

[0123] In some embodiments, Figure 8 As shown, the test case generation device may further include: a third acquisition module 807 and a training module 808. The third acquisition module 807 is used to acquire the repair solution information provided by the large model during multiple rounds of interactive dialogue with the large model; the training module 808 is used to train the large model based on the repair solution information. Figure 8 801-806 and Figure 7 701-706 have the same function and structure.

[0124] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0125] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0126] like Fig. 9, is a block diagram of an electronic device according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0127] like Fig. 9 As shown, the electronic device includes: one or more processors 901, a memory 902, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig. 9 A processor 901 is taken as an example.

[0128] The memory 902 is a non-transitory computer-readable storage medium provided by the present disclosure. The memory stores instructions executable by at least one processor to enable the at least one processor to perform the method for generating test cases based on a large model provided by the present disclosure. The non-transitory computer-readable storage medium of the present disclosure stores computer instructions, which are used to enable a computer to perform the method for generating test cases based on a large model provided by the present disclosure.

[0129] The memory 902 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the method for generating test cases based on a large model in the embodiment of the present disclosure (for example, the attached Figure 5 The processor 901 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 902, that is, the method for generating test cases based on the large model in the above method embodiment is implemented.

[0130] The memory 902 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 902 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 902 may optionally include a memory remotely arranged relative to the processor 901, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0131] The electronic device may further include: an input device 903 and an output device 904. The processor 901, the memory 902, the input device 903 and the output device 904 may be connected via a bus or other means. Fig. 9 The example of connecting through bus is taken in the following.

[0132] The input device 903 can receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator rod, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 904 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0133] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0134] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0135] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0136] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0137] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0138] Fig.10 A block diagram of an intelligent agent provided in an embodiment of the present disclosure. Fig.10 As shown, the agent may include an input module 1001, a processing module 1002 and an output module 1003. The input module 1001 is used to receive input information; the processing module 1002 is used to determine the target task based on the input information received by the input module, determine the big model based on the target task, and obtain output information by calling the big model to execute the method for generating test cases based on the big model in the above method embodiment; the output module 1003 is used to output the output information obtained by the processing module.

[0139] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0140] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for generating test cases based on a large model, comprising: When executing test cases to test the code under test, obtain assertion error information; Generate a first prompt word based on at least one of the test case, the tested code, and the assertion error information; Inputting the first prompt word into a large model to obtain a first output result of the large model; The first output result includes repair solution information, and the large model is used to output the repair solution information matching the first prompt word based on the first prompt word; Conduct multiple rounds of interactive dialogues with the large model based on the first output result to generate test cases for the code under test.

2. The method of claim 1, wherein: The generating a first prompt word based on at least one of the test case, the tested code, and the assertion error information includes: Based on a prompt word template, the test case, the tested code and the assertion error information are template rendered to obtain the first prompt word.

3. The method of claim 2, wherein: The step of performing template rendering on the test case, the tested code and the assertion error information based on the prompt word template to obtain the first prompt word includes: Based on the assertion error information, determining a use case at a corresponding position from the test cases as a use case to be verified; Based on the assertion error information, determine the code at the corresponding position from the tested code as the code to be verified; Based on the prompt word template, the use case to be verified, the code to be verified and the assertion error information are template rendered to obtain the first prompt word.

4. The method of claim 1, wherein: The step of inputting the first prompt word into a large model to obtain a first output result of the large model includes: Inputting the first prompt word into the large model, and comparing the tested code context through the large model to determine the source of the assertion error; Based on the first prompt word, repair solution information corresponding to the source of the assertion error is generated through the large module, and the first output result includes the repair solution information.

5. The method of claim 4, wherein: The generating, based on the first prompt word, repair solution information corresponding to the source of the assertion error through the large module includes: In the case where the source of the assertion error is the defect of the tested code, based on the defect of the tested code, code defect description information and repair suggestion information are generated through the large model; The repair solution information includes: the code defect description information and the repair suggestion information;.

6. The method according to claim 4 or 5, wherein: The method of generating repair solution information corresponding to the source of the assertion error through the large module based on the first prompt word also includes: In the case where the source of the assertion error is a defect in the tested code, the test case is stored on disk as a valid test case.

7. The method of claim 4, wherein: The generating, based on the first prompt word, repair solution information corresponding to the source of the assertion error through the large module includes: In the case where the source of the assertion error is an assertion content error, based on the tested code and the test case, the assertion content in the test case is repaired by the large model; The repair solution information includes repair information of the assertion content by the large model.

8. The method of claim 7, wherein: The step of conducting multiple rounds of interactive dialogues with the large model based on the first output result to generate test cases for the code under test includes: In the case that the test case fails, the test case repaired by the large model is executed, and the step of obtaining assertion error information when executing the test case to test the tested code is returned to perform the next round of interactive dialogue with the large model.

9. The method of claim 8, further comprising: In the process of executing the repaired test cases to test the tested code, the test cases that pass the test are stored on the disk as valid test cases.

10. The method of claim 1, further comprising: During multiple rounds of interactive dialogue with the large model, the restoration solution information is obtained; The large model is trained based on the repair solution information.

11. A device for generating test cases based on a large model, comprising: The first acquisition module is used to obtain assertion error information when executing the test case to test the tested code; A first generating module, configured to generate a first prompt word based on at least one of the test case, the tested code, and the assertion error information; A second acquisition module, used for inputting the first prompt word into a large model to obtain a first output result of the large model; The first output result includes repair solution information, and the large model is used to output the repair solution information matching the first prompt word based on the first prompt word; The second generation module is used to conduct multiple rounds of interactive dialogues with the large model based on the first output result to generate test cases for the code under test.

12. The device of claim 11, wherein: The first generating module is used for: Based on a prompt word template, the test case, the tested code and the assertion error information are template rendered to obtain the first prompt word.

13. The device of claim 12, wherein: The first generating module is used for: Based on the assertion error information, determining a use case at a corresponding position from the test cases as a use case to be verified; Based on the assertion error information, determine the code at the corresponding position from the tested code as the code to be verified; Based on the prompt word template, the use case to be verified, the code to be verified and the assertion error information are template rendered to obtain the first prompt word.

14. The device of claim 11, wherein: The acquisition module is used for: Inputting the first prompt word into the large model, and comparing the tested code context through the large model to determine the source of the assertion error; Based on the first prompt word, repair solution information corresponding to the source of the assertion error is generated through the large module, and the first output result includes the repair solution information.

15. The device of claim 14, wherein: The second acquisition module is used for: In the case where the source of the assertion error is the defect of the tested code, based on the defect of the tested code, code defect description information and repair suggestion information are generated through the large model; The repair solution information includes: the code defect description information and the repair suggestion information.

16. The device according to claim 14 or 15, further comprising: The first storage module is used to store the test case as a valid test case on a disk when the source of the assertion error is a defect in the tested code.

17. The device of claim 14, wherein: The second acquisition module is used for: In the case where the source of the assertion error is an assertion content error, based on the tested code and the test case, the assertion content in the test case is repaired by the large model; The repair solution information includes repair information of the assertion content by the large model.

18. The device of claim 17, wherein: The second generating module is used for: In the case that the test case fails, the test case repaired by the large model is executed, and the step of obtaining assertion error information when executing the test case to test the tested code is returned to perform the next round of interactive dialogue with the large model.

19. The apparatus of claim 18, further comprising: The second storage module is used to store the passed test cases as valid test cases in the disk during the process of executing the repaired test cases to test the tested code.

20. The apparatus of claim 11, further comprising: A third acquisition module is used to acquire the repair solution information provided by the large model during multiple rounds of interactive dialogue with the large model; A training module is used to train the large model based on the repair solution information.

21. An intelligent agent, comprising: An input module, used for receiving input information; a processing module, configured to determine a target task based on the input information received by the input module, determine a large model based on the target task, and obtain output information by calling the large model to execute the method according to any one of claims 1 to 10; An output module is used to output the output information obtained by the processing module.

22. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method for generating test cases based on a large model according to any one of claims 1 to 10.

23. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable the computer to execute the method for generating test cases based on a large model according to any one of claims 1 to 10.

24. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the computer program implements the steps of the method for generating test cases based on a large model according to any one of claims 1 to 10.

Images