Test case generation method and device based on large model and intelligent agent

Abstract

The invention provides a test case generation method and device based on a large model and an intelligent agent, and relates to the technical field of artificial intelligence, in particular to the technical field of software testing and large models. According to the specific implementation scheme, the method comprises the steps of obtaining historical data of a project, wherein the historical data at least comprises at least one of a historical test case, a historical code and a historical project document; performing knowledge extraction on the historical data through a natural language processing NLP model, and generating a project knowledge graph based on the extracted knowledge; obtaining test demand information of the project; and generating a test case set of the project through the large model according to the test demand information and the project knowledge graph.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to the fields of software testing and large model technology, and particularly to a method, device and agent for generating test cases based on a large model. Background Art

[0002] Currently, for the testing process of various projects, it still mainly relies on testers to manually write test cases and manually execute test cases, check system functions, verify user experience, etc. In the online stage, it is necessary to manually deploy code, configure the environment, and monitor the running stability, resulting in high costs and low efficiency for project testing and online. Summary of the Invention

[0003] The present disclosure provides a method, device and agent for generating test cases based on a large model.

[0004] According to one aspect of the present disclosure, there is provided a method for generating test cases based on a large model, including: obtaining historical materials of a project, the historical materials including at least one of historical test cases, historical code, and historical project documents; performing knowledge extraction on the historical materials through a natural language processing NLP model, and generating a project knowledge graph based on the extracted knowledge; obtaining test requirement information of the project; generating a set of test cases for the project through a large model according to the test requirement information and the project knowledge graph.

[0005] According to another aspect of the present disclosure, there is provided a device for generating test cases based on a large model, including: a first obtaining module for obtaining historical materials of a project, the historical materials including at least one of historical test cases, historical code, and historical project documents; a first generating module for performing knowledge extraction on the historical materials through a natural language processing NLP model, and generating a project knowledge graph based on the extracted knowledge; a second obtaining module for obtaining test requirement information of the project; a second generating module for generating a set of test cases for the project through a large model according to the test requirement information and the project knowledge graph.

[0006] According to another aspect of the present disclosure, an agent is provided, including: an input module for receiving input information; the input information includes historical materials and test requirement information of a project, where the historical materials at least include at least one of historical test cases, historical code, and historical project documents; a processing module for determining a target task based on the input information, where the historical materials of the project correspond to a first target task, the test requirement information corresponds to a second target task, calling an NLP model based on the first target task, performing knowledge extraction on the historical materials through the NLP model, generating a project knowledge graph based on the extracted knowledge, and calling a large model according to the second target task, generating a set of test cases for the project through the large model based on the test requirement information and the project knowledge graph as output information; an output module for outputting the output information obtained by the processing module.

[0007] According to another aspect of the present disclosure, an electronic device is provided, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for generating test cases based on a large model described in the above-mentioned embodiment of one aspect.

[0008] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, on which a computer program / instructions are stored, and the computer instructions are used to cause the computer to execute the method for generating test cases based on a large model described in the above-mentioned embodiment of one aspect.

[0009] According to another aspect of the present disclosure, a computer program product is provided, including computer program / instructions, and when the computer program / instructions are executed by a processor, the method for generating test cases based on a large model described in the above-mentioned embodiment of one aspect is implemented.

[0010] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. Description of the Drawings

[0011] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0012] Figure 1 It is a schematic flowchart of a method for generating test cases based on a large model provided by an embodiment of the present disclosure;

[0013] Figure 2Schematic flowchart of another test case generation method based on a large model provided by an embodiment of the present disclosure;

[0014] Figure 3 Schematic flowchart of another test case generation method based on a large model provided by an embodiment of the present disclosure;

[0015] Figure 4 Schematic flowchart of the process of online deployment of a project in a test case generation method based on a large model provided by an embodiment of the present disclosure;

[0016] Figure 5 Schematic diagram of the structure of an agent provided by an embodiment of the present disclosure;

[0017] Figure 6 Schematic flowchart of generating test cases based on a large model provided by an embodiment of the present disclosure;

[0018] Figure 7 Schematic diagram of the structure of a test case generation device based on a large model provided by an embodiment of the present disclosure;

[0019] Figure 8 Block diagram of an electronic device for implementing the test case generation method based on a large model of an embodiment of the present disclosure. Detailed implementation manners

[0020] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted below.

[0021] The following describes a test case generation method, device, and agent based on a large model according to embodiments of the present disclosure with reference to the accompanying drawings.

[0022] Artificial Intelligence (AI) is a discipline that studies how to make a computer simulate certain thinking processes and intelligent behaviors of humans (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level technologies and software-level technologies. Artificial intelligence hardware technologies generally include several aspects such as computer vision technology, speech recognition technology, natural language processing technology, machine learning / deep learning, big data processing technology, and knowledge graph technology.

[0023] Figure 1 Schematic flowchart of a test case generation method based on a large model provided by an embodiment of the present disclosure.

[0024] As Figure 1 shown, the test case generation method based on a large model may include:

[0025] S101. Obtain the historical materials of the project, where the historical materials include at least one of historical test cases, historical code, and historical project documents.

[0026] It should be noted that, in the embodiments of the present disclosure, the execution subject of the test case generation method based on a large model may be a hardware device with data processing capabilities and / or the necessary software for driving the hardware device to work. Optionally, the execution subject may include a server, a user terminal, and other intelligent devices. Optionally, the user terminal includes, but is not limited to, mobile phones, computers, intelligent voice interaction devices, etc. Optionally, the server includes, but is not limited to, web servers, application servers, and may also be a server of a distributed system or a server combined with a blockchain. The embodiments of the present disclosure do not make specific limitations.

[0027] It can be understood that after the project is tested, project information such as test cases, code, and project documents used during the project test can be stored. That is, at least one of historical test cases, historical code, and historical project documents can be obtained from the storage space storing project information as the historical materials of the project.

[0028] In some embodiments, historical test cases, historical code, and historical project documents can be stored in different storage spaces, and by accessing different storage spaces, the historical materials of the project can be obtained.

[0029] Optionally, the historical project documents may include historical test reports of the project, historical test requirement information, historical alarm information of the project, historical operation log information of the project, etc.

[0030] In some embodiments, in order to improve data quality, after obtaining the historical materials of the project, the historical materials can be preprocessed. By performing preprocessing operations such as cleaning, deduplication, and formatting on the historical materials, the format of the historical materials can be ensured to be unified.

[0031] S102. Perform knowledge extraction on the historical materials through a natural language processing (NLP) model, and generate a project knowledge graph based on the extracted knowledge.

[0032] In the embodiments of the present disclosure, through a natural language processing (Natural Language Processing, NLP) model, using NLP technology, knowledge extraction can be performed on the historical materials, and a project knowledge graph can be generated based on the extracted knowledge.

[0033] In some embodiments, an NLP model can be used to perform operations such as entity recognition and relationship extraction on historical materials to achieve knowledge extraction. That is, a large language model can extract information such as entities and the relationships between entities from historical materials as the extracted knowledge. Among them, the entities can be at least one of historical test cases, historical code, and historical project documents, and the relationships between entities can be dependencies between historical codes and associations between historical project documents.

[0034] In some embodiments, the extracted knowledge can be combined in the form of a graph structure to generate a project knowledge graph. Among them, the graph structure is a data structure composed of nodes and edges. In the graph structure, nodes represent entities, and edges represent the relationships between entities.

[0035] That is, by using the extracted entities as nodes of the graph structure and the relationships between entities as edges of the graph structure, a graph structure is constructed, and then a project knowledge graph is generated based on the constructed graph structure.

[0036] In some embodiments, the generated project knowledge graph can also be verified and optimized to improve the accuracy and integrity of the project knowledge graph. Optionally, by performing accuracy verification and consistency check on the entities and entity relationships in the project knowledge graph, the accuracy of the project knowledge graph can be improved. By verifying whether the nodes in the project knowledge graph contain all the entities in the historical materials, the integrity verification of the project knowledge graph can be achieved. Optionally, the optimization of the project knowledge graph can be realized by eliminating duplicate entities in the project knowledge graph.

[0037] S103, obtain the test requirement information of the project.

[0038] In some embodiments, the test requirement information of the project can be obtained by receiving the test requirement information sent by the client. The user inputs requirement information on the client, and the client takes the requirement information input by the user as the test requirement information and sends it to the large model.

[0039] In some embodiments, the user can input information such as the test objectives and test scopes corresponding to the project as the test requirement information. For example, the test objectives can be functional test objectives, performance test objectives, compatibility test objectives, etc. That is, the test requirement information includes at least one of the functional test objective, performance test objective, and compatibility test objective.

[0040] Optionally, the test scope refers to the project functions, features, modules, code segments, and related non-functional requirements that need to be covered and verified during the project testing process. Based on the test scope, the key points of the project testing can be clarified to improve the comprehensiveness of the project testing and effectively evaluate the quality and stability of the project.

[0041] S104. Generate a set of test cases for the project through a large model based on the test requirement information and the project knowledge graph.

[0042] In some embodiments, the large model can search in the project knowledge graph according to the test requirement information, and obtain at least one of historical test cases, historical code, and historical project documents corresponding to the test requirement information as the search result. Further, based on the test requirement information and the search result, the large model can generate multiple test cases for the project, and thus obtain the set of test cases.

[0043] According to the method for generating test cases based on a large model provided by the embodiments of the present disclosure, by obtaining the historical materials of the project and performing knowledge extraction on the historical materials through the large model, a project knowledge graph can be generated based on the extracted knowledge. Further obtain the test requirement information of the project, and generate a set of test cases for the project through the large model according to the test requirement information and the project knowledge graph. Thus, by generating test cases through the large model, the efficiency of generating test cases can be improved, and the test efficiency of the project can be further improved. Using the large model in the process of generating test cases can reduce the test cost.

[0044] In some embodiments, after receiving the test requirement information, the large model can parse the test requirement information to obtain the test objective and test scope, and search in the project knowledge graph according to the test objective and test scope to obtain the search result.

[0045] In some embodiments, in order to enable the generated test cases to meet various test scenarios, the test scenarios and boundary conditions that the test cases need to cover can also be obtained, and then the test scenarios and boundary conditions are input into the large model to search in the project knowledge graph according to the test requirement information, test scenarios, and boundary conditions.

[0046] In some embodiments, in order to improve the search efficiency, prompt words can be generated according to the test requirement information, test scenarios, and boundary conditions, so that the large model can generate a set of test cases for the project according to the prompt words and the project knowledge graph.

[0047] In some embodiments, the quality of the set of test cases generated by the large model according to the test requirement information and the project knowledge graph can be evaluated, and after the set of test cases passes the quality evaluation, the set of test cases is sent to the client.

[0048] Optionally, the quality of the generated set of test cases can be evaluated according to a preset evaluation criterion. If the generated set of test cases meets the preset evaluation criterion, it is determined that the set of test cases passes the quality evaluation.

[0049] In some embodiments, after the user receives a set of test cases for a project based on the client, the user can also input optimization requirements for the test cases on the client, so that the large model optimizes the set of test cases according to the optimization requirements and sends the optimized set of test cases to the client.

[0050] In some embodiments, after generating a set of test cases for a project, the set of test cases can be executed, and the large model can generate a test report based on the execution results. Further, the large model can perform online deployment of the project according to the test report, thereby realizing the automation of project testing and going live.

[0051] In the embodiments of the present disclosure, after generating test cases, online deployment can be automatically performed based on the test cases, realizing efficient and intelligent project testing and going live, which is beneficial to the rational utilization of resources, thereby improving the efficiency of project going live and reducing the time cost.

[0052] Figure 2 It is a schematic flowchart of a method for generating test cases based on a large model provided by an embodiment of the present disclosure.

[0053] As Figure 2 shown, the method for generating test cases based on a large model may include:

[0054] S201, obtaining historical materials of the project, where the historical materials include at least one of historical test cases, historical code, and historical project documents.

[0055] S202, performing knowledge extraction on the historical materials through an NLP model and generating a project knowledge graph based on the extracted knowledge.

[0056] S203, obtaining test requirement information of the project.

[0057] For the relevant content of steps S201 - S203, reference can be made to the above embodiments and will not be elaborated here.

[0058] S204, parsing the test requirement information through the large model to obtain the scope to be tested and the corresponding test objectives of the project.

[0059] In some embodiments, the large model can use NLP technology to parse the test requirement information to obtain the scope to be tested and the corresponding test objectives of the project in the test requirement information. For example, the large model determines the information such as the functions and modules to be tested carried in the test requirement information as the scope to be tested, and determines the results to be achieved in the test carried in the test requirement information as the test objectives corresponding to the scope to be tested.

[0060] S205. Search in the project knowledge graph according to the to-be-tested scope and the corresponding test objective to obtain target search information.

[0061] In some embodiments, the large model can search for historical test cases, historical code, and historical project documents in the project knowledge graph according to the to-be-tested scope and the corresponding test objective, so as to obtain target search information.

[0062] The target search information includes at least one of the first historical test case, the first historical code, and the first historical project document. That is to say, according to the to-be-tested scope and the corresponding test objective, searching for entities and entity relationships in the project knowledge graph can obtain at least one of the first historical test case, the first historical code, and the first historical project document as the target search information.

[0063] S206. Generate a test case set for the project according to the test requirement information and the target search information.

[0064] In some embodiments, the large model can generate a first test case set according to the test requirement information and the target search information, and perform quality evaluation on the test cases in the first test case set to obtain quality evaluation information of the test cases.

[0065] In some embodiments, the evaluation criteria for the preset quality evaluation can be determined, and the test cases in the first test case set are quality-evaluated according to the evaluation criteria to obtain quality evaluation information. Among them, the quality evaluation information indicates the test cases in the first test case set that pass the quality evaluation.

[0066] In some embodiments, the evaluation criteria can be determined according to whether the test case can run successfully and the running effect of the successful run, and the test cases that can run successfully and have a good running effect are used as the test cases that meet the evaluation criteria, that is, the test cases that pass the quality evaluation. Among them, a good running effect can be that the coverage rate of the test case running is greater than the coverage rate threshold, or the test efficiency of the test case is greater than the efficiency threshold.

[0067] Furthermore, according to the quality evaluation information, a second test case set is determined from the first test case set. Optionally, according to the quality evaluation information, the test cases that pass the quality evaluation can be determined from the first test case set, and the second test case set is generated according to the test cases that pass the quality evaluation.

[0068] In some embodiments, after the large model generates the second set of test cases, it can send the second set of test cases to the client, receive the use case optimization information sent by the client, and optimize and adjust the test cases according to the use case optimization information and the historical conversation information of this round to generate the set of test cases for the project.

[0069] In the embodiments of the present disclosure, according to the test requirement information and the target search information, a first set of test cases is generated, and the quality of the first set of test cases is evaluated to obtain a second set of test cases. Furthermore, the second set of test cases is optimized according to the user's use case optimization information to obtain the set of test cases for the project, which can realize continuous generation and optimization of test cases and improve the accuracy and effectiveness of the test cases.

[0070] In some embodiments, according to the test requirement information and the target search information, a use case generation prompt can be obtained, and the use case generation prompt is input into the large model. The large model generates a first set of test cases according to the use case generation prompt, thus realizing the automatic generation of test cases. Generating test cases according to the prompt can improve the efficiency of generating test cases.

[0071] In some embodiments, in order to avoid generating duplicate or invalid test cases and improve the applicability of the generated test cases, the test scenarios and boundary conditions to be covered can be determined, and the use case generation prompt is optimized according to the test scenarios and boundary conditions. Then, the optimized use case generation prompt is input into the large model to increase the applicable range of the test cases in the first set of test cases generated.

[0072] According to the method for generating test cases based on a large model provided by the embodiments of the present disclosure, the large model analyzes the test requirement information to obtain the scope to be tested and the corresponding test objectives of the project, searches for the target search information in the project knowledge graph according to the scope to be tested and the corresponding test objectives, and then generates a set of test cases according to the test requirement information and the target search information. Thus, by using the large model to generate test cases, automatic generation of test cases can be realized, and the generation efficiency of test cases can be improved. Generating test cases according to the test requirement information and the project knowledge graph can make the generated test cases meet the test requirement information, thereby improving the effectiveness of project testing.

[0073] Figure 3 It is a schematic flowchart of a method for generating test cases based on a large model provided by the embodiments of the present disclosure.

[0074] As Figure 3 shown, the method for generating test cases based on a large model may include:

[0075] S301. Obtain the historical materials of the project, where the historical materials include at least one of the following: historical test cases, historical code, and historical project documents.

[0076] S302. Perform knowledge extraction on the historical materials through an NLP model, and generate a project knowledge graph based on the extracted knowledge.

[0077] S303. Obtain the test requirement information of the project.

[0078] S304. Generate a set of test cases for the project through a large model based on the test requirement information and the project knowledge graph.

[0079] For the relevant content of steps S301 - S304, refer to the above embodiments, which will not be elaborated here.

[0080] S305. Invoke a test case execution tool through a large model to execute the test cases in the set of test cases, and obtain the test results corresponding to the test cases.

[0081] In some embodiments, after generating a set of test cases for the project, the large model can call a test case execution tool to automatically execute the test cases in the set of test cases to test the test cases, thereby obtaining the test results corresponding to the test cases. Optionally, by executing the test cases, it can be verified whether the project can work according to the set goals, and defects and problems in the project can be determined.

[0082] In some embodiments, the test results corresponding to the test cases may include information such as whether the test passes, the differences between the actual output and the target output, the test execution time, the test environment, etc. Optionally, if the test fails, information such as the reason for the failure and the defects of the test cases can be used as the test results.

[0083] S306. Select the target report template for the project from the candidate report templates generated by the large model based on the historical materials of the project.

[0084] In some embodiments, by obtaining the historical materials of the project, generating candidate report templates through a large model based on the historical project reports in the historical materials, and determining the target report template for the project from the candidate report templates according to the similarity between projects.

[0085] In some embodiments, the historical project report can be determined based on the historical project documents in the historical materials of the project. That is, the historical project documents contain information such as the historical project report.

[0086] In some embodiments, different projects correspond to different project reports. By generating candidate report templates corresponding to different projects based on the historical project reports of the projects and calculating the similarity between the projects, the target report template of the project can be determined from the candidate report templates according to the similarity.

[0087] Exemplarily, assume that project A corresponds to candidate report template A and project B corresponds to candidate report template B. If the target report template of project C is to be determined, the similarity between project C and project A and project B can be calculated. If the similarity between project C and project A is greater than the similarity between project C and project B, then candidate report template A is determined as the target report template.

[0088] S307. Use a large model to generate the test report data of the project based on the test cases and the corresponding test results.

[0089] In some embodiments, the large model can integrate the test cases and the corresponding test results to obtain the test report data of the project.

[0090] In some embodiments, the large model can also provide background knowledge and analysis support for the test report data based on the information in the project knowledge graph, so as to provide in-depth data analysis for the test report data, thereby improving the quality of the test report.

[0091] In the embodiments of the present disclosure, the test report data contains background knowledge information, which can make the test report more accurately present the test data and help evaluate the project performance more accurately. The test report can also perform trend analysis of the project performance according to the background knowledge information, so as to achieve accurate analysis and prediction of the project performance.

[0092] In some embodiments, use a large model to determine the background knowledge information associated with the test cases and the corresponding test results according to the project knowledge graph, and then generate the test report data of the project based on the background knowledge information, the test cases and the corresponding test results.

[0093] In some embodiments, the large model can search for relevant entities and entity relationships in the project knowledge graph according to the test cases and the corresponding test results, so as to locate relevant information in the project knowledge graph and use the located relevant information as the background knowledge information.

[0094] In some embodiments, the test report data also includes result comparison information to analyze the test cases according to the result comparison information, so as to accurately locate the problems of the test cases and help repair the test cases. The large model can determine the test results of the historical versions of the project according to the project knowledge graph and analyze the test results and the test results of the historical versions to obtain the result comparison information.

[0095] In some embodiments, the large model can extract the test results corresponding to different historical versions of the project from the project knowledge graph, and analyze the test results and the test results of historical versions, so as to obtain trends such as changes in test coverage and increases or decreases in the number of defects, as result comparison information.

[0096] S308. According to the target report template, convert the test report data to obtain the test report of the project.

[0097] In some embodiments, the large model can map the test report data to the corresponding positions in the target report template according to the structure and requirements of the target report template, so as to obtain the test report of the project.

[0098] In some embodiments, the format of the test report data can also be converted so that the converted test report data meets the format requirements of the target report template, such as converting the font, font size, alignment method, etc. of the test report data.

[0099] According to the test case generation method based on the large model provided by the embodiments of the present disclosure, after obtaining the test case set of the project, execute the test cases in the test case set, and generate the test report of the project in combination with the target report template corresponding to the project. Thus, testing the test cases and generating the test report of the project helps to determine the defects and problems of the project, thereby improving the stability and reliability of the project. Generating the test report according to the target report template helps to promote the standardization of the test report.

[0100] Based on the above embodiments, after obtaining the test report of the project, the embodiments of the present disclosure can also perform online deployment on the project according to the test report of the project, such as Figure 4 As shown, the process of performing online deployment on the project may include:

[0101] S401. Through the large model, according to the test report of the project, determine the first deployment task of the project. The first deployment task includes at least the maximum traffic allowed for the project, and the maximum traffic is less than the set traffic threshold.

[0102] In some embodiments, in order to reduce the risk of the project, the project can be deployed online in a small-traffic manner through the large model according to the test report of the project. Among them, small traffic means that the maximum traffic allowed for the project is less than the set traffic threshold.

[0103] In some embodiments, the large model can analyze the test report to obtain information such as the test pass rate and failure rate of the project from the test report, and according to the information in the test report, analyze the risks faced by the project and determine the time for the project to be deployed online, so as to determine the first deployment task according to the risks faced by the project and the time for the project to be deployed online.

[0104] In some embodiments, functions that pass the test and have a low risk in the project can be selected for online deployment, and a first deployment task is generated according to the function and the deployment time.

[0105] S402. Call the project deployment tool through the large model to execute the first deployment task for throttling the online deployment of the project.

[0106] It can be understood that executing the first deployment task for throttling the online deployment of the project means that when the new version or new function of the project is first deployed to the production environment, access traffic is restricted through technical means, and only a small number of user requests are allowed to access the new version, thereby reducing potential risks and improving the stability of the project.

[0107] In some embodiments, the online deployment of throttling the project can be performed according to the maximum traffic allowed for the project included in the first deployment task.

[0108] In some embodiments, after the online deployment of throttling the project, the large model can obtain the online operation log information and perform analysis according to the operation log information to monitor the operation of the project, so as to determine the problems on the project line according to the operation log information, repair the problems, and further perform a full-scale online deployment.

[0109] That is to say, the large model obtains the operation log information after the online deployment of the project, and generates a second deployment task for the project according to the operation log information, where the second deployment task at least includes the traffic increment corresponding to different batches of online deployments.

[0110] In some embodiments, the large model can analyze the operation log information, determine the abnormal problems existing in the project according to the analysis results, generate a problem repair task for the project through the large model, and execute the problem repair task for the project to obtain the problem repair result of the project. Further, according to the problem repair result, a strategy for gradually rolling out the project in full can be determined, so that the project can be smoothly transitioned. That is to say, a second deployment task can be generated according to the traffic increment corresponding to different batches of online deployments, and the second deployment task is executed through the project deployment tool to perform a full-scale online deployment of the project.

[0111] In some embodiments, in order to improve the repair efficiency and repair effect of the problems, after the large model executes the problem repair task for the project, the abnormal problems that have not been repaired in the project can be obtained, a first alarm message can be generated according to the abnormal problems that have not been repaired, and an alarm can be made according to the first alarm message, so that the abnormal problems that have not been repaired can be repaired manually according to the first alarm message.

[0112] In some embodiments, after a full-scale online deployment of a project, it is also possible to obtain the user behavior data after the online deployment of the project, and analyze the user behavior data through a large model to identify potential problems existing in the project based on the user behavior data. Further, second alarm information can be generated according to the potential problems, and the second alarm information can be processed manually.

[0113] In the embodiments of the present disclosure, by analyzing the user behavior data to determine potential problems existing in the project, thereby generating second alarm information, it is possible to achieve a rapid response to problems, enhance the stability of the project, and improve the user experience.

[0114] In some embodiments, in order to improve the user experience of the project, after obtaining the user behavior data, the user behavior data can be analyzed for vulnerabilities through a large model to obtain the key nodes that lead to user churn in the project, and optimization information for the project can be generated according to the key links.

[0115] In some embodiments, by analyzing the user behavior data through a large model for vulnerability analysis, the conversion rates at each stage after the project goes live can be determined, and the key nodes that lead to user churn can be determined according to the conversion rates. It is possible to determine the stage where the conversion rate is lower than the set threshold as the key node that leads to user churn.

[0116] Further, the reasons for churn can be analyzed based on the user behavior data of the key nodes, and optimization information can be generated according to the reasons for churn. Such as simplifying the operation process, optimizing the interface design, etc.

[0117] According to the method for generating test cases based on a large model provided by the embodiments of the present disclosure, the large model determines the first deployment task of the project according to the test report of the project, and executes the first deployment task to perform a throttled online deployment of the project. Further, the running log information after the online deployment of the project is obtained, so that a full-scale online deployment can be performed according to the running log information. Thus, performing a full-scale deployment after a throttled deployment can reduce the risk of the project and enable the project to transition smoothly.

[0118] Based on the above embodiments, the embodiments of the present disclosure also propose an agent.

[0119] Figure 5 As shown in the structural schematic diagram of an agent provided by the embodiments of the present disclosure, Figure 5 as shown, the agent 500 of the embodiments of the present disclosure includes: an input module 501, a processing module 502, and an output module 503.

[0120] In some embodiments, an input module 501 is configured to receive input information; the input information includes historical materials and test requirement information of a project, where the historical materials include at least one of historical test cases, historical code, and historical project documents.

[0121] In some embodiments, a processing module 502 is configured to determine a target task based on the input information, where the historical materials of the project correspond to a first target task, the test requirement information corresponds to a second target task, call an NLP model based on the first target task, perform knowledge extraction on the historical materials through the NLP model, generate a project knowledge graph based on the extracted knowledge, and call a large model according to the second target task, and generate a set of test cases for the project through the large model based on the test requirement information and the project knowledge graph as output information.

[0122] In some embodiments, an output module 503 is configured to output the output information obtained by the processing module.

[0123] In the embodiments of the present disclosure, the generation of test cases through an intelligent agent can improve the efficiency of generating test cases, further improve the test efficiency of the project, and using an intelligent agent in the process of generating test cases can reduce the test cost.

[0124] It should be noted that the foregoing explanation of the embodiments of the test case generation method based on a large model also applies to this intelligent agent, and will not be elaborated here.

[0125] Exemplary illustration, Figure 6 The figure shows a schematic flow diagram of generating test cases based on a large model. Figure 6 The intelligent agent in [the figure] is composed of a project knowledge graph and a large language model. The large model in [the figure] can be regarded as a digital employee, and this digital employee can replace manual labor to perform the test and online deployment work of the project. Figure 6

[0126] The large language model uses at least one of historical test cases, historical code, and historical project documents as historical materials, and performs knowledge extraction on the historical materials, so as to generate a project knowledge graph according to the extracted knowledge. Further, the large language model and the project knowledge graph are used as an intelligent agent, and the intelligent agent generates a set of test cases for the project according to the test requirement information and the project knowledge graph, executes the test cases in the set of test cases, and generates a corresponding project test report according to the corresponding test results, so that the problems existing in the test cases can be repaired according to the project test report. Further, after performing traffic limiting online deployment on the project, and according to the operation log information after the project is online deployed, perform full-scale online deployment on the project.

[0127] ​Corresponding to the method for generating test cases based on a large model provided in the above several embodiments, an embodiment of the present disclosure further provides a device for generating test cases based on a large model. Since the device for generating test cases based on a large model provided in the embodiments of the present disclosure corresponds to the method for generating test cases based on a large model provided in the above several embodiments, the implementation manners of the above method for generating test cases based on a large model are also applicable to the device for generating test cases based on a large model provided in the embodiments of the present disclosure, and will not be described in detail in the following embodiments.

[0128] Figure 7 It is a structural schematic diagram of a device for generating test cases based on a large model provided in an embodiment of the present disclosure.

[0129] As Figure 7 shown, the device 700 for generating test cases based on a large model in the embodiment of the present disclosure includes a first acquisition module 701, a first generation module 702, a second acquisition module 703, and a second generation module 704.

[0130] The first acquisition module 701 is configured to acquire historical materials of a project, and the historical materials include at least one of the following: historical test cases, historical code, and historical project documents;

[0131] The first generation module 702 is configured to perform knowledge extraction on the historical materials through a natural language processing (NLP) model, and generate a project knowledge graph based on the extracted knowledge;

[0132] The second acquisition module 703 is configured to acquire test requirement information of the project;

[0133] The second generation module 704 is configured to generate a set of test cases for the project through a large model according to the test requirement information and the project knowledge graph.

[0134] In an embodiment of the present disclosure, the second generation module 704 is further configured to: parse the test requirement information through a large model to obtain the to-be-tested scope of the project and the corresponding test objectives; search in the project knowledge graph according to the to-be-tested scope and the corresponding test objectives to obtain target search information, where the target search information includes at least one of the first historical test cases, the first historical code, and the first historical project documents; generate a set of test cases for the project according to the test requirement information and the target search information.

[0135] In one embodiment of the present disclosure, the second generation module 704 is further configured to: generate a first set of test cases through a large model based on test requirement information and target search information; perform quality evaluation on the test cases in the first set of test cases to obtain quality evaluation information of the test cases; determine a second set of test cases from the first set of test cases according to the quality evaluation information; send the second set of test cases to the client; receive the use case optimization information sent by the client, and optimize and adjust the test cases according to the use case optimization information and the historical conversation information of this round to generate a set of test cases for the project.

[0136] In one embodiment of the present disclosure, the second generation module 704 is further configured to: obtain use case generation prompt words according to test requirement information and target search information; input the use case generation prompt words into the large model, and generate a first set of test cases through the large model according to the use case generation prompt words.

[0137] In one embodiment of the present disclosure, the second generation module 704 is further configured to: determine the test scenarios and boundary conditions to be covered, and optimize the use case generation prompt words according to the test scenarios and boundary conditions.

[0138] In one embodiment of the present disclosure, the second generation module 704 is further configured to: call a use case execution tool through the large model to execute the test cases in the set of test cases, and obtain the test results corresponding to the test cases; select the target report template of the project from the candidate report templates generated by the large model according to the historical materials of the project; generate test report data of the project through the large model according to the test cases and the corresponding test results; convert the test report data according to the target report template to obtain the test report of the project.

[0139] In one embodiment of the present disclosure, the second generation module 704 is further configured to: determine the background knowledge information associated with the test cases and the corresponding test results through the large model according to the project knowledge graph; generate test report data of the project according to the background knowledge information, the test cases and the corresponding test results.

[0140] In one embodiment of the present disclosure, the second generation module 704 is further configured to: determine the test results of the historical versions of the project through the large model according to the project knowledge graph; analyze the test results and the test results of the historical versions to obtain result comparison information, wherein the test report data further includes the result comparison information.

[0141] In one embodiment of the present disclosure, the second generation module 704 is further configured to: determine, by means of a large model, a first deployment task of the project according to the test report of the project, where the first deployment task includes at least the maximum traffic allowed for the project, and the maximum traffic is less than the set traffic threshold; call a project deployment tool through the large model to execute the first deployment task to perform an online deployment of traffic limiting for the project.

[0142] In one embodiment of the present disclosure, the second generation module 704 is further configured to: obtain the operation log information after the online deployment of the project through the large model; generate a second deployment task of the project according to the operation log information, where the second deployment task includes at least the traffic increment corresponding to different batches of online deployment; execute the second deployment task through the project deployment tool to perform a full-scale online deployment of the project.

[0143] In one embodiment of the present disclosure, the second generation module 704 is further configured to: analyze the operation log information, determine the abnormal problems existing in the project according to the analysis results; generate a problem repair task of the project according to the abnormal problems through the large model, and execute the problem repair task for the project.

[0144] In one embodiment of the present disclosure, the second generation module 704 is further configured to: obtain the un-repaired abnormal problems of the project, and generate a first alarm message according to the un-repaired abnormal problems.

[0145] In one embodiment of the present disclosure, the device further includes: obtaining the user behavior data after the online deployment of the project; identifying the potential problems existing in the project according to the user behavior data; generating a second alarm message according to the potential problems.

[0146] In one embodiment of the present disclosure, the device further includes: performing vulnerability analysis on the user behavior data through the large model to obtain the key nodes causing user loss in the project; generating optimization information of the project according to the key links.

[0147] According to the test case generation device based on a large model provided by the embodiments of the present disclosure, by obtaining the historical materials of the project and performing knowledge extraction on the historical materials through the large model, a project knowledge graph can be generated based on the extracted knowledge. Further obtain the test requirement information of the project, and generate a test case set of the project through the large model according to the test requirement information and the project knowledge graph. Thus, by generating test cases through the large model, the efficiency of generating test cases can be improved, and the test efficiency of the project can be further improved. Using the large model in the process of generating test cases can reduce the test cost.

[0148] In the technical solution of the present disclosure, the acquisition, storage, and application of the user's personal information involved all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0149] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0150] Figure 8 FIG. shows a schematic block diagram of an exemplary electronic device 800 that can be used to implement embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, for example, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, for example, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0151] As Figure 8 shown, the device 800 includes a computing unit 801 that can perform various appropriate actions and processes according to computer programs / instructions stored in a read-only memory (ROM) 802 or computer programs / instructions loaded from a storage unit 806 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0152] A plurality of components in the device 800 are connected to the I / O interface 805, including: an input unit 806 such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0153] The computing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 executes the various methods and processes described above, such as the large model-based test case generation method. For example, in some embodiments, the large model-based test case generation method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as the storage unit 806. In some embodiments, part or all of the computer program / instructions can be loaded and / or installed onto the device 800 via the ROM 802 and / or the communication unit 809. When the computer program / instructions are loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the large model-based test case generation method described above can be executed. Alternatively, in other embodiments, the computing unit 801 can be configured to execute the large model-based test case generation method by any other suitable means (e.g., by means of firmware).

[0154] Various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs / instructions that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0155] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as an independent software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0156] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0157] To provide for interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic, speech, or tactile input).

[0158] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0159] A computer system may include a client and a server. The client and the server are generally far from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs / instructions that run on the respective computers and have a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0160] It should be understood that various forms of the processes shown above can be used, steps can be reordered, added or deleted. For example, the steps described in the disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and no limitations are imposed herein.

[0161] The above specific embodiments do not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. A test case generation method based on a large model, wherein, The method includes: Obtaining historical materials of the project, where the historical materials include at least one of historical test cases, historical code, and historical project documents; Performing knowledge extraction on the historical materials through a natural language processing (NLP) model, and generating a project knowledge graph based on the extracted knowledge; Obtaining the test requirement information of the project; Generating a test case set for the project through a large model according to the test requirement information and the project knowledge graph.

2. The method according to claim 1, wherein, The step of generating a test case set for the project through a large model according to the test requirement information and the project knowledge graph includes: Parsing the test requirement information through the large model to obtain the scope to be tested and the corresponding test objectives of the project; Searching in the project knowledge graph according to the scope to be tested and the corresponding test objectives to obtain target search information, where the target search information includes at least one of first historical test cases, first historical code, and first historical project documents; Generating a test case set for the project according to the test requirement information and the target search information.

3. The method according to claim 2, wherein, The step of generating a test case set for the project according to the test requirement information and the target search information includes: Generating a first test case set through the large model according to the test requirement information and the target search information; Performing quality evaluation on the test cases in the first test case set to obtain quality evaluation information of the test cases; Determining a second test case set from the first test case set according to the quality evaluation information; Sending the second test case set to the client; Receiving use case optimization information sent by the client, and optimizing and adjusting the test cases according to the use case optimization information and the historical conversation information of this round to generate a test case set for the project.

4. The method according to claim 3, wherein The step of generating a first test case set through the large model according to the test requirement information and the target search information includes: Obtaining use case generation prompt words according to the test requirement information and the target search information; Inputting the use case generation prompt words into the large model, and generating the first test case set through the large model according to the use case generation prompt words.

5. The method according to claim 4, wherein, Before inputting the use case generation prompt words into the large model and generating the first test case set through the large model according to the use case generation prompt words, it further includes: Determining the test scenarios and boundary conditions to be covered, and optimizing the use case generation prompt words according to the test scenarios and the boundary conditions.

6. The method according to any one of claims 1 - 5, wherein, After generating the test case set for the project, it further includes: Invoking a use case execution tool through the large model to execute the test cases in the test case set, and obtaining the test results corresponding to the test cases; Selecting a target report template for the project from the candidate report templates generated by the large model according to the historical materials of the project; Generating test report data for the project through the large model according to the test cases and the corresponding test results. Convert the test report data according to the target report template to obtain the test report of the project.

7. The method according to claim 6, wherein, The large model generates the test report data of the project based on the test cases and corresponding test results, including: The large model determines the background knowledge information associated with the test cases and corresponding test results according to the project knowledge graph; Generate the test report data of the project according to the background knowledge information, the test cases and the corresponding test results.

8. The method according to claim 7, wherein The method further includes: The large model determines the test results of the historical versions of the project according to the project knowledge graph; Analyze the test results and the test results of the historical versions to obtain result comparison information, where the test report data further includes the result comparison information.

9. The method according to claim 6, wherein After obtaining the test report of the project, it further includes: The large model determines the first deployment task of the project according to the test report of the project. The first deployment task at least includes the maximum traffic allowed for the project to occupy, and the maximum traffic is less than the set traffic threshold; The large model calls the project deployment tool to execute the first deployment task to perform online deployment of traffic limiting on the project.

10. The method according to claim 9, wherein, The method further includes: The large model obtains the operation log information after the online deployment of the project; Generate the second deployment task of the project according to the operation log information, where the second deployment task at least includes the traffic increment corresponding to different batches of online deployment; The project deployment tool executes the second deployment task to perform full-scale online deployment of the project.

11. The method according to claim 10, wherein, The generating the second deployment task of the project according to the operation log information includes: Analyze the operation log information, and determine the abnormal problems existing in the project according to the analysis results; The large model generates a problem repair task for the project according to the abnormal problems, and executes the problem repair task for the project.

12. The method according to claim 11, wherein After executing the problem repair task for the project, it further includes: Obtain the abnormal problems that have not been repaired in the project, and generate a first alarm message according to the abnormal problems that have not been repaired.

13. The method according to claim 11, wherein, The method further includes: Obtain the user behavior data after the online deployment of the project; Identify the potential problems existing in the project according to the user behavior data; Generate a second alarm message according to the potential problems.

14. The method according to claim 13, wherein, The method further includes: The large model performs vulnerability analysis on the user behavior data to obtain the key nodes that cause user loss in the project; Generate the optimization information of the project according to the key links.

15. A test case generation device based on a large model, wherein, The device includes: The first acquisition module is used to acquire the historical materials of the project. The historical materials at least include at least one of historical test cases, historical code, and historical project documents; The first generation module is used to perform knowledge extraction on the historical materials through a natural language processing NLP model, and generate a project knowledge graph based on the extracted knowledge; The second acquisition module is used to acquire the test requirement information of the project; A second generation module, configured to generate a set of test cases for the project through a large model based on the test requirement information and the project knowledge graph.

16. An intelligent agent includes: An input module, configured to receive input information; The input information includes historical materials and test requirement information of a project, where the historical materials at least include at least one of historical test cases, historical code, and historical project documents; A processing module, configured to determine a target task based on the input information, where the historical materials of the project correspond to a first target task, the test requirement information corresponds to a second target task, call an NLP model based on the first target task, perform knowledge extraction on the historical materials through the NLP model, generate a project knowledge graph based on the extracted knowledge, and call a large model according to the second target task, and generate a set of test cases for the project through the large model based on the test requirement information and the project knowledge graph as output information; An output module, configured to output the output information obtained by the processing module.

17. An electronic device includes: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1-14.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the method according to any one of claims 1-14.

19. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, the method according to any one of claims 1-14 is implemented.

Images