Test case generation method and device, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides a test case generation method and device, electronic equipment, a storage medium and a program product, and relates to the technical fields of computer technologies, software testing, artificial intelligence and the like. The method comprises the steps of obtaining a request log corresponding to a target interface, and obtaining a test case generation prompt text and a case template; generating a prompt text based on the request log, the case template and the test case to determine input information; instructing the large model to execute a test case generation operation through the input information to obtain an interface test case for the target interface; wherein the test case generation operation comprises the steps of generating a prompt text based on case description and generating the case description based on a request log; generating a use case assertion based on the use case assertion generation prompt text and the request log; and generating a use case main body based on the use case main body generation prompt text, the use case template and the request log. According to the embodiment of the invention, the method is better compatible with various different services, and the coverage range of the interface for generating the test case is larger.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and in particular, the present disclosure relates to a test case generation method and device, electronic equipment, storage medium and program product. BACKGROUND

[0002] Interface test cases refer to test cases used to test interfaces in a software system. The purpose of interface test cases is to verify whether these interfaces work as expected and ensure that data can be correctly transmitted and processed between different systems or components.

[0003] Traditional methods of manually generating interface test cases cannot cope with the increasing complexity of software systems and the shortening of development cycles. Therefore, automatic generation methods of interface test cases have emerged. Existing automatic generation methods of interface test cases are usually based on machine learning or deep learning models. The existing interface test cases are used to train the model, and the trained model is used to automatically generate interface test cases. However, the current automatic generation method of interface test cases relies on a large amount of training data and cannot be applied to businesses with less or no training data, which has poor applicability. SUMMARY

[0004] The embodiments of the present disclosure provide a test case generation method, device, electronic equipment, storage medium and program product, which can solve the problem of poor applicability of the current automatic generation method of interface test cases. The technical solutions provided by the present disclosure are as follows:

[0005] According to an aspect of an embodiment of the present disclosure, a test case generation method is provided, which comprises:

[0006] Obtaining a request log corresponding to a target interface, and obtaining a test case generation prompt text and a case template; the test case generation prompt text comprises a case description generation prompt text, a case assertion generation prompt text and a case body generation prompt text;

[0007] Determining input information based on the request log, the case template and the test case generation prompt text;

[0008] Using the input information to instruct the large model to perform a test case generation operation to obtain an interface test case for the target interface; the interface test case comprises a case description, a case assertion and a case body;

[0009] The test case generation operation comprises:

[0010] Generating the case description based on the case description generation prompt text and the request log;

[0011] generate the use case assertion based on the use case assertion generation prompt text and the request log;

[0012] generate the use case subject based on the use case subject generation prompt text, the use case template, and the request log.

[0013] Optionally, the method further comprises:

[0014] performing at least one update operation on the interface test case until the running result corresponding to the interface test case is passed;

[0015] The update operation comprises:

[0016] determining a to-be-processed test case; the to-be-processed test case corresponding to the first update operation is the interface test case;

[0017] running the test case file corresponding to the to-be-processed test case to obtain a corresponding running result;

[0018] If the running result is not passed, the to-be-processed test case is updated to obtain an updated to-be-processed test case, and the updated to-be-processed test case is taken as a to-be-processed test case corresponding to the next update operation.

[0019] Optionally, the updating of the to-be-processed test case to obtain the updated to-be-processed test case comprises:

[0020] obtaining running exception information corresponding to the to-be-processed test case; the running exception information comprises at least one of running error logs and code change information;

[0021] indicate, through a use case repair prompt text, the large model to repair the to-be-processed test case based on the running exception information to obtain the updated to-be-processed test case.

[0022] Optionally, the updating of the to-be-processed test case to obtain the updated to-be-processed test case comprises:

[0023] obtaining a new request log corresponding to the target interface;

[0024] generating a first interface test case corresponding to the new request log based on the new request log;

[0025] taking the first interface test case as the updated to-be-processed test case.

[0026] Optionally, the method further comprises:

[0027] determining an interface name of the target interface;

[0028] generating, according to a preset file format, an interface test case file corresponding to the target interface based on the interface test case corresponding to the target interface, and taking the interface name as a file name of the interface test case file;

[0029] If the interface test case corresponding to the target interface is updated, the file name of the updated interface test case file is kept as the interface name.

[0030] Optionally, the obtaining the request log comprises:

[0031] obtaining a plurality of initial request logs from a database;

[0032] analyzing each initial request log by the large model based on the error log screening prompt text to obtain an error label and an error reason corresponding to each initial request log;

[0033]

[0033] screening at least one error log from the plurality of initial request logs based on the error label and the error reason corresponding to each initial request log;

[0034] determining the request log based on the plurality of initial request logs except the at least one error log.

[0035] Optionally, the method further comprises:

[0036] analyzing the association relationship between a plurality of interfaces corresponding to a plurality of request logs by the large model based on the association analysis prompt text to obtain at least one interface group; the interface group comprises at least two related interfaces for the same scene;

[0037] For each interface group, obtaining a scene test case for a scene corresponding to the interface group based on the interface test case corresponding to each interface in the interface group and the association relationship between each interface in the interface group.

[0038] Optionally, the method further comprises:

[0039] analyzing the request log by the large model based on the attribute judgment prompt text to obtain a candidate interface attribute of the target interface and a judgment reason;

[0040] determining the interface attribute of the target interface based on the candidate interface attribute and the judgment reason corresponding to the target interface;

[0041] If the interface attribute of the target interface meets a preset test condition, testing the interface test case corresponding to the target interface.

[0042] Optionally, the method further comprises:

[0043] updating the large model as a target large model;

[0044] wherein the target large model is obtained by training the large model based on a plurality of positive samples and a plurality of negative samples;

[0045] the plurality of positive samples are determined based on a plurality of request logs and second interface test cases respectively corresponding to the plurality of request logs; the second interface test cases correspond to passing running results;

[0046] the plurality of negative samples are determined based on a plurality of request logs and third interface test cases respectively corresponding to the plurality of request logs; the third interface test cases include interface test cases with non-passing running results and interface test cases obtained by modifying the second interface test cases.

[0047] According to another aspect of the embodiments of the present disclosure, a test case generation apparatus is provided, which comprises:

[0048] a data acquisition module configured to acquire request logs corresponding to a target interface, and acquire a test case generation prompt text and a case template; the test case generation prompt text comprises a case description generation prompt text, a case assertion generation prompt text and a case body generation prompt text;

[0049] an input information determination module configured to determine input information based on the request logs, the case template and the test case generation prompt text;

[0050] a test case generation module configured to instruct the large model to perform a test case generation operation through the input information, to obtain an interface test case for the target interface; the interface test case comprises a case description, a case assertion and a case body;

[0051] wherein the test case generation operation comprises:

[0052] generating the case description based on the case description generation prompt text and the request logs;

[0053] generating the case assertion based on the case assertion generation prompt text and the request logs;

[0054] generating the case body based on the case body generation prompt text, the case template and the request logs.

[0055] According to another aspect of the embodiments of the present disclosure, an electronic device is provided, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps of any of the above test case generation methods when executing the program.

[0056] According to still another aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, and the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of any of the test case generation methods described above.

[0057] According to an aspect of the embodiments of the present disclosure, a computer program product is provided, and the computer program product includes a computer program which, when executed by a processor, implements the steps of any of the test case generation methods described above.

[0058] The technical solutions provided by the embodiments of the present disclosure have the following beneficial effects:

[0059] By obtaining the request log corresponding to the target interface, and based on the request log, generating the interface test case corresponding to the target interface through the large model, the semantic understanding ability of the large model is fully utilized to automatically generate the interface test case, without additional training of the model, avoiding the limitation that a large amount of training data cannot be obtained in a specific business, and better compatibility with multiple different businesses is achieved, with stronger applicability and larger interface coverage of the generated test case.

[0060] Further, by dividing the interface test case into a case description, a case assertion and a case body, and setting the case description generation prompt text, the case assertion generation prompt text and the case body generation prompt text respectively, the integrity of the generated interface test case is ensured, the situation that the generated interface test case is unusable due to missing information is avoided, and the usability of the interface test case is improved.

[0061] Further, by generating the corresponding interface test case based on the request log and the case template, different interface test cases corresponding to different request logs for the same business can all conform to a unified format, which helps the tester to better understand the content and structure of the interface test case, enhances the readability of the interface test case, avoids the inconsistency caused by different code writing styles of different business personnel when manually writing the interface test case, and avoids the inconsistency caused by different interface frameworks of different interfaces, which is conducive to subsequent maintenance and modification of the interface test case. BRIEF DESCRIPTION OF DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings needed to be used in the description of the embodiments of the present disclosure will be briefly introduced.

[0063] Figure 1 The application environment schematic diagram of the test case generation method provided by the embodiments of the present disclosure is shown in FIG. 1.

[0064] Figure 2A flowchart of a test case generation method provided by an embodiment of the present disclosure is shown in FIG. 1.

[0065] Figure 3 A schematic diagram of a test case generation process provided by an embodiment of the present disclosure is shown in FIG. 2.

[0066] Figure 4 A flowchart of a test case update process provided by an embodiment of the present disclosure is shown in FIG. 3.

[0067] Figure 5 A schematic diagram of an error log screening process provided by an embodiment of the present disclosure is shown in FIG. 4.

[0068] Figure 6 A schematic diagram of an interface attribute judgment process provided by an embodiment of the present disclosure is shown in FIG. 5.

[0069] Figure 7 A schematic diagram of a test case processing process provided by an embodiment of the present disclosure is shown in FIG. 6.

[0070] Figure 8 A system architecture diagram of a test system provided by an embodiment of the present disclosure is shown in FIG. 7.

[0071] Figure 9 A structural schematic diagram of a test case generation apparatus provided by an embodiment of the present disclosure is shown in FIG. 8.

[0072] Figure 10 A structural schematic diagram of an electronic device provided by an embodiment of the present disclosure is shown in FIG. 9. DETAILED DESCRIPTION

[0073] Embodiments of the present disclosure are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions of the technical solutions of the embodiments of the present disclosure, and do not constitute a limitation on the technical solutions of the embodiments of the present disclosure.

[0074] Those skilled in the art can understand that the singular forms "a", "an", "said" and "the" used herein also include the plural forms unless specifically stated otherwise. It should be further understood that the terms "comprise" and "include" used in the embodiments of the present disclosure mean that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation of other features, information, data, steps, operations, elements, components and / or their combinations supported by the present technology. It should be understood that when we say that an element is "connected" or "coupled" to another element, the element can be directly connected or coupled to the other element, or it can mean that the element and the other element establish a connection relationship through an intermediate element. In addition, "connected" or "coupled" used herein can include wireless connection or wireless coupling. The term "and / or" used herein indicates that at least one of the items defined by the term, for example, "A and / or B" or "A, B" indicates implementation as "A", or implementation as "B", or implementation as "A and B".

[0075] For the purpose, technical solutions and advantages of the present disclosure to be clearer, the embodiments of the present disclosure will be described in further detail below with reference to the drawings.

[0076] Software testing refers to the process of running software systems or applications to find errors, defects or problems therein, and its purpose is to identify errors and defects in software products, ensure that software meets design requirements and user needs, and verify the functions and performance of software. Software testing is a key component of the software development life cycle.

[0077] Test cases refer to a series of test steps and expected results designed to verify whether a certain function or feature works as expected during software testing. Interface test cases refer to test cases used to test interfaces in software systems, where interfaces can be interaction points between hardware and hardware, software and software, or software and hardware. The purpose of interface test cases is to verify that these interaction points work as expected and ensure that data can be correctly transmitted and processed between different systems or components.

[0078] The traditional method of manually generating interface test cases cannot cope with the increasing complexity of software systems and the shortening of development cycles, so the automatic generation method of interface test cases has emerged. Existing automatic generation methods of interface test cases are usually based on machine learning or deep learning models, which train the model with existing interface test cases and use the trained model to automatically generate interface test cases.

[0079] The current interface test case automatic generation method relies on a large amount of training data and cannot be applied to businesses with less or no training data, and the applicability is poor. Moreover, for some interfaces for which corresponding interface test cases cannot be obtained, corresponding interface test cases cannot be automatically generated, and the coverage of the interfaces is low. In addition, the quality of the automatically generated interface test cases is affected by the performance of the model, and the usability of the generated interface test cases cannot be guaranteed

[0080] The test case generation method, device, electronic equipment, storage medium and program product provided by the present disclosure aim to solve the above technical problems of the prior art.

[0081] Optionally, the test case generation method provided by the embodiments of the present disclosure can be implemented based on machine learning (ML) in artificial intelligence (AI).

[0082] Artificial intelligence is the use of digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which aims to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0083] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, automatic driving, intelligent transportation and several other major directions.

[0084] Machine learning (ML) is a multi-disciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It is a branch of computer science that studies how computers simulate or implement human learning behavior to acquire new knowledge or skills, reorganize existing knowledge structure to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent, and its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and example-based learning.

[0085] Optionally, the data processing involved in the method provided by the embodiments of the present disclosure can also be implemented based on cloud technology. For example, various calculations in generating the interface test case can be implemented using cloud computing technology, and data such as the request log and the generated interface test case can be stored using cloud storage. Cloud computing is a computing mode that distributes computing tasks on a resource pool formed by a large number of computers, so that various application systems can obtain computing power, storage space and information services according to needs, and a network providing resources is referred to as a "cloud". The resources in the "cloud" can be infinitely expanded in the eyes of the user, and can be obtained at any time, used on demand, expanded at any time, and paid according to use. Cloud storage is a new concept extended and developed on the basis of the concept of cloud computing. Cloud storage can collect a large number of various types of storage devices (storage nodes) in the network through application software or application interfaces to work together to provide a storage system for data storage and business access functions.

[0086] In the specific embodiments of the present disclosure, any data related to the object is involved, and when the embodiments of the present disclosure are applied to specific products or technologies, the permission or consent of the object needs to be obtained, and the collection, use and processing of the related data need to comply with the relevant laws, regulations and standards of the country and region. That is, if any of the above data related to the object is involved in the embodiments of the present disclosure, these data need to be obtained under the condition of authorization and consent of the object, and in compliance with the relevant laws, regulations and standards of the country and region.

[0087] The technical solutions of the embodiments of the present disclosure and the technical effects of the technical solutions of the present disclosure will be described below by describing several exemplary embodiments. It should be pointed out that the following embodiments can be mutually referenced, borrowed or combined. For the same terms, similar features and similar implementation steps in different embodiments, they will not be described repeatedly.

[0088] Figure 1An application environment schematic diagram of the test case generation method provided by the embodiments of the present disclosure is shown in the figure. The application environment can include a server 101 and a terminal 102. The server 101 obtains a request log corresponding to a target interface from the terminal 102, and obtains a test case generation prompt text and a test case template. The test case generation prompt text includes a test case description generation prompt text, a test case assertion generation prompt text, and a test case main body generation prompt text. The input information is generated based on the request log, the test case template, and the test case generation prompt text. The input information is used to instruct the large model to perform a test case generation operation, obtain an interface test case for the target interface, and return the interface test case to the terminal 102. The interface test case includes a test case description, a test case assertion, and a test case main body. The test case generation operation includes generating a test case description based on the test case description generation prompt text and the request log, generating a test case assertion based on the test case assertion generation prompt text and the request log, and generating a test case main body based on the test case main body generation prompt text, the test case template, and the request log.

[0089] The test case generation method provided by the embodiments of the present disclosure can be executed by any electronic device. The electronic device can be a server or a terminal as shown in the figure. Figure 1

[0090] The server can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server or a server cluster providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal can be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a notebook computer, a digital broadcast receiver, a MID (Mobile Internet Device), a PDA (Personal Digital Assistant), a desktop computer, a smart home appliance, a vehicle-mounted terminal (such as a vehicle-mounted navigation terminal, a vehicle-mounted computer, etc.), a smart speaker, a smart watch, etc. The terminal and the server can be directly or indirectly connected through wired or wireless communication, but are not limited thereto.

[0091] Figure 2 A flowchart of a test case generation method provided by the embodiments of the present disclosure is shown in the figure. The method includes the following steps. Figure 2

[0092] In step S110, a request log corresponding to a target interface is obtained, and a test case generation prompt text and a test case template are obtained. The test case generation prompt text includes a test case description generation prompt text, a test case assertion generation prompt text, and a test case main body generation prompt text.​​

[0093] In step S120, prompt text is generated based on the request log, the use case template, and the test case, and input information is determined;

[0094] In step S130, the input information is used to instruct the large model to perform a test case generation operation, and an interface test case for the target interface is obtained; the interface test case includes a use case description, a use case assertion, and a use case body.

[0095] The test case generation operation includes:

[0096] The use case description is generated based on the use case description, the prompt text, and the request log.

[0097] The use case assertion is generated based on the use case assertion, the prompt text, and the request log.

[0098] The use case body is generated based on the use case body, the prompt text, and the request log.

[0099] Specifically, the target interface can be an interface that needs to be tested, and the request log corresponding to the target interface can be a record generated when the target interface request is processed. The request log can include detailed information about the request, for example, the request log can include URL (Uniform Resource Locator, Uniform Resource Locator), request body, and response information, etc. The specific process of obtaining the request log will be described in detail below.

[0100] The test case generation method provided by the embodiments of the present disclosure is based on a large model (Large Language Model, Large Language Model) to automatically generate an interface test case, wherein the large model can be any type of generative language model, such as chatGPT (a large language model), chatGLM (a large language model), etc. The specific model structure of the large model is not limited by the embodiments of the present disclosure.

[0101] The large model can have multiple functions, and can be instructed to perform a specific function by inputting different prompts. Therefore, to instruct the large model to automatically generate a test case, a test case generation prompt text can also be obtained, wherein the test case generation prompt text can include a use case description generation prompt text, a use case assertion generation prompt text, and a use case body generation prompt text.

[0102] After obtaining the request log corresponding to the target interface and the test case generation prompt text, a use case template can also be obtained, wherein the use case template can be used to describe the format information of the output interface test case. In actual applications, for the same type of business, a corresponding use case template can be set.

[0103] The request log, the test case generation prompt text, and the case template can be input information, and the input information can be input into the large model. The large model can perform a test case generation operation to obtain an interface test case corresponding to the target interface.

[0104] The interface test case can include a case description, a case assertion, and a case body. The case description can be a brief description of the purpose and expected behavior of the interface test case. The case assertion can be a description of the expected result of the interface to be tested, used to verify whether the interface meets the expected behavior. The case assertion can include a comparison of the expected result and the actual result. If the actual result is consistent with the expected result, the test passes. Otherwise, the test fails. The case body can be a description of the request parameters and data of the interface to be tested, used to specify the request content sent to the interface when performing the test. The case body can include information such as the URL, request method, request header, and request body of the interface.

[0105] The three parts in the test case generation text correspond one-to-one to the three parts in the interface test case. That is, the case description generation prompt text is used to instruct the large model to generate the case description, the case assertion generation prompt text is used to instruct the large model to generate the case assertion, and the case body generation prompt text is used to instruct the large model to generate the case body.

[0106] The specific process of the test case generation operation can include: the large model can generate the case description based on the case description generation prompt text by analyzing the request log to obtain the information of the target interface function; the large model can generate the case assertion based on the case assertion generation prompt text by parsing the response information in the request log and selecting appropriate fields from the response information to generate the corresponding case assertion; and the large model can generate the corresponding interface test case by mapping the generated case description and case assertion, and the information in the request log to the format provided by the case template.

[0107] That is, the three components of the case description, the case assertion, and the case body are generated based on a complete request log and corresponding prompt text. The information input into the large model when generating each component is more comprehensive, making the results output by the large model more in line with actual needs, which is beneficial to improving the usability of the interface test case.

[0108] Optionally, the large model can generate the interface test case using the context information in the question and answer process. For example, the large model can analyze the request log to obtain the information of the target interface function, and generate the case assertion based on the case assertion generation prompt text in combination with the information of the target interface function and the request log.

[0109] Figure 3A schematic diagram of a test case generation process provided by an embodiment of the present disclosure is shown in Figure 3 As shown, the request log and the use case template are input into the large model, and the large model generates the corresponding interface test case through the test case generation prompt text, wherein the request log can include URL, request body (i.e., body), and response packet, etc. As can be seen from Figure 3 the data in the request log can be a structured text, and the texts corresponding to the URL, the request body, and the response packet in the request log can be labeled respectively; the use case template can be used to describe the format information of the output interface test case, and the use case template is arranged in the order of use case description, use case body, and use case assertion. The use case body defines parameters such as "owner", "tags", and "priority" and test methods, etc. The large model can map the generated use case description and use case assertion and use case body to the format provided by the use case template correspondingly based on the request log, so that the format of the interface test case output by the large model is consistent with the use case template.

[0110] The test case generation method provided by the embodiment of the present disclosure fully utilizes the semantic understanding ability of the large model to automatically generate the interface test case, without the need for additional training of the model, avoids the limitation that a large amount of training data cannot be obtained in a specific business, can better compatible with a variety of different businesses, has stronger applicability, and has a larger interface coverage range of the generated test case.

[0111] Further, by dividing the interface test case into use case description, use case assertion, and use case body, and setting the use case description generation prompt text, the use case assertion generation prompt text, and the use case body generation prompt text respectively, the integrity of the generated interface test case is ensured, the situation that the generated interface test case is unusable due to missing information is avoided, and the usability of the interface test case is improved.

[0112] Further, by generating the corresponding interface test case based on the request log and the use case template, different interface test cases corresponding to different request logs for the same business can all conform to a unified format, which helps the tester to better understand the content and structure of the interface test case, enhances the readability of the interface test case, avoids the inconsistency caused by the code writing style of different business personnel when manually writing the interface test case, and the inconsistency caused by the different interface frameworks of different interfaces, and is conducive to the subsequent maintenance and modification of the interface test case.

[0113] As an optional embodiment, the method further comprises:

[0114] The interface test case is executed at least once update operation, until the running result corresponding to the interface test case is passed;

[0115] The update operation includes:

[0116] Determine the test case to be processed; the test case to be processed corresponding to the first update operation is the interface test case;

[0117] Run the test case file corresponding to the test case to be processed to obtain the corresponding running result;

[0118] If the running result is not passed, the test case to be processed is updated to obtain the updated test case to be processed, and the updated test case to be processed is used as the test case to be processed corresponding to the next update operation.

[0119] Specifically, the interface test case generated based on the large model cannot be guaranteed to be available, for example, after generating the interface test case, the local will use the script to automatically deploy the environment for testing, due to the difference between different business environment deployments, it cannot be guaranteed that the generated interface test case can run through. For example, the hallucination problem of the large model may cause the generated interface test case to have syntax or content problems. In addition, the online interface test case may be wrong due to subsequent interface changes.

[0120] To solve the above problems, after obtaining the interface test case, at least one update operation is performed on the interface test case, the interface test case is used as the test case to be processed corresponding to the first update operation, the test case file corresponding to the test case to be processed is obtained, and the test case file is run to obtain the corresponding running result.

[0121] If the running result is passed, the update operation is stopped; if the running result is not passed, it indicates that the corresponding interface test case is not available, the test case to be processed can be updated, and the updated test case to be processed is used as the test case to be processed corresponding to the next update operation. By repeatedly performing the above update operation, until the running result corresponding to the interface test case is passed.

[0122] In the embodiment of the disclosure, by updating the interface test case with a running result that does not pass, until the running result of the updated interface test case does not pass, the generated interface test case is always available, the usability of the interface test case is improved, the automatic repair of the interface test case is realized, and the cost caused by manual repair is saved.

[0123] As an optional embodiment, updating the test case to be processed to obtain the updated test case to be processed includes:

[0124] Obtain running exception information corresponding to the to-be-processed test case; the running exception information includes at least one of running error logs and code change information;

[0125] The large model is instructed by the test case repair prompt text to repair the to-be-processed test case based on the running exception information, and updated to-be-processed test cases are obtained.

[0126] Specifically, if the running result corresponding to the to-be-processed test case is not passed, the corresponding running error logs can be obtained, wherein the running error logs can include information describing the running exception of the to-be-processed test case, for example, can include error reasons, positions of the exception, etc.

[0127] In addition, the to-be-processed test case can also be a previously passed interface test case (i.e., an online interface test case), and in the subsequent development process, the running result of the to-be-processed test case can become not passed due to the change of the interface by the business personnel. In this case, the code change information in the development process can also be obtained.

[0128] At least one of the running error logs or the code change information can be used as the running exception information, and the running exception information and the to-be-processed test case are input to the large model. The large model is instructed by the test case repair prompt text to repair the to-be-processed test case based on the running exception information, and updated to-be-processed test cases output by the large model are obtained.

[0129] Figure 4 A flowchart of a test case updating process provided by the embodiments of the present disclosure is shown in FIG. 1. Figure 4 The test case file corresponding to the to-be-processed test case is run, and the corresponding running result is obtained. When the running result is not passed, the path information (i.e., path) error in the to-be-processed test case is detected, and the corresponding running error logs are generated. The to-be-processed test case and the running error logs are input to the large model. The large model is instructed by the test case repair prompt text to output the repair of the path information in the to-be-processed test case, and correct path information (i.e., the italic part) is obtained, and then the updated to-be-processed test case is generated.

[0130] In the embodiments of the present disclosure, the running exception information is obtained, and the large model is instructed by the test case repair prompt text to repair the interface test case based on the running exception information. The semantic understanding ability of the large model is fully utilized, and the model for test case repair is not additionally trained, so that the automatic repair of the interface test case is realized, and the usability of the interface test case is improved.

[0131] Further, for the interface test case of abnormal interruption, the interface test case is updated by combining the running error log and the code change information, which provides more rich prompt information for the large model, helps to improve the repair effect of the large model on the interface test case, and ensures the continuous availability of the interface test case.

[0132] As an optional embodiment, the to-be-processed test case is updated to obtain an updated to-be-processed test case, including:

[0133] Obtaining a new request log corresponding to the target interface;

[0134] Based on the new request log, a first interface test case corresponding to the new request log is generated;

[0135] The first interface test case is used as the updated to-be-processed test case.

[0136] Specifically, if the running result corresponding to the to-be-processed test case is not passed, a new request log for the same target interface can be reacquired, and based on the new request log, a first interface test case corresponding to the new request log is generated according to the test case generation method provided in the above embodiment, and the first interface test case is used as the updated to-be-processed test case.

[0137] Optionally, when the target interface only has one request log corresponding thereto, or all request logs corresponding to the target interface cannot generate an interface test case that passes the running, a method based on a large model can be used to update the test case.

[0138] Optionally, the to-be-processed test case can also be updated in combination with the two update modes, for example, when the number of execution of the update operation based on the large model is greater than the preset execution number, such as when the running result of the 5 update operations is still not passed, a new request log can be acquired, and the test case is updated based on the new request log.

[0139] It should be noted that different update modes of the to-be-processed test case can be adaptively selected for different application scenarios. By providing a plurality of different update modes of the interface test case, the interface test case can be effectively automatically repaired in various situations, and the availability of the interface test case is further improved.

[0140] As an optional embodiment, the method further includes:

[0141] Determining the interface name of the target interface;

[0142] The interface test case file corresponding to the target interface is generated according to a preset file format, and the interface name is taken as the file name of the interface test case file.

[0143] If the interface test case corresponding to the target interface is updated, the file name of the updated interface test case file is kept as the interface name.

[0144] Specifically, after obtaining the interface test case corresponding to the target interface, the interface name of the target interface can be obtained by parsing the URL in the corresponding request log. The generated interface test case corresponding to the target interface is in a preset file format, and the interface name is taken as the file name of the interface test case file.

[0145] For example, the preset file format can be python (a program language), and the interface test case corresponding to the target interface can be written into a python file to generate a python executable file with the file name [interface name].py.

[0146] On this basis, when the interface test case corresponding to the target interface is updated, the content of the interface test case in the corresponding interface test case file may be updated, but the file name of the updated interface test case file is still kept as the interface name. For example, after updating the interface test case to obtain the updated interface test case file, the file name of the updated interface test case file can be overwritten with the interface name corresponding to the target interface.

[0147] In the embodiments of the present disclosure, the interface name corresponding to the target interface is taken as the file name of the corresponding test case file, so that the main function of the corresponding interface can be known through the file name of the test case file, and any interface test case file corresponding to an interface can be quickly queried and located through the file name of the test case file.

[0148] Further, when the interface test case corresponding to the target interface is updated, the file name of the updated interface test case file is kept as the interface name, so that the file name of the interface test case file corresponding to the target interface is always the interface name, that is, whether the content of the interface test case is updated or not, the corresponding interface test case file can be uniquely indexed according to the interface name, which helps to better manage the interface test case file.

[0149] As an optional embodiment, the request log is obtained, including:

[0150] The plurality of initial request logs are obtained from a database;

[0151] The large model analyzes each initial request log according to the error log screening prompt text to obtain an error label and an error reason corresponding to each initial request log.

[0152] Based on the error label and the error reason corresponding to each initial request log, at least one error log is screened from the plurality of initial request logs.

[0153] Based on the plurality of initial request logs excluding the at least one error log, a request log is determined.

[0154] Specifically, to obtain the request log, a plurality of initial request logs can be obtained from a database, wherein the plurality of initial request logs can be collected request logs related to the interface.

[0155] Optionally, a plug-in can be set to collect original request logs, and the collected original request logs can be judged to filter out original request logs irrelevant to the interface to obtain a plurality of original request logs for requesting the interface. For each original request log, data cleaning can be performed on the original request log to clean up information irrelevant to generating the interface test case, and key information for generating the interface test case is retained, for example, the key information can include URL, request body and response packet. Then, the original request log including the key information is converted into a standard format to obtain a plurality of initial request logs, and the plurality of initial request logs are persistently stored in the database.

[0156] In a large number of initial request logs, a large number of error request logs can be mixed, in order to ensure the effectiveness of the request log for generating the interface test case, the plurality of initial request logs can be screened. The plurality of initial request logs are input into the large model, and the large model analyzes each initial request log according to the error log screening prompt text to obtain an error label and an error reason corresponding to each initial request log, wherein the error label can be a judgment result of the large model outputting whether the initial request log is an error log, and the error reason can be a reason corresponding to the result of the large model obtaining the error label.

[0157] Figure 5 A schematic diagram of an error log screening process provided by the embodiments of the present disclosure is shown in Figure 5 The initial request log is input into the large model, and the large model can combine the "code" information and the "message" information in the initial request log to determine the corresponding error label and error reason according to the error log screening prompt text.

[0158] In the prior art, it is usually directly judged whether it is an error log based on the data of "code", when the "code" is 0, it is a correct log; when the "code" is "-1", it is an error log, but in fact, there may be a case that the function logic in the test process is wrong and the "code" is still "0". In the embodiment of the present disclosure, the "code" information and the "message" information in the initial request log are combined for comprehensive judgment, and the reliability of the error label is judged by the error reason auxiliary judgment, thereby improving the accuracy of error log screening.

[0159] For each initial request log, the corresponding error label and error reason can be comprehensively analyzed to determine whether the initial request log is an error log, and at least one error log is screened out. At least one error log is deleted from the plurality of initial request logs to obtain a set composed of a plurality of correct logs, and a request log is determined from the plurality of correct logs.

[0160] In the embodiment of the present disclosure, by screening the correct logs in the plurality of initial requests, the effectiveness of the request log used to generate the interface test case is ensured, and the usability of the interface test case generated based on the request log is ensured.

[0161] Further, the error log screening prompt text instructs the large model to analyze each initial request log to obtain the error label and error reason corresponding to each initial request log, and the error label and error reason corresponding to each initial request log are screened based on the error label and error reason corresponding to each initial request log, instead of directly screening the error label output by the large model, thereby avoiding the judgment error of the error log caused by the illusion of the large model. The error reason is used as an explainable basis for the error label, and the two are combined for comprehensive judgment, thereby improving the accuracy of error log screening and further ensuring the effectiveness of the request log used to generate the interface test case.

[0162] As an optional embodiment, the method further comprises:

[0163] The association analysis prompt text instructs the large model to analyze the association relationship between a plurality of interfaces corresponding to a plurality of request logs respectively to obtain at least one interface group; the interface group includes at least two related interfaces for the same scene;

[0164] For each interface group, based on the interface test cases corresponding to each interface in the interface group and the association relationship between each interface in the interface group, a scene test case for the scene corresponding to the interface group is obtained.

[0165] Specifically, in actual application, the calls between interfaces are not independent of each other, and there is a certain correlation between multiple interfaces in the same scenario. For example, in the scenario of hospital treatment, the object to be treated needs to register an object account in the system of the hospital first, and the registration operation corresponds to an interface A. Then the object can use the object account to make an appointment in the hospital system, and the registration operation corresponds to an interface B. Then the object needs to make a payment in the hospital system, and the payment operation corresponds to an interface C. In the above scenario, there is a dependent correlation between interface A, interface B and interface C in terms of data use. Therefore, constructing a scenario test case for the same scenario is more in line with the real use of the object in the business.

[0166] The multiple request logs can be input to the large model, and the large model is instructed to analyze the correlation between the multiple interfaces corresponding to the multiple request logs respectively, to mine typical scenarios in the business, and to take multiple related interfaces for the same scenario as an interface group.

[0167] For each interface group, after obtaining the interface test cases corresponding to the multiple interfaces in the interface group respectively, the multiple interface test cases can be combined based on the correlation between the interfaces to obtain a scenario test case for the scenario corresponding to the interface group.

[0168] Optionally, considering that the behaviors of multiple objects in the same scenario are relatively close in time, the multiple request logs can be grouped based on time information corresponding to the multiple request logs respectively, the multiple request logs within a preset time period are grouped into a group, or the multiple request logs with a time interval less than a preset time interval threshold are grouped into a group, and the multiple request logs in each group are input to the large model respectively, and the large model analyzes the correlation between the interfaces corresponding to the multiple request logs in each group. By grouping the multiple request logs in advance, the data amount processed by the large model is reduced, and the effect of analyzing the correlation by the large model is improved.

[0169] In the embodiments of the present disclosure, the correlation between the interfaces is mined by the large model, and a scenario test case composed of a serial whole of the interface test cases corresponding to the interfaces in the interface group is formed, the discrete object use behavior information is connected into an object use scenario, so that the generated scenario test case is more complete and closer to the actual situation, so that the problems of each interface can be found more effectively based on the scenario test case for testing, and the effectiveness of the interface test is improved. At the same time, manual identification and assembly by understanding the scenario artificially are avoided, the labor cost is saved, and the result is also not affected by the subjective experience of human beings.

[0170] As an optional embodiment, the method further includes:

[0171] The attribute judgment prompt text instructs the large model to analyze the request log to obtain the candidate interface attribute of the target interface and the judgment reason;

[0172] Based on the candidate interface attribute and the judgment reason corresponding to the target interface, the interface attribute of the target interface is determined.

[0173] If the interface attribute of the target interface meets the preset test condition, the interface test case corresponding to the target interface is tested.

[0174] Specifically, the interface also has an interface attribute corresponding thereto, and the interface attribute can include a read interface and a write interface. The read interface is not easy to affect the object using front-end device, while the write interface is easy to affect the object using front-end device. For example, when a write interface needs to modify the name of a small program during testing, if the object is accessing the small program at this time, it will be found that the name of the small program has changed, and the object experience is poor.

[0175] Therefore, the interface attribute of the interface can also be judged. The request log corresponding to the target interface is input to the large model, the large model is instructed to analyze the request log through the attribute judgment prompt text, and the candidate interface attribute of the target interface and the judgment reason are obtained. The candidate interface attribute and the judgment reason output by the large model are comprehensively analyzed to obtain the interface attribute of the target interface.

[0176] Figure 6 A schematic diagram of an interface attribute judgment process provided by the embodiments of the present disclosure is shown in FIG. 1. Figure 6 As shown in FIG. 1, for request log 1, the request log 1 can be input to the large model, the large model can comprehensively analyze the URL, request body and response information in the request log, and obtain the candidate interface attribute of the interface corresponding to the request log 1 as “read” and give the corresponding judgment reason. Similarly, for request log 2, the request log 2 can be input to the large model, and the candidate interface attribute of the interface corresponding to the request log 2 is obtained as “write” and the corresponding judgment reason is given, so that the final interface attribute can be determined based on the candidate interface attribute and the judgment reason.

[0177] In fact, not all interface names are named according to unified specifications, and there may be some unknown interface names. If the interface attribute is directly identified based on the interface name, it may lead to incorrect identification of the interface attribute. In the embodiments of the present disclosure, the final interface attribute is determined by combining the candidate interface attribute and the judgment reason output by the large model. The judgment reason is used as an explainable basis for the candidate interface attribute, and the two are comprehensively judged to improve the accuracy of the interface attribute judgment.

[0178] Before the interface test case corresponding to the target interface is put online, it can be judged whether the interface attribute of the target interface meets the preset test condition. If it meets, the interface test case can be put online for testing. If it does not meet, the interface test case can not be put online for testing or the time or frequency of putting the interface test case online can be adjusted.

[0179] For example, after obtaining the interface attribute of the target interface, it can be judged whether the interface attribute is a read interface. If it is a read interface, the interface test case can be put online. If it is a write interface, the interface test case can not be put online or the interface test case can be arranged for testing within a test time period, so as to minimize the influence of interface testing on the object use front-end equipment and realize non-invasive interface testing of business.

[0180] As an optional embodiment, the method comprises:

[0181] updating the large model to a target large model;

[0182] The target large model is obtained by training the large model based on a plurality of positive samples and a plurality of negative samples.

[0183] The plurality of positive samples are determined based on a plurality of request logs and second interface test cases corresponding to the plurality of request logs respectively; and the running result of the second interface test case is passed.

[0184] The plurality of negative samples are determined based on a plurality of request logs and third interface test cases corresponding to the plurality of request logs respectively; and the third interface test case comprises an interface test case with a corresponding running result of not passed and an interface test case obtained by modifying the second interface test case.

[0185] Specifically, the test case generation method provided by the embodiments of the present disclosure can automatically generate corresponding interface test cases based on the general large model based on request logs. In actual application process, since the request logs are endless, a large number of interface test cases can be automatically generated. Therefore, the large number of generated interface test cases can be used as training data to train the large model. The trained large model is used again to generate subsequent interface test cases. By optimizing the large model, the quality of the generated interface test cases is further improved.

[0186] After the plurality of interface test cases are generated based on the plurality of request logs by the large model, interface test case files corresponding to the plurality of interface test cases are generated, and running results corresponding to each interface test case are obtained by running the plurality of interface test case files. If the running result is pass, the interface test case can be used as a second interface test case; if the running result is fail, the interface test case can be used as a third interface test case, or the interface test case that fails can be updated until the running result corresponding to the interface test case is pass, and the updated interface test case that passes is used as a second interface test case.

[0187] In addition, after a plurality of second interface test cases that pass are obtained, for each second interface test case, the second interface test case can be modified, and the modified second interface test case is used as a third interface test case.

[0188] For each second interface test case, the second interface test case and the request log corresponding thereto can be used as a positive sample; for each third interface test case, the third interface test case and the request log corresponding thereto can be used as a negative sample.

[0189] The large model is trained based on the plurality of positive samples and the plurality of negative samples, the trained large model is used as a target large model, and the large model is updated to the target large model for subsequent automatic generation of interface test cases based on the target large model based on request logs.

[0190] In the embodiments of the present disclosure, a large number of interface test cases generated by the large model are used to construct a plurality of positive samples and a plurality of negative samples, and the large model is trained based on the plurality of positive samples and the plurality of negative samples, so that the ability of the trained target large model to generate interface test cases based on request logs is improved, which helps to further improve the quality of the generated interface test cases.

[0191] Figure 7 A schematic diagram of a test case processing process provided by the embodiments of the present disclosure is shown in FIG. 1. Figure 7 As shown in FIG. 1, in the preprocessing stage, a plurality of initial request logs can be stored in the database, and the large model can be prompted to screen the plurality of initial request logs by error log filtering prompt text, and a plurality of error-free logs are obtained.

[0192] For example, F' represents a Boolean function, which represents using LLM to analyze whether the log is correct or not, l' represents an initial request log, L' represents a set of initial request logs, and C represents a set of error-free logs. The process of generating error-free logs can be represented as:

[0193] C = {F'(l') = True | l' ∈ L'}

[0194] The request log can be obtained from multiple error-free logs, and the interface properties of the target interface corresponding to the request log are obtained by prompting the large model to analyze the request log according to the attribute judgment prompt text.

[0195] In the test case generation phase, the input information is determined based on the request log, the test case generation prompt text and the case template, the input information is input into the large model, the test case generation operation is performed through the large model, the interface test case including the case description, the case body and the case assertion is obtained, and the corresponding executable interface test case file is generated in the preset format.

[0196] The specific process of the test case generation operation can include:

[0197] The LLM automatically analyzes the interface function purpose and returns a clear and concise case description. F represents the LLM operation, l is a log in C, and p d represents the case description generation prompt text, and the case description tc d of the test case tc is generated as follows:

[0198]

[0199] The LLM parses the request response packet and extracts the case assertion, thereby ensuring the stability of the interface function in multiple dimensions. F represents the LLM operation, l is a log in C, and p a represents the case assertion generation prompt text, and the case assertion generation prompt text tc a of the test case tc is generated as follows:

[0200]

[0201] The LLM can analyze the parts that need to be generated in the new case according to the case template and the request log, such as the requested URL, the request body, the case name, etc. F represents the LLM operation, l is a log in C, and p b represents the case body generation prompt text, and the case body generation prompt text tc b of the test case tc is generated in combination with the case template T as follows:

[0202]

[0203] The above case elements are summarized as tc={tc d ,tc a ,tc b}, and are automatically written into a python file, and the interface name is automatically parsed as a case file name, that is, a python executable file f of {interface name}.py is generated.

[0204] In the test case update phase, a python sub-process is started to run the generated executable case file f. If the running result is passed, the subsequent message notification and case submission process is entered; otherwise, if the running result is not passed, the running error log e is obtained, and the generated tc information and the running error log e are input into the LLM together to obtain the case repair prompt text p r Indicating that the large model regenerates a new test case tc repaired and completed new :

[0205]

[0206] Obtaining tc new , and the interface name is used as the case file name, and the python executable file f of {interface name}.py is overwritten. The newly generated f is re-input into the case executor, and if the running is successful, the subsequent steps are directly entered; if the running is not passed, a new error running log is obtained, and the above steps are re-executed to complete the repair.

[0207] The test case processing method provided by the embodiment of the present disclosure uses a one-time code template of a business, analyzes the template and the request data through AI, and then generates an interface test case unique to the business. Due to the characteristics of AI, the diversification of the input parameter type can be intervened by AI to generate parameters of different types, value ranges, and different descriptions. AI can dynamically determine whether the information of the return packet is consistent with the expected result of the input parameter according to the difference of the input parameter, so as to increase the usability of the interface automatic case. After the automatic case is generated, a script is used to automatically deploy the environment locally. Due to the differences in the deployment of different business environments, some problems may exist. The error output when the environment is started can be analyzed through AI, and self-repair is attempted according to the suggestions given by AI to reduce the labor input in the process. The interface automatic case may also exist abnormal interruption when trying to pre-execute. AI can perform self-repair of the case through the program information thrown by the abnormal interruption, and then generate a brand new interface automatic case to ensure the continuous usability of the case.

[0208] Figure 8 The system architecture diagram of the test system provided by the embodiment of the present disclosure is shown in Figure 8 The system architecture of the test system can include a log layer, a logic layer and a data layer.

[0209] The log layer can include a data reporting module, a data formatting module, and a persistent storage module. The data reporting module is mainly used to collect logs, and is mainly used to implement Node.js (a JavaScript runtime based on Chrome V8 engine), and is used to report request data of a Whistle (a framework mainly used for mobile terminal automated testing) proxy to a designated API server. It uses the Axios library (a Promise-based HTTP client for browsers and node.js environments) to send HTTP requests, and uses the Crypto (a module provided by Node.js for encrypting and decrypting data) and Util (a Node.js auxiliary module that provides a series of utility functions) modules of Node.js to process request data and generate signatures. The data reporting module can also include two filters, ExcludeFilter (exclusion filter) and IncludeFilter (include filter). If the URL of a request hits the exclusion filter, it is excluded and not reported; if left blank, it means that the exclusion filter is not enabled. If the URL of a request hits the include filter, it is reported; if left blank, it means that the filter is not enabled, and the logs unrelated to the interface are filtered out through the exclusion filter and the include filter.

[0210] The data formatting module is mainly used to convert the format of request logs into a unified standard format, and is specifically used to convert HTTP session information into a "dataDraft" format (a data format) that includes all key information of requests and responses; generate a SHA256 (a secure hash algorithm) signature using a timestamp and a token for request authorization; generate an HTTP request header containing AppID, timestamp, user, and signature; send an HTTP request to the API server through the axios library and print the request and response logs; upload "dataDraft" to the API server, and throw an error if the request fails; create a new API definition on the API server, and also throw an error if the request fails.

[0211] The persistent storage module is used to store URL, hostname, path, protocol, request / response header information, and request / response body data.

[0212] In actual application, the whistle plug-in is installed through a command, and the business personnel activates the plug-in through one-time configuration. According to the product characteristics of the business, the test personnel in the business configure the mobile phone, PC or other networkable devices to be proxied. Subsequent interface information of the business will be uploaded to the server for persistent storage through the plug-in.

[0213] The logic layer includes an artificial intelligence module, a service management module, a transaction management module, and a use case management module. The artificial intelligence module further includes a plurality of sub-modules, and the specific functions of each sub-module can be referred to the description of the corresponding embodiments in the foregoing, which will not be described herein again.

[0214] The service management module further includes an add service sub-module, an update service sub-module, a query service sub-module, and a delete service sub-module. The add service sub-module is configured to add a new service to the system by user input or automatic recognition, including a service name, interface information, and the like. The service information further includes a business interface use case template T. The update service sub-module is configured to modify an existing service, such as updating interface information, modifying a service name, and the like. The query service sub-module is configured to quickly find a specified service by keyword search, classification filtering, and the like. The delete service sub-module is configured to remove a service no longer needed from the system.

[0215] The transaction management module further includes an add transaction sub-module, an update transaction sub-module, a query transaction sub-module, and a delete transaction sub-module. The add transaction sub-module is configured to add a new transaction to the system by user input or automatic recognition, including a transaction name, associated interfaces, and the like. The update transaction sub-module is configured to modify an existing transaction, such as updating associated interfaces, modifying a transaction name, and the like. The query transaction sub-module is configured to quickly find a specified transaction by keyword search, classification filtering, and the like. The delete transaction sub-module is configured to remove a transaction no longer needed from the system.

[0216] The use case management module further includes an add use case sub-module, an update use case sub-module, a query use case sub-module, and a delete use case sub-module. The add use case sub-module is configured to add a new use case to the system by object input or AI automatic generation, including a use case name, associated interfaces, parameter settings, assertion conditions, and the like. The update use case sub-module is configured to modify an existing use case, such as updating associated interfaces, modifying a use case name, adjusting parameter settings, modifying assertion conditions, and the like. The query use case sub-module is configured to quickly find a specified use case by keyword search, classification filtering, and the like. The delete use case sub-module is configured to remove a use case no longer needed from the system.

[0217] The data layer can use a database model defined by the SQLAlchemy ORM framework, and can include a service table (Service), a transaction table (Transaction), and a use case table (Testcase). The relationship among the three can be: one Service corresponds to multiple Transactions, which are associated through the id field of the Service table and the id field of the Transaction table. The sub_name field is used to support scenarios of different modules of the same business, and the template (i.e., use case template) is used to enable the business to have a more expected result. One Transaction corresponds to multiple TestCases, which are associated through the id field of the Transaction table and the transaction_id field of the TestCase table. The transaction itself is associated with the effect of the testcase, which helps the platform and the business to quickly measure the value brought by AI to the business in the process by realizing the binding on the data. The generation of these data also helps the business to have a better process data perception. At the same time, training samples for optimizing large models can be collected, so that the core capabilities of large models can be continuously improved.

[0218] The test system provided by the embodiments of the present disclosure can intelligently analyze test log data and automatically generate interface automation test cases, and has the capabilities of intelligently generating test case descriptions, intelligently repairing test cases, intelligently analyzing interface read-write characteristics, and the like. The whole process does not require human intervention, and the software product interface quality monitoring is efficiently guaranteed. Based on the combination of data and target automation test case templates, the inference capability of LLM is used to batch and real-time generate automation test cases, and the generated test cases have the characteristics of high quality, high efficiency, and self-repairing in the stages of test case generation, environment dependency installation, smoking test, and updating.

[0219] Figure 9 A structural diagram of a test case generation device provided by the embodiments of the present disclosure is shown in FIG. 1, which can include the following modules: Figure 9

[0220] The data acquisition module 210 is configured to acquire a request log corresponding to a target interface, and acquire a test case generation prompt text and a test case template. The test case generation prompt text includes a test case description generation prompt text, a test case assertion generation prompt text, and a test case body generation prompt text.

[0221] The input information determination module 220 is configured to determine input information based on the request log, the test case template, and the test case generation prompt text.

[0222] ​The test case generation module 230 is configured to instruct the large model to perform a test case generation operation based on the input information, to obtain an interface test case for the target interface; the interface test case includes a use case description, a use case assertion, and a use case body.

[0223] The test case generation operation includes:

[0224] The use case description is generated based on the use case description generation prompt text and the request log.

[0225] The use case assertion is generated based on the use case assertion generation prompt text and the request log.

[0226] The use case body is generated based on the use case body generation prompt text, the use case template, and the request log.

[0227] As an optional embodiment, the device further includes:

[0228] The test case update module is configured to:

[0229] Perform at least one update operation on the interface test case until the running result corresponding to the interface test case is passed.

[0230] The update operation includes:

[0231] Determine a to-be-processed test case; the to-be-processed test case corresponding to the first update operation is the interface test case.

[0232] Run a test case file corresponding to the to-be-processed test case to obtain a corresponding running result.

[0233] If the running result is not passed, update the to-be-processed test case to obtain an updated to-be-processed test case, and use the updated to-be-processed test case as the to-be-processed test case corresponding to the next update operation.

[0234] As an optional embodiment, when the test case update module updates the to-be-processed test case to obtain an updated to-be-processed test case, the test case update module is specifically configured to:

[0235] Obtain running exception information corresponding to the to-be-processed test case; the running exception information includes at least one of a running error log and code change information.

[0236] Instruct the large model to repair the to-be-processed test case based on the running exception information, to obtain the updated to-be-processed test case, through a use case repair prompt text.

[0237] As an optional embodiment, the test case updating module, when updating the to-be-processed test case to obtain an updated to-be-processed test case, is specifically configured to:

[0238] obtain a new request log corresponding to the target interface;

[0239] generate a first interface test case corresponding to the new request log based on the new request log;

[0240] use the first interface test case as the updated to-be-processed test case.

[0241] As an optional embodiment, the device further comprises a file generation module configured to:

[0242] determine an interface name of the target interface;

[0243] generate a corresponding interface test case file in a preset file format based on the interface test case corresponding to the target interface, and use the interface name as a file name of the interface test case file;

[0244] if the interface test case corresponding to the target interface is updated, keep the file name of the updated interface test case file as the interface name.

[0245] As an optional embodiment, the data acquisition module, when acquiring the request log, is specifically configured to:

[0246] acquire a plurality of initial request logs from a database;

[0247] instruct the large model to analyze each initial request log to obtain an error label and an error reason corresponding to each initial request log by error log screening prompt text;

[0248] select at least one error log from the plurality of initial request logs based on the error label and the error reason corresponding to each initial request log;

[0249] determine the request log based on the plurality of initial request logs other than the at least one error log.

[0250] As an optional embodiment, the device further comprises a scene test case generation module configured to:

[0251] instruct the large model to analyze the association between a plurality of interfaces corresponding to a plurality of request logs to obtain at least one interface group by association analysis prompt text; the interface group comprises at least two related interfaces for the same scene;

[0252] For each interface group, based on the interface test cases respectively corresponding to each interface in the interface group and the association relationship between each interface in the interface group, a scenario test case corresponding to a scenario of the interface group is obtained.

[0253] As an optional embodiment, the apparatus further includes an attribute judgment module configured to:

[0254] The attribute judgment prompt text instructs the large model to analyze the request log to obtain candidate interface attributes and judgment reasons of the target interface;

[0255] Based on the candidate interface attributes and judgment reasons corresponding to the target interface, the interface attribute of the target interface is determined.

[0256] If the interface attribute of the target interface meets a preset test condition, the target interface is tested based on the interface test case corresponding to the target interface.

[0257] As an optional embodiment, the apparatus further includes a model updating module configured to:

[0258] The large model is updated to a target large model.

[0259] The target large model is obtained by training the large model based on a plurality of positive samples and a plurality of negative samples.

[0260] The plurality of positive samples are determined based on a plurality of request logs and second interface test cases respectively corresponding to the plurality of request logs; the second interface test cases correspond to passing running results.

[0261] The plurality of negative samples are determined based on a plurality of request logs and third interface test cases respectively corresponding to the plurality of request logs; the third interface test cases include interface test cases with non-passing running results and interface test cases obtained by modifying the second interface test cases.

[0262] The apparatus of the embodiments of the present disclosure can execute the method provided by the embodiments of the present disclosure, the implementation principles of which are similar, and have corresponding technical effects. The actions performed by each module in the apparatus of each embodiment of the present disclosure correspond to the steps in the method of each embodiment of the present disclosure. For detailed descriptions of the functions of each module of the apparatus, refer to the descriptions of the corresponding method in the foregoing, which will not be repeated here.

[0263] In the embodiments of the present disclosure, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined target, and can be implemented entirely or partially by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an integral module or unit that includes the functions of the module or unit.

[0264] An electronic device is provided in an embodiment of the present disclosure, including a memory, a processor, and a computer program stored on the memory, and the processor executes the computer program to implement the steps of the method provided in any of the optional embodiments of the present disclosure. Compared with the prior art, the following can be achieved: by obtaining the request log corresponding to the target interface, and based on the request log, generating the interface test case corresponding to the target interface through a large model, the semantic understanding ability of the large model is fully utilized to automatically generate the interface test case, without additional training of the model, avoiding the limitation that a large amount of training data cannot be obtained in a specific business, and better compatibility with multiple different businesses, stronger applicability, and larger interface coverage of the generated test case.

[0265] Further, by dividing the interface test case into a case description, a case assertion, and a case body, and setting the case description generation prompt text, the case assertion generation prompt text, and the case body generation prompt text, the integrity of the generated interface test case is ensured, the situation that the generated interface test case is unusable due to missing information is avoided, and the usability of the interface test case is improved.

[0266] Further, by generating the corresponding interface test case based on the request log and the case template, different interface test cases corresponding to different request logs for the same business can all conform to a unified format, which helps the tester to better understand the content and structure of the interface test case, enhances the readability of the interface test case, avoids inconsistencies caused by different business personnel's code writing styles when manually writing the interface test case, and inconsistencies caused by different interface frameworks, and is conducive to subsequent maintenance and modification of the interface test case.

[0267] In one optional embodiment, an electronic device is provided, as shown in Figure 10 Figure 10 ​The electronic device 4000 shown includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 can also include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception, etc. It should be noted that the transceiver 4004 is not limited to one in actual application, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present disclosure.

[0268] The processor 4001 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logical blocks, modules and circuits described in combination with the disclosure. The processor 4001 can also be a combination of computing functions, such as one or more microprocessor combinations, combinations of DSP and microprocessor, etc.

[0269] The bus 4002 can include a path for transmitting information between the above-mentioned components. The bus 4002 can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 10 Only one thick line is used in the middle, but it does not mean that there is only one bus or one type of bus.

[0270] The memory 4003 can be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, a magnetic disk storage or other magnetic storage device, or any other medium capable of storing computer instructions and capable of being read by a computer, without limitation.

[0271] The memory 4003 is configured to store a computer program for implementing the embodiments of the present disclosure, and the processor 4001 is configured to control the execution of the computer program stored in the memory 4003. The processor 4001 is configured to execute the computer program stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.

[0272] The electronic device includes, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Tablet Personal Computer), a PMP (Portable Multimedia Player), a car terminal (for example, a car navigation terminal), a wearable device, and the like, and a fixed terminal such as a digital TV, a desktop computer, and the like.

[0273] The embodiments of the present disclosure provide a computer readable storage medium having a computer program stored thereon, and the computer program is executable by a processor to implement the steps and corresponding contents of the foregoing method embodiments.

[0274] The embodiments of the present disclosure also provide a computer program product including a computer program, and the computer program is executable by a processor to implement the steps and corresponding contents of the foregoing method embodiments.

[0275] It should be understood that although the various operation steps in the flowcharts of the embodiments of the present disclosure are indicated by arrows, the implementation order of the steps is not limited to the order indicated by the arrows. Unless otherwise specified herein, in some implementation scenarios of the embodiments of the present disclosure, the implementation steps in each flowchart can be executed in other orders as required. In addition, part or all of the steps in each flowchart can include multiple sub-steps or multiple stages based on the actual implementation scenario. Part or all of these sub-steps or stages can be executed at the same time, and each of these sub-steps or stages can also be executed at different times. In the scenario where the execution times are different, the execution order of these sub-steps or stages can be flexibly configured as required, and the embodiments of the present disclosure do not limit this.

[0276] The above is only an optional implementation of some implementation scenarios of the present disclosure, and it should be pointed out that, for ordinary skilled persons in the technical field, other similar implementation means based on the technical idea of the present disclosure without departing from the technical concept of the present disclosure also belong to the protection scope of the embodiments of the present disclosure.

Claims

1. A test case generation method, characterized in that, include: Obtain the request logs corresponding to the target interface, and obtain the test case generation prompt text and test case template; The test case generation prompt text includes test case description generation prompt text, test case assertion generation prompt text, and test case body generation prompt text; Based on the request log, the test case template, and the test case, a prompt text is generated to determine the input information; The input information is used to instruct the large model to perform a test case generation operation, thereby obtaining interface test cases for the target interface. The interface test cases include test case descriptions, test case assertions, and test case bodies; The test case generation operation includes: Based on the use case description, generate the prompt text and the request log, and generate the use case description; Based on the use case assertion, generate the prompt text and the request log, and generate the use case assertion; The use case body is generated based on the use case body, the prompt text, the use case template, and the request log.

2. The test case generation method according to claim 1, characterized in that, The method further includes: Perform at least one update operation on the interface test case until the execution result of the interface test case is passed. The update operation includes: Identify the test cases to be processed; the test case to be processed corresponding to the first update operation is the interface test case. Run the test case file corresponding to the test case to be processed, and obtain the corresponding running results; If the running result is unsuccessful, the test case to be processed is updated to obtain the updated test case to be processed, and the updated test case to be processed is used as the test case to be processed for the next update operation.

3. The test case generation method according to claim 2, characterized in that, The step of updating the test cases to be processed to obtain updated test cases includes: Obtain runtime exception information corresponding to the test case to be processed; the runtime exception information includes at least one of runtime error logs and code change information. The test case repair prompt text instructs the large model to repair the test cases to be processed based on the runtime exception information, thereby obtaining the updated test cases to be processed.

4. The test case generation method according to claim 2, characterized in that, The step of updating the test cases to be processed to obtain updated test cases includes: Obtain the new request log corresponding to the target interface; Based on the new request log, generate a first interface test case corresponding to the new request log; The first interface test case is used as the updated test case to be processed.

5. The test case generation method according to claim 2, characterized in that, The method further includes: Determine the interface name of the target interface; Based on the interface test cases corresponding to the target interface, generate the corresponding interface test case file according to the preset file format, and use the interface name as the file name of the interface test case file; If the interface test cases corresponding to the target interface are updated, the file name of the updated interface test case file shall be the same as the interface name.

6. The test case generation method according to claim 1, characterized in that, The process of obtaining request logs includes: Retrieve multiple initial request logs from the database; By filtering the error log prompt text, the large model is instructed to analyze each initial request log to obtain the error label and error reason corresponding to each initial request log; Based on the error tags and error reasons corresponding to each initial request log, at least one error log is selected from the multiple initial request logs; The request log is determined based on multiple initial request logs, excluding at least one error log.

7. The test case generation method according to claim 1, characterized in that, The method further includes: The correlation analysis prompt text instructs the large model to analyze the correlation between multiple interfaces corresponding to multiple request logs, and obtain at least one interface group; the interface group includes at least two related interfaces for the same scenario; For each interface group, based on the interface test cases corresponding to each interface in the interface group and the association relationships between the interfaces in the interface group, scenario test cases for the scenario corresponding to the interface group are obtained.

8. The test case generation method according to claim 1, characterized in that, The method further includes: The attribute judgment prompt text instructs the large model to analyze the request log to obtain the candidate interface attributes of the target interface and the judgment reason; Based on the candidate interface attributes and judgment reasons corresponding to the target interface, the interface attributes of the target interface are determined; If the interface attributes of the target interface meet the preset test conditions, then the test is performed based on the interface test cases corresponding to the target interface.

9. The test case generation method according to claim 2, characterized in that, The method further includes: Update the large model to the target large model; The target large model is obtained by training the large model based on multiple positive samples and multiple negative samples; The multiple positive samples are determined based on multiple request logs and their corresponding second interface test cases; the execution result of the second interface test cases is "pass". The multiple negative samples are determined based on multiple request logs and their corresponding third interface test cases; the third interface test cases include interface test cases whose corresponding execution results are failures, and interface test cases obtained by modifying the second interface test cases.

10. A test case generation device, characterized in that, include: The data acquisition module is used to acquire the request logs corresponding to the target interface, and to acquire the test case generation prompt text and test case template; The test case generation prompt text includes test case description generation prompt text, test case assertion generation prompt text, and test case body generation prompt text; The input information determination module is used to generate prompt text based on the request log, the test case template, and the test case to determine the input information; The test case generation module is used to instruct the large model to perform a test case generation operation based on the input information, so as to obtain interface test cases for the target interface; The interface test cases include test case descriptions, test case assertions, and test case bodies; The test case generation operation includes: Based on the use case description, generate the prompt text and the request log, and generate the use case description; Based on the use case assertion, generate the prompt text and the request log, and generate the use case assertion; The use case body is generated based on the use case body, the prompt text, the use case template, and the request log.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

Images