Test case generation method and related device

Abstract

A test case generation method. The method comprises: acquiring a structured rule for a first application programming interface (API) and a natural language description for the first API; querying a domain-specific language (DSL) database on the basis of the natural language description for the first API to obtain a query result, wherein the query result comprises at least one of: a DSL sample where a natural language description part matches the natural language description for the first API, a DSL error correction example, or a DSL syntax; constructing a prompt on the basis of the query result, the natural language description and the structured rule; and inputting the prompt into a language model to generate a target DSL code, wherein the target DSL code is used for testing the first API, or is used for generating a test case for performing integration testing on the first API and a second API. In the method, a DSL sample, an error correction example, and a syntax are dynamically matched from a DSL database to generate a highly controllable and structured context, and a test case is generated by means of language model inference on the basis of the context, thereby integrating unit testing and integration testing, and reducing repeated development.

Description

Test case generation method and related device

[0001] The present application claims priority to the Chinese Patent Application No. 202411295428.3, filed on September 14, 2024, and entitled "A Test Case Generation Method and Related Device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of code development and testing, and in particular to a test case generation method, a test platform, a computing device cluster, a computer readable storage medium, and a computer program product. BACKGROUND

[0003] With the continuous evolution of code development technology, the API-First (Application Programming Interface-First) development mode has gradually become mainstream. API-First refers to a development mode that prioritizes API in the code development process, rather than developing API after writing code. API-First can ensure that the developed application can seamlessly connect with internal and external services, and improve the reusability and maintainability of the application. Moreover, API-First supports parallel work of development teams, and can avoid repeated development and improve application development efficiency by reusing API.

[0004] During development, code testing is usually required to detect and discover potential problems, reduce defects, ensure code correctness and reliability, and improve code quality. In the API-First development mode, unit testing and integration testing are usually performed in units of API. Unit testing and integration testing become important steps in the development phase.

[0005] Currently, for API unit testing and integration testing, the industry provides related tools or components to assist in testing. However, API unit testing and integration testing are not completely connected in the API life cycle process, resulting in a large number of repeated development operations and increasing testing costs. SUMMARY

[0006] The present application provides a test case generation method that can connect unit testing and integration testing in the API life cycle, reduce a large number of repeated development operations, and reduce testing costs. The present application also provides a test platform, a computing device cluster, a computer readable storage medium, and a computer program product corresponding to the above method.

[0007] In a first aspect, the present application provides a test case generation method. The method is applied to a test platform. The test platform is configured to generate test cases for unit testing or integration testing of an application programming interface (API) according to a domain-specific language (DSL). The test platform can be a standalone software system, for example, a standalone test case generation software. The test platform can also be a software system integrated in other software, for example, a software system integrated in an API testing tool, which can take the form of, but not limited to, a software module, a plug-in, a component, or an applet. In some examples, the test platform can also be a hardware system, for example, a cluster of computing devices with test case generation capability, which runs to execute the test case generation method of the present application.

[0008] Specifically, the test platform can obtain a structured rule for a first API and a natural language description for the first API. The natural language description is configured to describe a function or an implementation code of the first API using natural language. Then the test platform queries a DSL database according to the natural language description for the first API, and obtains a query result. The DSL database includes at least one DSL example, DSL error correction example, or DSL syntax, and the at least one DSL example, DSL error correction example, or DSL syntax includes a natural language description part. Accordingly, the query result includes at least one of a DSL example, DSL error correction example, or DSL syntax that matches the natural language description part in the natural language description for the first API. Then the test platform can construct a prompt according to the query result, the natural language description, and the structured rule, input the prompt into a language model, and generate a target DSL code. The target DSL code is configured to test the first API, or to generate test cases for integration testing of the first API and a second API.

[0009] The method supports dynamically matching a DSL example, DSL error correction example, or DSL syntax that matches the natural language description of the API from the DSL database, and can generate a highly controllable and structured context based on the DSL example, DSL error correction example, or DSL syntax and the structured rule for the API. The prompt can be assembled based on the context, and the language model can be prompted for learning, so that the test cases related to the user's intention can be generated in a unified manner. Since the unit testing and the integration testing use the same programming language and the same process, the integration testing can reuse the test cases of the unit testing, without the need for repeated development, thereby reducing the testing cost. Moreover, the method does not require additional learning of other programming languages, thereby reducing the development threshold of the test cases. In addition, the method uses the language model to generate the test cases, which can be applied to the generation of test cases with complex logic, and has high flexibility and usability.

[0010] In some possible implementation manners, the test platform can preprocess the natural language description to obtain an atomic interface description tuple of the first API. The atomic interface description tuple includes an atomic API operation described using the natural language. Then the test platform queries the DSL database according to the atomic interface description tuple of the first API.

[0011] The method can improve the query efficiency and the query accuracy by processing the natural language description into the atomic interface description tuple and matching the DSL database based on the atomic interface description tuple.

[0012] In some possible implementation manners, the test platform can further parse the target DSL code to obtain a syntax tree, and generate a rule tree according to the structured rule. For a first node in the syntax tree, the test platform can correct the first node using a rule indicated by a second node corresponding to the first node in the rule tree to obtain a corrected syntax tree. Then the test platform can obtain the corrected target DSL code according to the corrected syntax tree.

[0013] The method further supports parsing the generated target DSL code into the syntax tree, and automatically correcting the syntax tree by using the rule tree, to improve the robustness.

[0014] In some possible implementation manners, the query result includes a DSL example and a DSL correction example. Accordingly, the test platform can generate an initial rule tree according to the structured rule, and generate an extended rule according to the DSL example and the DSL correction example. Then the test platform can update the initial rule tree according to the extended rule to obtain the rule tree. In the rule tree, a plurality of rules indicated by a same node are arranged according to the priority, or are processed according to the set logic. The set logic can be set according to experience, for example, the set logic can be an intersection.

[0015] In the method, the test platform further performs rule extension in combination with the DSL example and the DSL correction example, and performs correction in combination with the extended rule, which can improve the comprehensiveness and the accuracy of the correction.

[0016] In some possible implementation manners, the test platform can further display the target DSL code through a graphical editing interface. Alternatively, the test platform can further display the target DSL code through a script editing interface. In this way, the user can process the target DSL code, and the code quality can be improved.

[0017] In some possible implementation manners, the test platform can further receive feedback of the target DSL code of the user through the graphical editing interface or the script editing interface. The feedback includes the structured rule or the natural language description. In this way, the natural language description or the structured rule based on the user feedback can be implemented, and the generation capability of the language model can be used for iterative optimization, to further improve the code quality.

[0018] In some possible implementation manners, the test platform can obtain a first vector and at least one second vector. The first vector is obtained by vectorizing the atomic interface description tuple of the first API, and the second vector is obtained by vectorizing the natural language description part in at least one DSL example or at least one DSL error correction example in the DSL database. The test platform can determine the similarity of the first vector and the at least one second vector, and then obtain at least one of the DSL example, the DSL error correction example or the DSL grammar matched with the atomic interface description tuple according to the similarity. The method can improve the query efficiency and the query accuracy by querying the DSL database according to the vector similarity.

[0019] In some possible implementation manners, the DSL includes JavaScript, a JavaScript-based derivative language or Extensible Markup Language (XML). The DSL can be used as a test script to connect to each functional module in the service, so as to realize interaction with the service. Moreover, the DSL has scalability, for example, the DSL supports custom JS objects and functions, and can implement unit testing of a single API and integration testing of multiple APIs.

[0020] In a second aspect, the present application provides a test platform. The test platform is configured to generate a test case for unit testing or integration testing of an application programming interface (API) according to a domain-specific language (DSL), and the test platform comprises:

[0021] a description and structure rule input module configured to obtain a structured rule for a first API and a natural language description for the first API, the natural language description being configured to describe a function or an implementation code of the first API using a natural language;

[0022] a dynamic prompt word construction module configured to query a DSL database according to the natural language description for the first API, to obtain a query result, the DSL database comprising at least one DSL example, DSL error correction example or DSL grammar, the at least one DSL example, DSL error correction example or DSL grammar comprising a natural language description part, the query result comprising at least one of the DSL example, DSL error correction example or DSL grammar matched with the natural language description part for the first API; and constructing a prompt according to the query result, the natural language description and the structured rule;

[0023] a generation module configured to input the prompt into a language model, to generate a target DSL code, the target DSL code being configured to test the first API, or to generate a test case for integration testing of the first API and a second API.

[0024] In some possible implementation manners, the dynamic prompt word construction module is specifically configured to:

[0025] The natural language description is preprocessed to obtain an atomic interface description tuple of the first API, and the atomic interface description tuple includes an atomic API operation described using natural language.

[0026] The atomic interface description tuple of the first API is used to query a DSL database.

[0027] In some possible implementation manners, the test platform further includes:

[0028] An interpretation and correction module is configured to parse the target DSL code to obtain a syntax tree, and generate a rule tree according to the structured rule.

[0029] For a first node in the syntax tree, the first node is corrected using a rule indicated by a second node corresponding to the first node in the rule tree to obtain a corrected syntax tree, and the target DSL code is corrected according to the corrected syntax tree.

[0030] In some possible implementation manners, the query result includes the DSL sample and the DSL error correction example.

[0031] The interpretation and correction module is specifically configured to:

[0032] An initial rule tree is generated according to the structured rule.

[0033] An extended rule is generated according to the DSL sample and the DSL error correction example.

[0034] The initial rule tree is updated according to the extended rule to obtain the rule tree, and a plurality of rules indicated by a same node in the rule tree are arranged according to a priority or processed according to a set logic.

[0035] In some possible implementation manners, the interpretation and correction module is further configured to:

[0036] The target DSL code is displayed through a graphical editing interface; or

[0037] The target DSL code is displayed through a script editing interface.

[0038] In some possible implementation manners, the interpretation and correction module is further configured to:

[0039] Feedback of the user on the target DSL code through the graphical editing interface or the script editing interface is received, and the feedback includes a structured rule or a natural language description.

[0040] In some possible implementations, the DSL includes JavaScript, a JavaScript-based derivative language, or Extensible Markup Language (XML).

[0041] In a third aspect, the present application provides a computing device cluster. The computing device cluster includes at least one computing device, and the at least one computing device includes at least one processor and at least one memory. The at least one processor and the at least one memory are in communication with each other. The at least one processor is configured to execute instructions stored in the at least one memory, so that the computing device or the computing device cluster performs the test case generation method according to the first aspect or any implementation manner of the first aspect.

[0042] In a fourth aspect, the present application provides a computer-readable storage medium, and the computer-readable storage medium stores instructions. The instructions instruct a computing device or a computing device cluster to perform the test case generation method according to the first aspect or any implementation manner of the first aspect.

[0043] In a fifth aspect, the present application provides a computer program product including instructions, which, when executed on a computing device or a computing device cluster, cause the computing device or the computing device cluster to perform the test case generation method according to the first aspect or any implementation manner of the first aspect.

[0044] On the basis of the implementation manners of the aspects described above, the present application can be further combined to provide more implementation manners. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical method of the present application, the required drawings will be briefly introduced as follows.

[0046] FIG. 1 is a schematic diagram of a test case generated by a graphical interface for integrated testing according to the present application;

[0047] FIG. 2 is a schematic diagram of an architecture of a test platform according to the present application;

[0048] FIG. 3 is a flowchart of a test case generation method according to the present application;

[0049] FIG. 4 is a flowchart of pre-processing of input corpus and construction of prompts according to the present application;

[0050] FIG. 5 is a schematic diagram of dynamic matching of DSL examples according to the present application;

[0051] FIG. 6 is a flowchart of dynamic matching of DSL examples according to the present application;

[0052] FIG. 7 is a schematic diagram of error correction of target DSL code according to the present application;

[0053] FIG. 8 is a schematic diagram of a design of a DSL provided in the present application;

[0054] FIG. 9 is a schematic diagram of an application scenario of a test case generation method provided in the present application;

[0055] FIG. 10 is a schematic diagram of a structure of a computing device provided in the present application;

[0056] FIG. 11 is a schematic diagram of a structure of a computing device cluster provided in the present application;

[0057] FIG. 12 is a schematic diagram of a structure of another computing device cluster provided in the present application;

[0058] FIG. 13 is a schematic diagram of a structure of yet another computing device cluster provided in the present application. DETAILED DESCRIPTION

[0059] The terms "first", "second", etc. in the embodiments of the present application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features.

[0060] First, some technical terms involved in the embodiments of the present application are introduced.

[0061] Application programming interface (API) is a kind of computing interface, which defines the interaction between multiple software intermediaries, the types of calls or requests that can be made, how to make calls or requests, the data format to be used, the conventions to be followed, etc. It can also provide an extension mechanism so that users can extend the existing functions to different degrees in various ways.

[0062] API life cycle refers to a series of steps that a team must take to successfully design, develop, deploy and use an API, including API definition, design, development, testing, deployment, distribution, maintenance, etc. Among them, testing can include testing of different granularity, such as unit testing and integration testing.

[0063] Unit testing, also called module testing, is a correctness verification test work for program modules (the smallest unit or the smallest unit of software design). Among them, the program module can be an API. Integration testing, also known as assembly testing or joint testing, specifically assembles multiple program modules into a subsystem or system according to design requirements on the basis of unit testing, and performs integration testing. Integration testing is a logical extension of unit testing. An example of integration testing can be combining two tested units into a component to test the interface between the units.

[0064] For unit testing and integration testing of APIs, the industry provides related tools or components to assist in testing. FIG. 1 shows a schematic diagram of integration testing by arranging test cases through a graphical interface. A user can arrange test cases for different interfaces, such as POST interface 1, GET interface 1, GET interface 2, POST interface 2, PUT interface 1, and DEL interface 1, in a graphical interface. POST interface 1 is used for login or registration, GET interface 1 is used for querying a list of goods, GET interface 2 is used for querying specified goods information, POST interface is used for adding goods, PUT is used for updating goods information, and DEL is used for deleting goods. The user can add judgment / cycle logic between interface calls to achieve integration testing of multiple interfaces.

[0065] The actual recording method of the specific test process, case arrangement, and restrictions (or constraints) may differ depending on the test tool or component. Most test tools present the test process, case arrangement, restrictions, and the like on the web side in a graphical interface, and the logic is carried in JavaScript Object Notation (JSON) notation. Unit testing of interfaces is usually described using a script language such as JavaScript that finely debugs the interface, which has strong flexibility and expandability.

[0066] Single-interface testing and multi-interface integration testing include repeated test case writing, increasing the cost of testing. In addition, the programming languages and test script syntax of unit testing and integration testing are not unified, increasing the additional learning cost. Furthermore, the integration testing organized in the graphical interface is mostly simple field filling based on rules, and the fields that can be modified are relatively fixed, only suitable for some simple logic. The generated content of the auxiliary testing process has limited effectiveness and range, and the testing process still requires a large amount of manual development.

[0067] Therefore, the present application provides a test case generation method. The method can be applied to a test platform. The test platform is used to generate test cases for unit testing or integration testing of an application programming interface (API) according to a domain-specific language (DSL). The test platform can be a standalone software system, for example, a standalone test case generation software. The test platform can also be a software system integrated with other software, for example, a software system integrated with an API testing tool. The software system can take the form of a software module, a plug-in, a component, or an applet, but is not limited to these forms. The software system can be provided to a client in the form of a client-side software or a web-side software, and the client can deploy the software system on a private computing device. The software system can also be provided to a user in the form of a cloud service, and the user can subscribe to the cloud service to use the functions or capabilities of the software system. In some examples, the test platform can also be a hardware system, for example, a computing device cluster with test case generation capabilities, which executes the test case generation method of the present application.

[0068] Specifically, the test platform can obtain a structured rule for a first API and a natural language description for the first API, where the natural language description is used to describe the function or implementation code of the first API in natural language. Then the test platform queries a DSL database (for example, at least one of a DSL sample library or a DSL grammar library) according to the natural language description for the first API, and obtains a query result. The DSL database includes at least one DSL example, DSL error correction example, or DSL grammar, and the at least one DSL example, DSL error correction example, or DSL grammar includes a natural language description part. Accordingly, the query result includes at least one of a DSL example, DSL error correction example, or DSL grammar whose natural language description part matches the natural language description for the first API. Then the test platform can construct a prompt according to the query result, the natural language description, and the structured rule, input the prompt into a language model, and generate a target DSL code. The target DSL code is used to test the first API, or to generate a test case for integration testing of the first API and a second API.

[0069] The method supports dynamically matching the DSL examples, the DSL error correction examples or the DSL grammar matching the natural language description of the API from the DSL database, can generate a highly controllable and structured context based on the DSL examples, the DSL error correction examples or the DSL grammar and the structured rules for the API, assembles a prompt based on the context, and through prompt learning by a language model, can generate test cases related to the user's intention in a unified manner. Since the unit test and the integration test use the same programming language and the same flow, the integration test can reuse the test cases of the unit test, without the need of repeated development, thereby reducing the test cost. Moreover, the method does not need to additionally learn other programming languages, thereby reducing the development threshold of the test cases. In addition, the method uses the language model to generate the test cases, and can be applied to the test case generation of complex logic, and has high flexibility and availability.

[0070] In order to make the technical solutions of the present application clearer and easier to understand, the system architecture of the test platform of the present application will be introduced first in combination with the drawings.

[0071] Referring to a schematic diagram of the architecture of a test platform shown in FIG. 2, the test platform 20 includes a description and structured rule input module 202, a dynamic prompt word construction module 204 and a generation module 206. Further, the test platform 20 can further include an interpretation and correction module 208. The functions of each module will be introduced below.

[0072] The description and structuring input module 202 is configured to obtain a structured rule for the first API and a natural language description for the first API. The natural language description is configured to describe a function or an implementation code of the first API using natural language. The description and structuring input module 202 can obtain the structured rule for the first API from an original API design document and obtain the natural language description for the first API from a requirement design document. The structured rule refers to a rule expressed by structured data, which can be represented by a structured rule expression. For example, the structured rule can be represented as "maxLength = 32". Further, the description and structuring input module 202 is configured to provide an editing interface to a user to facilitate the user to input the structured rule or the natural language description through the editing interface. The editing interface can include a graphical editing interface or a script editing interface. The description and structuring input module 202 is configured to receive a custom structured rule or a natural language description input by the user through the graphical editing interface or the script editing interface. The custom structured rule or the natural language description can reflect the intention of the user. For example, the custom structured rule can include "var x = 1", and the natural language description can include "limit the value of variable x to 1-10000". It should be noted that the description and structuring input module 202 can receive the structured rule or the natural language description input by the user in the initialization stage, or receive feedback of the generated target DSL code from the user, which includes the structured rule or the natural language description.

[0073] The dynamic prompt word construction module 204 is configured to query a DSL database according to the natural language description for the first API, obtain a query result, and then construct a prompt according to the query result and the natural language description and the structured rule. The DSL database includes at least one DSL example, DSL error correction example or DSL syntax. The at least one DSL example, DSL error correction example or DSL syntax includes a natural language description part. Accordingly, the query result can include at least one of the DSL example, DSL error correction example or DSL syntax whose natural language description part matches the natural language description for the first API.

[0074] In a specific implementation, the dynamic prompt word construction module 204 can preprocess the natural language description to obtain an atomic interface description tuple of the first API. The preprocessing can include, but is not limited to, segmentation, combination, and cleaning. The segmentation can be segmenting the requirement design document according to the identifier hierarchy, the combination can be combining the segmentation result with the semantic segmentation result of the natural language description input by the user, and the cleaning can be processing invalid values or missing values. The atomic interface description tuple includes an atomic level API operation described using natural language. In this way, when querying the DSL database, the dynamic prompt word construction module 204 can query the DSL database according to the atomic interface description tuple of the first API to obtain a query result. The query result includes at least one of a DSL example, a DSL error correction example, or a DSL syntax that matches the natural language description part of the atomic interface description tuple.

[0075] The generation module 206 is configured to input the prompt into a language model to generate target DSL code. The target DSL code can also be referred to as structured DSL. The target DSL code is used to test the first API, or to generate a test case for integrated testing of the first API and the second API. The language model can be a large language model (LLM). The LLM can include a generative pre-trained transformer (GPT) or a bidirectional encoder representations from transformers (BERT). It should be noted that the present application only takes the LLM as an example, and in actual application, the language model can also be a small model, such as a small-scale language model.

[0076] The explanation and correction module 208 is configured to parse the target DSL code to obtain a syntax tree, correct the syntax tree based on a structured rule to obtain a corrected syntax tree, and then obtain a corrected target DSL code according to the corrected syntax tree. The explanation and correction module 208 can obtain the corrected target DSL code by interpreting the corrected syntax tree. The interpretation can be understood as the opposite of the parsing. The parsing can be a process of converting the DSL code into a syntax tree, and the interpretation can be a process of converting the syntax tree into the DSL code.

[0077] It should be noted that the interpretation and correction module 208 is specifically configured to generate a rule tree according to the structured rule, correct the first node in the syntax tree using the rule indicated by the second node corresponding to the first node in the rule tree, and obtain a corrected syntax tree. Wherein, the interpretation and correction module 208 can determine whether the content (such as node attribute value) of the first node is incorrect according to the rule indicated by the second node, and if so, replace the first node with incorrect content.

[0078] The interpretation and correction module 208 is also configured to present the corrected target DSL code to the user, and receive feedback of the user on the corrected target DSL code. The feedback can include a structured rule or a natural language description. For example, the graphical editing interface can include a feedback area, and the user can input the natural language description in the feedback area. Alternatively, the user can manually limit specific content in the graphical editing interface using a structured rule, such as limiting var x = 1. The interpretation and correction module 208 can also return the structured specification or natural language description of the feedback to the description and structured rule input module 202 for use in the next round of test case generation.

[0079] Based on the test platform 20 shown in FIG. 2, the present application provides a test case generation method. The test case generation method of the present application will be introduced below with reference to the accompanying drawings.

[0080] Referring to a flowchart of a test case generation method shown in FIG. 3, the method comprises:

[0081] S302, the test platform 20 obtains a structured rule for the first API and a natural language description for the first API.

[0082] The first API is an API to be generated for the test case. In specific implementation, the test platform 20 can present an API list to the user, the API list comprising at least one API, and the user can trigger a test case generation operation on the API in the API list. The test platform 20 determines the first API in response to the test case generation operation triggered by the user.

[0083] Specifically, the test platform 20 can obtain an API design document and extract a structured rule for the first API from the API design document. The test platform 20 can obtain a requirement design document and extract a natural language description for the first API from the requirement design document. The natural language description is used to describe the function or implementation code of the first API using natural language. It should be noted that in some documents, the function of the first API can also be represented as the purpose of the first API.

[0084] In some possible implementation manners, the test platform 20 can further present an editing interface, for example, a graphical editing interface or a script editing interface, to the user. The test platform 20 can receive the natural language description for the first API or the structured rule for the first API configured by the user through the editing interface.

[0085] In S304, the test platform 20 pre-processes the natural language description to obtain an atomic interface description tuple of the first API.

[0086] The pre-processing can also be referred to as pre-processing, which refers to pre-processing of the natural language description, and can generally include at least one of segmentation or division and combination. Further, the pre-processing can further include cleaning. The test platform 20 obtains atomic-level API operations described by the natural language description by segmenting or dividing the natural language description. The atomic-level API operation is an API operation that cannot be further divided (cannot be further subdivided). The test platform 20 can combine the atomic-level API operations to obtain the atomic interface description tuple of the first API. The atomic interface description tuple of the first API includes atomic-level API operations described using natural language. In some possible implementation manners, the test platform 20 can segment or divide the natural language description to further obtain a user intent corresponding to the atomic-level API operation. Accordingly, the atomic interface description tuple can further include the user intent corresponding to the atomic-level API operation. For example, the atomic interface description tuple can be represented as <interface description of atomic-level API operation, (user intent)>.

[0087] Further, the test platform 20 can further include cleaning the natural language description, for example, cleaning the segmented or divided corpus to improve data quality. For ease of understanding, the present application further illustrates an example. Referring to a flowchart of pre-processing of input corpus and constructing a prompt shown in FIG. 4, for the natural language description of the first API in the requirement design document, the test platform 20 can segment according to the identifier level to obtain atomic-level API operations. In some examples, the segmentation result can further include a user intent corresponding to the atomic-level API operation. Further, when the user additionally inputs the natural language description of the first API, the test platform 20 can perform semantic division on the natural language description. The test platform 10 can combine the segmented natural language description, the segmented natural language description (or collectively referred to as segmented corpus) to obtain the atomic interface description tuple.

[0088] The test platform 10 can determine the similarity of the segmented corpus, for example, based on a pre-trained similarity calculation model or other unsupervised similarity algorithm. Then, the test platform 10 can combine the segmented corpus based on the similarity of the segmented corpus, thereby obtaining the atomic interface description tuple. In this way, the test platform 10 can combine the segmented corpus of the same API as much as possible to obtain the atomic interface description tuple of the API.

[0089] In S306, the test platform 20 queries the DSL database according to the atomic interface description tuple of the first API to obtain a query result.

[0090] The query result includes at least one of a DSL sample, a DSL error correction example or a DSL syntax matched with the atomic interface description tuple. The DSL sample includes a DSL code sample. The DSL can be a use case code for generating a test case. Meanwhile, the DSL can be used as an interface of a language model such as LLM to construct a prompt, thereby generating a test case. In this way, the programming languages of unit testing and integration testing can be unified, thereby realizing test case reuse. For example, the test case of unit testing can be reused by integration testing, thereby reducing repeated development work.

[0091] In some possible implementation manners, the DSL can include JavaScript, a derivative language based on JavaScript or an extensible markup language (XML). For ease of description, the DSL is taken as a derivative language based on JavaScript for illustration in the present application.

[0092] In the present application, the DSL is designed as an executable and interpretable scripting language using a JavaScript description, retaining the debugging part of the JavaScript syntax used by unit testing (or debugging), and being extended based on the Behavior Driven Development (BDD) style. In this way, the syntax style can be kept simple and biased towards natural language while being structured, and can complete complex logic based on a single interface or overall function (such as multiple interfaces), and unify programming languages in different scenarios. The DSL can not only directly interact with service environment variables, Mock rule settings, request responses, and other service functions, but also serve as an LLM interface to adapt to the semantic consistency, context length control, and language style of the LLM. Specifically, the DSL can convert natural language descriptions (such as language chains input by users) into unique expressions in a specific order, and can use short functions to encapsulate complex loops or judgments, reducing model input length, which can reduce reasoning computation and cost. In addition, the DSL supports limiting executable functions with a whitelist, and can apply an automatic correction function based on an Abstract Syntax Tree (AST) to enhance robustness and be friendly to the generation model (a model that generates DSL code, such as a language model).

[0093] The DSL sample can be a test case for unit testing and integration testing of a sample API in historical software engineering or projects, and the test case includes sample DSL code for unit testing and integration testing of the sample API. The DSL correction example can be an example of correcting the DSL sample.

[0094] As shown in FIG. 4, when querying the DSL sample library and the DSL syntax library, the test platform 20 can first filter the atomic interface description tuple of the first API through a context filter. The context filter includes a word weight model trained with DSL detailed documents and a retrieval rule setting module. The context filter can filter out irrelevant content in the atomic interface description tuple from the DSL sample, the description (such as the function) of the DSL sample, and the like. In this way, the retrieval content can be similar to the pattern of the training data of the similarity reasoning model. The similarity reasoning model can be trained by a dataset composed of the DSL sample and the corresponding description (such as the DSL description), the DSL syntax, and the DSL correction example.

[0095] Then, the test platform 20 can dynamically match at least one of the DSL sample, the DSL error correction example or the DSL grammar matching the atomic interface description tuple in an incremental matching manner. The dynamic matching process can be achieved by transmitting the atomic interface description tuple into the dynamic prompt word construction module 204 through the content aggregation interface by the description and structured input module 202 of the test platform 20.

[0096] Specifically, the test platform 20 can obtain a first vector and at least one second vector, the first vector is obtained by vectorizing the atomic interface description tuple of the first API, and the second vector is obtained by vectorizing the natural language description in the at least one DSL sample or the at least one DSL error correction example in the DSL sample library, or is obtained by vectorizing the at least one DSL grammar in the DSL grammar library, and then the test platform 20 determines the similarity of the first vector and the at least one second vector, for example, the test platform 20 can determine the similarity of the first vector and the at least one second vector through a similarity reasoning model. Then, the test platform 20 can obtain at least one of the DSL sample, the DSL error correction example or the DSL grammar matching the atomic interface description tuple according to the similarity of the first vector and the at least one second vector. Among them, the DSL sample, the DSL error correction example or the DSL grammar matching the atomic interface description tuple can be the DSL sample, the DSL error correction example or the DSL grammar with the highest similarity, or the DSL sample, the DSL error correction example or the DSL grammar with a similarity greater than a threshold value.

[0097] The process of dynamically matching the DSL sample, the DSL error correction example or the DSL grammar matching the atomic interface description tuple will be described below in conjunction with the accompanying drawings. As shown in FIG. 5, the test platform 20 vectorizes the filtered atomic interface description tuple to obtain a first vector, calls a similarity reasoning model to determine the similarity of the first vector and the second vector obtained by vectorizing the natural language description part in each DSL sample, DSL error correction example and DSL grammar in the DSL database, and matches the DSL sample, the DSL error correction example or the DSL grammar matching the atomic interface description tuple according to the similarity. It should be noted that the atomic interface description tuple can include multiple atomic-level API operations, and the test platform 20 can perform the dynamic selection operation multiple times to determine the DSL sample, the DSL error correction example or the DSL grammar corresponding to each atomic-level API operation. After each dynamic matching, the test platform 20 can also reselect the similarity comparison center according to the proportion of the hit word weight in the filtered atomic interface description tuple, so that the DSL sample related to the description is more likely to be selected in the next dynamic matching.

[0098] For ease of understanding, an example is described below. As shown in FIG. 6, the atomic interface description tuple can include the description “…… should ensure that the value of the string type variable x in the request parameter is between 1 and 10000, if not, the connection should be made in the background, and the error response body should be responded. The response body format is …… the response body of the normal response interface should be ……”. The result of context filtering of the above atomic interface description tuple can be “… limit the request variable x type to String, and the value interval is between 1 and 10000”. The description after context filtering can be decomposed into two atomic-level API operations, which are: limiting the request variable x type to String and limiting the value of a String type variable to be between 1 and 10000. The test platform 20 can select a DSL example for each of the above two atomic-level API operations.

[0099] For the first atomic-level API operation, the test platform 20 uses a fixed segmentation method to obtain 7 tokens in the description, which are {limit, request, variable, x, type, is, String}. It is known through querying the keyword dictionary that 6 of the above 7 tokens are key tokens. Among them, the proportion of the hit words in the filtered context of the sentence is 6 / 12. Assuming that the matched document additionally contains 5 tokens at this time, the proportion of the hit words in the filtered document is 6 / 12. For the second atomic-level API operation, the test platform 20 can offset the interface description vector corresponding to the atomic-level API operation by a distance of 6 / 12, so that the DSL example related to the sentence description is more likely to be selected at the next dynamic matching.

[0100] The above S304 and S306 are a specific implementation of querying the DSL database according to the natural language description of the first API. In actual application, the DSL database can also be queried in other ways. For example, the test platform 20 can directly query the DSL database in a keyword search manner according to the natural language description of the first API. The present application does not limit this.

[0101] S308, the test platform 20 constructs a prompt according to the query result and the atomic interface description tuple and the structured rule.

[0102] Specifically, the test platform 20 can match each atomic interface description tuple with a DSL example, a DSL error correction example, and a DSL syntax filling prompt template to construct a prompt. The prompt template can include placeholders for the DSL example, the DSL error correction example, and the DSL syntax. The test platform 20 can replace the DSL example placeholder with the DSL example matched by the atomic interface description tuple, replace the DSL error correction example placeholder with the DSL error correction example matched by the atomic interface description tuple, and replace the DSL syntax placeholder with the DSL syntax matched by the atomic interface description tuple, thereby obtaining the prompt. In addition, the test platform 20 can assemble the atomic interface description tuple and the structured rule into the prompt template, thereby obtaining the prompt. The structured rule can include a structured rule for each round of feedback or a structured rule extracted from the original API design document. Further, the prompt can also include system content related to the introduction of the current task or system rules for adjusting the generated model (such as a language model). The system rules can be used to specify the request format and / or response format (input / output format) of the language model. The system rules can be in natural language or structured language. The combined prompt can be used to input the language model, and the language model can be prompted to generate the DSL code.

[0103] It should be noted that the above S308 is one specific implementation of constructing a prompt according to the query result and the natural language description and the structured rule. For example, when the test platform 20 directly queries the DSL database according to the natural language description, the test platform 20 can also directly construct a prompt according to the query result, the natural language description, and the structured rule.

[0104] S310, the test platform 20 inputs the prompt into the language model to generate the target DSL code.

[0105] The language model can be a large language model LLM, where the LLM can be a GPT or a BERT. In some examples, the language model can also be a small-scale language model. The language model has the ability to generate DSL code, and the test platform 20 can input the prompt into the language model for prompt learning to generate the target DSL code. The target DSL code is used to test the first API or to generate test cases for integrated testing of the first API and the second API. In other words, the target DSL code can not only be used as a description language for the testing process, but also be used as a debugging script for direct execution.

[0106] The test platform 20 can combine the user intent with different source data and other heterogeneous inputs to ensure the robustness of the generated content (such as the target DSL code) and lay a foundation for subsequent stable parsing into a graphical editing interface.

[0107] S312, the test platform 20 parses the target DSL code to obtain a syntax tree.

[0108] Specifically, the test platform 20 can parse the target DSL code to obtain a syntax tree by using a parser.

[0109] In S314, the test platform 20 generates a rule tree according to the structured rules.

[0110] Specifically, the test platform 20 can organize the structured rules in a tree structure to obtain the rule tree. The test platform 20 can parse the structured rules in a similar way as parsing the DSL code to obtain the rule tree.

[0111] In some possible implementation manners, the query results include DSL examples and DSL error correction examples. Accordingly, the test platform 20 can generate an initial rule tree according to the structured rules. In addition, the test platform 20 can generate extended rules according to the DSL examples and the DSL error correction examples. Then the test platform 20 can update the initial rule tree according to the extended rules to obtain the rule tree. In the rule tree, multiple rules indicated by a same node can be arranged in priority or processed according to a set logic. For example, the multiple rules indicated by the same node can be intersected.

[0112] For ease of understanding, an example is described below. As shown in FIG. 7, the target DSL code generated by the language model can be parsed into an abstract syntax tree AST, and the DSL examples and the DSL error correction examples queried and the structured rules input by the user can be combined. The DSL examples and the DSL error correction examples can be input in a unified format, for example, in a JSON format. In addition, the DSL examples, the DSL error correction examples, and the structured rules can be assigned priorities when input. The test platform 20 can generate extended rules according to the DSL examples and the DSL error correction examples, and generate a combined rule tree according to the extended rules and the structured rules input by the user. In the combined rule tree, nodes correspond to nodes in the AST of the target DSL code, and different rules of each node in the combined rule tree are intersected or arranged in priority.

[0113] In S316, for a first node in the syntax tree, the test platform 20 corrects the first node by using a rule indicated by a second node corresponding to the first node in the rule tree, to obtain a corrected syntax tree.

[0114] The test platform 20 can traverse the nodes in the syntax tree. For a first node in the syntax tree, the test platform 20 can determine whether the content (for example, a node attribute value) of the first node is incorrect by using a rule indicated by a second node corresponding to the first node in the rule tree. If so, the test platform 20 replaces the first node with incorrect content, to obtain a corrected syntax tree.

[0115] Still taking FIG. 7 as an example. The test platform 20 can match the AST parsed from the target DSL code with the corresponding nodes on the combination rule tree, and the test platform 20 can correct the matching nodes according to the corresponding structured rules to obtain a corrected syntax tree.

[0116] In S318, the test platform 20 obtains the corrected target DSL code according to the corrected syntax tree.

[0117] Specifically, the test platform 20 can perform reverse parsing on the corrected syntax tree to obtain the corrected target DSL code. Taking FIG. 7 as an example, the AST after correction is transmitted back in the form of DSL and parsed into a graphical editing interface output. At this time, the graphical editing interface is interpreted in DSL. In some examples, the graphical editing interface can be switched to a script editing interface to facilitate the user to view or edit through the script editing interface. The editing of the graphical editing interface and the natural language description of the user can be used as feedback for the regeneration of the test process, for example, the regeneration of the test case.

[0118] It should be noted that S312 to S318 are optional steps of the embodiments of the present application, and the test case generation method of the present application can also not perform the above S312 to S318. For example, when the target DSL code can meet the needs of the user, it can not be necessary to correct the above target DSL code.

[0119] Based on the above description, the present application provides a test case generation method. The method supports dynamically matching DSL examples, DSL correction examples or DSL syntaxes that match the atomic interface description tuples of the API from the DSL database, and can generate highly controllable and structured contexts based on the DSL examples, DSL correction examples or DSL syntaxes and the structured rules for the API. Based on the context assembly prompt, the prompt learning is performed through the language model, and the test case related to the user's intention can be generated in a unified manner. Since the unit test and the integration test use the same programming language and the same process, the integration test can reuse the test case of the unit test, without the need for repeated development, thereby reducing the test cost. Moreover, the method does not need to learn other programming languages, thereby reducing the development threshold of the test case.

[0120] Next, the test case generation method of the present application is introduced in combination with a specific application scenario.

[0121] In this scenario, the DSL can be an extended Chai.JS, as shown in FIG. 8, the DSL can be used as a test script to interface with various functional modules in the service, thereby realizing interaction with the service. Among them, the functional modules can include environment variables, advanced mocks, or request response modules. The DSL is also extensible, for example, the DSL supports custom JS objects and functions, and can bridge unit tests of single APIs and integration tests of multiple APIs. In this embodiment, the DSL is designed to be compatible with the generation model (such as LLM), in other words, the DSL is a language friendly to LLM. The DSL supports fixing different method orders and limiting available methods, so that a single function uses a single expression for LLM, ensuring semantic consistency. In addition, the DSL can provide methods such as expect() and repeat(), which can compress the code of long-span native JS judgment / circulation logic, on the one hand, reducing the amount of reasoning calculation, on the other hand, using DSL code as input to the language model, which can improve input stability. In addition, the DSL also has a BDD style, which retains the structured nature while using simple methods such as language chain getters (such as.not / .nested / .ok) to approach natural language expression.

[0122] As an interface with LLM, the DSL supports limiting or automatically correcting the DSL code to ensure robustness. Limiting the DSL code can be to trim the native JS, avoid unsafe usage in a whitelist manner, remove statements that cannot be parsed, reduce the complexity of the syntax analysis tree during parsing, and reduce the confusion during model generation. Automatically correcting the DSL code can be to automatically match rules for correction with the help of DSL correction examples.

[0123] Referring to FIG. 9, an application scenario diagram of a test case generation method is shown, in this scenario, test case generation can include the following stages: natural language description and structured rule input, dynamic prompt word construction, DSL code generation, and DSL code correction.

[0124] In the natural language description and structured rule input stage, the user can input the original API design document and the requirement design document. The original API design document is a YAML file containing the definition of API details, which includes structured rules that specify that the maximum length of variable y is 32. The requirement design document is a text file described in natural language. Further, the user can additionally input natural language description and custom structured rules. For example, the user can input structured rules in the service interface text box, in this example, the user can specify that the length of variable y is 1, and the user can also specify in the externally set text box that "the length of variable y is limited to 1-10000".

[0125] At the dynamic prompt construction stage, the test platform 20 can segment the requirement design document in a recursive segmentation manner according to the identifier hierarchy, split the user natural language description using Cross-segment BERT, match the corresponding interface description and user intent pair using the external model like BERT, and combine the atomic interface description tuples into the context filter of the content aggregation interface. After cleaning, the test platform 20 calls Sentence-BERT for each atomic interface description tuple in turn, infers the similarity of the atomic interface description tuple and the description part of the DSL sample in the DSL sample library or the similarity of the atomic interface description tuple and the description part of the DSL grammar in the DSL grammar library, and dynamically matches the DSL sample, DSL error correction example, and DSL grammar with the highest similarity. Specifically, the test platform 20 constantly searches for the DSL sample with the highest similarity according to the interface function description of the currently searched DSL sample after splicing the atomic interface description tuple content. The test platform 20 modifies the calculation center point in the sample space according to the word weight proportion of the interface description context in the atomic interface description tuple in the vector space during the search process, and reselects the retrieved DSL sample within a certain range. After the search is completed, the test platform 20 constructs the prompt of the preset system content, system rule, atomic interface description tuple, input single interface structured rule, and queried DSL grammar and DSL sample, DSL error correction example, and inputs the prompt into the language model.

[0126] It should be noted that the test platform 20 can also use other schemes when splitting the API design document and user intent, such as using SeqModel for splitting. In some examples, the test platform 20 can also ignore the user intent, and accordingly, the test platform 20 can only use the interface description when combining the atomic interface description tuple. The test platform 20 can also use other segmentation schemes when segmenting the requirement design document, such as only splitting according to the specified delimiter.

[0127] Other word weight models can also be used in the content aggregation interface. For example, the word weight model can include term frequency-inverse document frequency (TF-IDF). It should be noted that when using TF-IDF, the extraction of keywords in the document including the DSL sample is still required. This process aims to obtain keywords and filter the original atomic interface description tuple. At the same time, the test platform 20 can also use different models or algorithms to determine the similarity of the description of the DSL sample and the atomic interface description tuple. The preset system rule and system content in the prompt constructed by the test platform 20 can be modified as appropriate.

[0128] In the DSL code generation stage, the test platform 20 prompts as the input of the language model, generates the target DSL code in the DBB style, i.e., the result DSL, through language model inference. The result DSL is used for subsequent parsing.

[0129] In the DSL code correction stage, the test platform 20 can parse the generated result DSL in the method of Babel parsing AST, and combine the natural language description and all structured rules in the structured rule input module, as well as the DSL sample and DSL correction example obtained through query, in the form of JSON, to give priority to the user intention (the natural language description and the structured rule input by the user) higher than the original API design document and the requirement design document. Similarly, the test platform 20 organizes the JSON nodes in the tree structure obtained after parsing the AST by Babel, so that the tree structure is the same as the structure of the AST parsed from the result DSL. Subsequently, the test platform 20 can match the nodes of the two trees, and correct the matched nodes according to the structured rules. The test platform 20 can directly modify the AST nodes corresponding to the result DSL, which is reflected in the specific position of the result DSL. After the correction is completed, the test platform 20 can parse the result DSL into a graphical editing interface, and deliver the result DSL in the form of a script and the graphical editing interface to the user side. The test platform 20 can save the changes of the user in the graphical editing interface as structured rules, and save the natural language description input in the text box of the graphical editing interface, to update the atomic combination description tuple, and regenerate the test case based on the atomic combination description tuple.

[0130] The tool for parsing the AST can also select other schemes, for example, jcodeshift. The tool for parsing the AST and the tool for building the combination rule tree can use the same tool, so as to ensure that the tree structures are the same. The correction mode between the AST and the combination rule tree can be processed according to the priority or simple logic, or other complex rules can be externally hung on the corresponding nodes.

[0131] It should be noted that the above embodiments are illustrated by taking the Chai.JS example expanded by the DSL, and in actual application, other suitable DSLs or new languages constructed by the user can also be selected. The DSL can construct a sample library, a grammar library and a correction example for training a similarity inference model, and can be parsed into an AST.

[0132] In addition, the above embodiments are illustrated by taking the example of connecting the unit test and the integration test process, and in actual application, the present application can also connect other stages of the API life cycle, for example, the API development, debugging and test process, unify the debugging language, and make it possible to debug the data in the real process.

[0133] Further, the dynamic DSL example matching method split out in the present application can be applied when it is needed to match associated data which is difficult to compare directly according to user intention, such as extracting intermediate semantics and intention expressed in natural language in a processing process of non-natural language, and matching and identifying in an escaped sample library. For example, the dynamic DSL example matching method can convert the features of a certain image and the user's evaluation of the graph into intermediate results described in natural language using a multi-modal model, and dynamically match some DSL or non-DSL information.

[0134] Based on the foregoing test case generation method, the present application further provides a test platform 20. The test platform 20 is used to generate test cases for unit testing or integration testing of an application programming interface (API) according to a domain-specific language (DSL). As shown in FIG. 2, the test platform 20 includes:

[0135] A description and structured rule input module 202 is configured to obtain a structured rule for a first API and a natural language description for the first API, the natural language description being used to describe a function or implementation code of the first API using natural language;

[0136] A dynamic prompt word construction module 204 is configured to query a DSL database according to the natural language description for the first API, obtain a query result, the DSL database including at least one DSL example, DSL error correction example or DSL syntax, the at least one DSL example, DSL error correction example or DSL syntax including a natural language description part, the query result including at least one of a DSL example, DSL error correction example or DSL syntax matching the natural language description part and the natural language description for the first API; and construct a prompt according to the query result and the natural language description and the structured rule.

[0137] A generation module 206 is configured to input the prompt into a language model to generate a target DSL code, the target DSL code being used to test the first API or to generate a test case for integration testing of the first API and the second API.

[0138] Exemplarily, the description and structured rule input module 202, the dynamic prompt word construction module 204 and the generation module 206 described above can be implemented by hardware or can be implemented by software.

[0139] When implemented by software, the description with the structured rule input module 202, the dynamic prompt word construction module 204, and the generation module 206 can be an application program running on a computing device, such as a computing engine, etc. The above-mentioned application program can also be virtualized to provide a user with a virtualized service. The virtualized service can include a virtual machine (VM) service, a bare metal server (BMS) service, or a container service. Among them, the VM service can be a service that virtually creates a VM resource pool on multiple physical hosts through a virtualization technology to provide a user with a VM for use on demand. The BMS service is a service that virtually creates a BMS resource pool on multiple physical hosts to provide a user with a BMS for use on demand. The container service is a service that virtually creates a container resource pool on multiple physical hosts to provide a user with a container for use on demand. The VM is a simulated virtual computer, that is, a logical computer. The BMS is a kind of elastic scalable high-performance computing service, and the computing performance is no different from that of a traditional physical machine, and has the characteristics of safe physical isolation. The container is a kind of kernel virtualization technology, which can provide light virtualization to achieve the purpose of isolating user space, processes and resources. It should be understood that the VM service, the BMS service and the container service in the above-mentioned virtualized service are only specific examples, and in actual application, the virtualized service can also be other light or heavy virtualized services, which are not limited here.

[0140] When implemented by hardware, the description with the structured rule input module 202, the dynamic prompt word construction module 204, and the generation module 206 can include at least one computing device, such as a server, etc. Alternatively, the description with the structured rule input module 202, the dynamic prompt word construction module 204, and the generation module 206 can also be a device implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), etc. Among them, the above-mentioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0141] In some possible implementation manners, the dynamic prompt word construction module 204 is specifically configured to:

[0142] The natural language description is preprocessed to obtain an atomic interface description tuple of the first API, the atomic interface description tuple including an atomic level API operation described using natural language.

[0143] The atomic interface description tuple of the first API is used to query a DSL database.

[0144] In some possible implementation manners, the test platform 20 further includes:

[0145] An interpretation and correction module 208 is configured to parse the target DSL code to obtain a syntax tree, and generate a rule tree according to the structured rule.

[0146] For a first node in the syntax tree, the first node is corrected using a rule indicated by a second node corresponding to the first node in the rule tree, to obtain a corrected syntax tree, and the target DSL code is corrected according to the corrected syntax tree.

[0147] In some possible implementation manners, the query result includes the DSL example and the DSL error correction example.

[0148] The interpretation and correction module 208 is specifically configured to:

[0149] An initial rule tree is generated according to the structured rule.

[0150] An extended rule is generated according to the DSL example and the DSL error correction example.

[0151] The initial rule tree is updated according to the extended rule to obtain the rule tree, and a plurality of rules indicated by a same node in the rule tree are arranged according to a priority or processed according to a set logic.

[0152] In some possible implementation manners, the interpretation and correction module 208 is further configured to:

[0153] The target DSL code is displayed through a graphical editing interface; or

[0154] The target DSL code is displayed through a script editing interface.

[0155] In some possible implementation manners, the interpretation and correction module 208 is further configured to:

[0156] Feedback of the user on the target DSL code through the graphical editing interface or the script editing interface is received, and the feedback includes a structured rule or a natural language description.

[0157] In some possible implementations, the DSL includes JavaScript, a JavaScript-based derivative language, or Extensible Markup Language (XML).

[0158] The present application also provides a computing device 1000. As shown in FIG. 10, the computing device 1000 includes a bus 1002, a processor 1004, a memory 1006, and a communication interface 1008. The processor 1004, the memory 1006, and the communication interface 1008 communicate via the bus 1002. The computing device 1000 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 1000.

[0159] The bus 1002 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one line is shown in FIG. 10, but it does not mean that there is only one bus or only one type of bus. The bus 1002 can include a path for transmitting information between various components (e.g., the memory 1006, the processor 1004, the communication interface 1008) of the computing device 1000.

[0160] The processor 1004 can include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), etc.

[0161] The memory 1006 can include volatile memory, such as random access memory (RAM) comprising a number of modules. The memory 1006 also can include non-volatile memory, such as read-only memory (ROM), optical or flash memory, hard disk drives (HDDs), or solid-state drives (SSDs). An executable of the aforementioned test case generation method is stored in the memory 1006, which the processor 1004 executes. Specifically, the memory 1006 stores instructions for the test platform 20 to execute the test case generation method. For example, the memory 1006 stores instructions for executing the functions of the structured rule input module 202, the dynamic prompt word construction module 204, and the generation module 206. Further, the memory 1006 can also store instructions for executing the functions of the interpretation and correction module 208.

[0162] The communication interface 1008 enables communication among the computing device 1000 and other devices or communication networks using, for example and without limitation, a transceiver, a network interface card, a modem, or other type of interface.

[0163] The embodiments of the present disclosure also provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device, such as a desktop computer, a notebook computer, or a smart phone.

[0164] As shown in FIG. 11, the computing device cluster includes at least one computing device 1000. The memory 1006 in one or more computing devices 1000 in the computing device cluster can store the same instructions of the test platform 20 to execute the test case generation method.

[0165] In some possible implementations, one or more computing devices 1000 in the computing device cluster can also be used to execute part of the instructions of the test platform 20 to execute the test case generation method. In other words, the combination of one or more computing devices 1000 can collectively execute the instructions of the test platform 20 to execute the test case generation method.

[0166] It should be noted that the memory 1006 in different computing devices 1000 in the computing device cluster can store different instructions to execute part of the functions of the test platform 20.

[0167] FIG. 12 illustrates one possible implementation. As shown in FIG. 12, two computing devices 1000A and 1000B are connected through communication interfaces 1008. The memory in computing device 1000A has instructions stored thereon for performing the functions of structured rule input module 202 and dynamic prompt construction module 204. The memory in computing device 1000B has instructions stored thereon for performing the functions of generation module 206. Optionally, the memory in computing device 1000B also has instructions stored thereon for performing the functions of interpretation and correction module 208. In other words, the memories 1006 of computing devices 1000A and 1000B collectively store instructions for test platform 20 to perform the test case generation method.

[0168] The manner in which the computing device cluster shown in FIG. 12 is connected can be such that the test case generation method provided in the present application requires more computing resources for prompt construction and test case generation. Therefore, it is contemplated that the functions of structured rule input module 202 and dynamic prompt construction module 204 are performed by computing device 1000A, and the functions performed by generation module 206 are performed by computing device 1000B. When test platform 20 includes interpretation and correction module 208, the functions performed by interpretation and correction module 208 can also be performed by computing device 1000B.

[0169] It should be understood that the functions of computing device 1000A shown in FIG. 12 can also be performed by multiple computing devices 1000. Similarly, the functions of computing device 1000B can also be performed by multiple computing devices 1000.

[0170] In some possible implementations, one or more computing devices in the computing device cluster can be connected through a network. The network can be a wide area network, a local area network, or the like. FIG. 13 illustrates one possible implementation. As shown in FIG. 13, two computing devices 1000C and 1000D are connected through a network. Specifically, the computing devices are connected to the network through communication interfaces in the respective computing devices. In this type of possible implementation, the memory 1006 in computing device 1000C has instructions stored thereon for performing the functions of structured rule input module 202 and dynamic prompt construction module 204. Meanwhile, the memory 1006 in computing device 1000D has instructions stored thereon for performing the functions of generation module 206. Optionally, the memory 1006 in computing device 1000D also has instructions stored thereon for performing the functions of interpretation and correction module 208.

[0171] The connection between the computing device clusters shown in FIG. 13 can be prompted in view of the large amount of computing resources required for the test case generation method provided in the present application to construct the prompt word and generate the test case, and therefore the functions implemented by the dynamic prompt word construction module 204 and the generation module 206 are considered to be executed by different computing devices.

[0172] It should be understood that the functions of the computing device 1000C shown in FIG. 13 can also be completed by multiple computing devices 1000. Similarly, the functions of the computing device 1000D can also be completed by multiple computing devices 1000.

[0173] The embodiments of the present application also provide a computer readable storage medium. The computer readable storage medium can be any available medium or data storage device that can be accessed by a computing device, such as a data center containing one or more available media. The available media can be a magnetic medium, such as a floppy disk, a hard disk, a magnetic tape, an optical medium, such as a DVD, or a semiconductor medium, such as a solid state hard drive, etc. The computer readable storage medium includes instructions that instruct the computing device to execute the test case generation method described above for the test platform 20.

[0174] The embodiments of the present application also provide a computer program product containing instructions. The computer program product can be a software or program product containing instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to execute the test case generation method described above.

[0175] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A test case generation method characterized by, The method is applied to a test platform for generating test cases for unit testing or integration testing of an application programming interface (API) according to a domain-specific language (DSL), and the method comprises: obtaining a structured rule for a first API and a natural language description for the first API, the natural language description being used to describe a function or implementation code of the first API using a natural language; querying a DSL database according to the natural language description for the first API to obtain a query result, the DSL database comprising at least one DSL example, DSL error correction example or DSL syntax, the at least one DSL example, DSL error correction example or DSL syntax comprising a natural language description part, and the query result comprising at least one of a DSL example, DSL error correction example or DSL syntax whose natural language description part matches the natural language description for the first API; constructing a prompt according to the query result, the natural language description and the structured rule; inputting the prompt into a language model to generate target DSL code, the target DSL code being used to test the first API or to generate test cases for integration testing of the first API and a second API.

2. The method of claim 1, wherein, The querying of the DSL data according to the natural language description for the first API comprises: preprocessing the natural language description to obtain an atomic interface description tuple of the first API, the atomic interface description tuple comprising an atomic-level API operation described using a natural language; querying the DSL database according to the atomic interface description tuple of the first API.

3. The method according to claim 1 or 2, characterized in that, The method further comprises: parsing the target DSL code to obtain a syntax tree; generating a rule tree according to the structured rule; for a first node in the syntax tree, correcting the first node using a rule indicated by a second node corresponding to the first node in the rule tree to obtain a corrected syntax tree; obtaining the target DSL code after correction according to the corrected syntax tree.

4. The method of claim 3, wherein, The query result comprises the DSL example and the DSL error correction example. The generation of the rule tree according to the structured rule comprises: generating an initial rule tree according to the structured rule; generating an extended rule according to the DSL example and the DSL error correction example; updating the initial rule tree according to the extended rule to obtain the rule tree, in which multiple rules indicated by a same node are arranged in priority or processed according to a set logic.

5. The method according to any one of claims 1 to 4, characterized in that, The method further comprises: displaying the target DSL code through a graphical editing interface; or displaying the target DSL code through a script editing interface.

6. The method of claim 5, wherein, The method further comprises: receiving feedback of the target DSL code by the user through the graphical editing interface or the script editing interface, the feedback comprising a structured rule or a natural language description.

7. The method according to any one of claims 1 to 6, characterized in that, The DSL comprises JavaScript, a JavaScript-based derivative language or an extensible markup language (XML).

8. A test platform characterized by, The test platform is used for generating test cases for unit testing or integration testing of an application programming interface (API) according to a domain-specific language (DSL), and the test platform comprises: a description and structure rule input module configured to obtain a structured rule for a first API and a natural language description for the first API, the natural language description being used to describe a function or implementation code of the first API using a natural language; a dynamic prompt word construction module configured to query a DSL database according to the natural language description for the first API, to obtain a query result, the DSL database comprising at least one DSL example, DSL error correction example or DSL syntax, the at least one DSL example, DSL error correction example or DSL syntax comprising a natural language description part, the query result comprising at least one of a DSL example, DSL error correction example or DSL syntax that matches the natural language description part and the natural language description for the first API, and to construct a prompt according to the query result and the natural language description and the structured rule; a generation module configured to input the prompt into a language model to generate target DSL code, the target DSL code being used to test the first API or to generate test cases for integration testing of the first API and a second API.

9. The test platform of claim 8, wherein, The dynamic prompt word construction module is specifically configured to: preprocess the natural language description to obtain an atomic interface description tuple of the first API, the atomic interface description tuple comprising an atomic level API operation described using a natural language; query the DSL database according to the atomic interface description tuple of the first API.

10. The test platform of claim 8 or 9, wherein, The test platform further comprises: an interpretation and correction module configured to parse the target DSL code to obtain a syntax tree, and to generate a rule tree according to the structured rule; for a first node in the syntax tree, to correct the first node using a rule indicated by a second node corresponding to the first node in the rule tree to obtain a corrected syntax tree, and to obtain a corrected target DSL code according to the corrected syntax tree.

11. The test platform of claim 10, wherein, The query result comprises the DSL example and the DSL error correction example. The interpretation and correction module is specifically configured to: generate an initial rule tree according to the structured rule; generate an extended rule according to the DSL example and the DSL error correction example; update the initial rule tree according to the extended rule to obtain the rule tree, a plurality of rules indicated by a same node in the rule tree being arranged in priority or being processed according to a set logic.

12. The test platform of any one of claims 8 to 11, wherein, The interpretation and correction module is further configured to: display the target DSL code through a graphical editing interface; or display the target DSL code through a script editing interface.

13. The test platform of claim 12, wherein, The interpretation and correction module is further configured to: receive feedback of the target DSL code of the user through the graphical editing interface or the script editing interface, the feedback comprising a structured rule or a natural language description.

14. The test platform of any one of claims 8 to 13, wherein, The DSL includes JavaScript, a JavaScript-based derivative language, or Extensible Markup Language (XML).

15. A cluster of computing devices, characterized in that, The computing device cluster includes at least one computing device, the at least one computing device including at least one processor and at least one memory having stored therein computer readable instructions; the at least one processor executes the computer readable instructions to cause the computing device cluster to perform the test case generation method of any one of claims 1-7.

16. A computer-readable storage medium, characterized in that, comprising computer readable instructions; the computer readable instructions are used to implement the test case generation method of any one of claims 1-7.

17. A computer program product, characterised in that, comprising computer readable instructions; the computer readable instructions are used to implement the test case generation method of any one of claims 1-7.

Images