Application Programming Interface (API) Testing Method and Device

Abstract

An application program interface (API) testing method and device. In the application, the API testing device first obtains the API file transmitted by the user, parses the API file, generates an API operation relationship diagram and a test prophet; the API testing device generates a first test case set according to the API operation relationship diagram, triggers the service node to execute the test cases in the first test case set, and obtains the API request and corresponding API response initiated in the process of executing the test cases in the first test case set. The user is provided with the matching status of the API request and API response initiated in the process of executing the test cases in the first test case set and the test prophet. The API testing device can complete the entire process from test case generation, triggering the execution of test cases, and providing test results by itself without human intervention, and can efficiently complete the API testing process.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to an application programming interface (API) testing method and device. Background Art

[0002] As software architecture evolves towards cloud and service, application programming interface (API) has gradually become the main functional entry point for software. With the widespread application of API, the performance of API will affect the operation of software. In order to discover API defects in a timely manner, API testing is required.

[0003] API testing is also called interface testing. Its test object is API, and its purpose is to verify whether the API meets the requirements of functionality, performance, reliability, security, etc.

[0004] Current API testing methods are ineffective and have limited automation. Summary of the invention

[0005] The present application provides an application programming interface (API) testing method and device to improve API testing efficiency and realize automated API testing.

[0006] In the first aspect, an embodiment of the present application provides an application program interface API testing method, which can be executed by an API testing device. In the method, the API testing device can first obtain an API file transmitted by a user. For example, the user can directly operate the API testing device to upload the API file to the API testing device. For another example, the user can send the API file to the API testing device through a client deployed on the user side. The API file can be an API definition document, or it can be API traffic data, or it can include both an API definition document and API traffic data. Among them, the API definition document includes description information of the API, and the API traffic data includes the API request when calling the API and the API response corresponding to the API request.

[0007] After obtaining the API file, the API testing device can parse the API file and generate an API operation relationship diagram and a test prophet. The API operation relationship diagram is used to describe the dependency relationship of the API operation, and the test prophet is the prediction rule of the API request and the corresponding API response when calling the API; then, the API testing device can generate a first test case set according to the API operation relationship diagram, and the first test case set includes at least one test case. Then, the API testing device can send the test case in the first test case set to the service node, instruct the service node to execute the test case, and obtain the API request and the corresponding API response initiated by the service node in the process of executing the test case in the first test case set. After obtaining the API requests and corresponding API responses initiated by the service node in the process of executing the test cases in the first test case set, the API testing device can analyze the API requests and API responses initiated by the service node in the process of executing the test cases in the first test case set, determine the matching status of the API requests and API responses initiated by the service node in the process of executing the test cases in the first test case set with the test prophet, and can also provide a test report to the user, the test report including the matching status of the API requests and API responses initiated in the process of executing the test cases in the first test case set with the test prophet. Optionally, the test report also includes the first test case set.

[0008] Through the above method, the entire process from test case generation, triggering test case execution, and displaying test reports is completed by the API testing device itself, without human intervention, and the API testing process can be completed efficiently. By providing test reports to users, users can understand the API test status in a timely and convenient manner, effectively improving user experience.

[0009] In a possible implementation, the API testing device may perform multiple rounds of iterations, each round of iteration using the output of the previous round of iteration as input, and a subsequent round of iteration is used as an example for description below.

[0010] The API testing device can determine the target test case that does not match the test prophet from the first test case set according to the test prophet, and the API request and API response initiated by the target test case when it is executed do not match the test prophet; then, analyze the target test case, dig out the dependency relationship between new API operations, and update it to the API operation relationship diagram. Then, generate the second test case set according to the updated API operation relationship diagram; trigger the service node to execute the test cases in the second test case set, and obtain the API request and API response initiated by the service node in the process of executing the test cases in the second test case set.

[0011] Provide the user with the matching status of the API requests and corresponding API responses initiated during the execution of the test cases in the second test case set and the test prophet. That is, the test report provided to the user may also include the matching status of the API requests and API responses initiated during the execution of the test cases in the second test case set and the test prophet, and may also include the second test case set.

[0012] From the above process, it can be seen that the test case sets (such as the first test case set and the second test case set) generated in each round of iteration can determine the test cases that do not match the test prophet, and the operation relationship diagram can be updated according to such test cases, and the updated operation relationship diagram can be used for the next round of iteration. This cycle is repeated until the preset number of iterations or iteration time is reached.

[0013] Through the above method, through multiple rounds of iterations, it is possible to dig out relatively rich dependencies between API operations, and then generate corresponding test cases to test the API, making the testing process more accurate and ensuring the accuracy of API testing.

[0014] In one possible implementation, when the API file includes API traffic data, the API testing device can generate an API operation relationship diagram and a parameter constraint dictionary when parsing the API file. The parameter constraint dictionary includes the value range of the API parameters, that is, the parameter constraint dictionary records the possible values ​​of the API input parameters and output parameters, such as the parameters in the API request.

[0015] Through the above method, the API testing device can mine the possible values ​​of API parameters in the actual API calling process from the API traffic data, which is convenient for the subsequent generation of test cases and ensures the effectiveness of the test cases.

[0016] In one possible implementation, the parameters of the API request defined by the test case in the first test case set conform to the parameter constraint dictionary. The test case records the parameter data instance of the API request, which records the specific value of the parameter in the API request, and the specific value may be a value recorded in the parameter constraint dictionary.

[0017] Through the above method, the parameter constraint dictionary is used to constrain the parameters of the API request defined in the test case, which can ensure the authenticity and validity of the API request and can try to ensure that the test case will not fail during execution due to incorrect parameter values.

[0018] In a possible implementation, in addition to transmitting the API file, the user may also send configuration information to the API testing device. The configuration information is used to indicate the rules that the test case must follow. The configuration information includes part or all of the following information:

[0019] Test scenarios of test cases, API blacklist or whitelist, API parameter blacklist or whitelist, and whether to provide test case coverage information.

[0020] Through the above method, users can configure the rules that test cases need to follow by themselves, ensuring that the generated test cases are more in line with user needs and the test direction can be close to the test direction expected by users. In this way, the test report finally generated can also better meet user needs and ensure user experience.

[0021] In a possible implementation manner, the test cases in the first test case set conform to the configuration information.

[0022] In a possible implementation, the API testing device may also determine coverage information of the test cases in the first test case set, where the coverage information of the test cases describes the coverage of the API by the test cases, and provide the user with the coverage information of the test cases. The test report may also include the coverage information of the test cases in the first test case set.

[0023] Through the above method, the coverage information of the test case can reflect the performance of the test case to a certain extent, which is convenient for understanding the performance of the test case, so that users can select test cases from it to produce test case scripts for reuse.

[0024] In a possible implementation manner, the API testing device may also perform another iteration method, in which the API testing device may also perform multiple rounds of iterations, each round of iteration using the output of the previous round of iteration as input, and a subsequent round of iteration is used as an example for explanation below.

[0025] The API testing device can determine a candidate test case set from the first test case set, wherein the coverage information of the test cases in the candidate test case set meets the preset coverage condition; such as selecting test cases whose coverage information meets the required range as test cases in the candidate test case set. Afterwards, based on the candidate test case set, a derived test case set is generated, such as by using a genetic algorithm (such as crossover mutation) to generate a derived test case set, and then, the service node is triggered to execute the test cases in the derived test case set, and the API request and API response initiated by the service node in the process of executing the test cases in the derived test case set are obtained.

[0026] The API testing device can determine the matching status of the API requests and corresponding API responses initiated during the execution of the test cases in the derived test case set and the test prophet, and provide the user with the matching status of the API requests and corresponding API responses initiated during the execution of the test cases in the derived test case set and the test prophet. That is, the test report also includes the matching status of the API requests and API responses initiated during the execution of the test cases in the derived test case set and the test prophet, and may also include the derived test case set.

[0027] That is to say, in each round of iteration, a derived test case set can be generated according to the test case set generated in each round of iteration (such as the first test case set and the second test case set), and then the test cases in the derived test case set can be executed.

[0028] Through the above method, new test cases can be generated based on the test cases in the existing test case set, which can explore the possible dependencies in the API and ensure that the API testing is more comprehensive.

[0029] In the second aspect, the embodiment of the present application further provides an API testing device, which has the function of implementing the behavior in the method example of the first aspect above. The beneficial effects can be found in the description of the first aspect and will not be repeated here. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In one possible design, the structure of the device includes an acquisition module, a parsing module, a generation module, a collection module, and a provision module. These modules can perform the corresponding functions in the method example of the first aspect above. Please refer to the detailed description in the method example for details, which will not be repeated here.

[0030] In a third aspect, the present application further provides a computing device, the computing device comprising a processor and a memory, and may also comprise a communication interface, the processor executing program instructions in the memory to execute the method provided by the first aspect or any possible implementation of the first aspect. The memory is coupled to the processor, and stores the necessary program instructions and data in the process of testing the application program interface. The communication interface is used to communicate with other devices, such as obtaining API files, sending test reports, sending test cases, etc.

[0031] In a fourth aspect, the present application provides a computing device system, which includes at least one computing device. Each computing device includes a memory and a processor. The processor of at least one computing device is used to access the code in the memory to execute the method provided by the first aspect or any possible implementation of the first aspect.

[0032] In a fifth aspect, the present application provides a non-transitory readable storage medium, and when the non-transitory readable storage medium is executed by a computing device, the computing device executes the method provided in the aforementioned first aspect or any possible implementation of the first aspect. The storage medium stores a program. The storage medium includes but is not limited to a volatile memory, such as a random access memory, a non-volatile memory, such as a flash memory, a hard disk drive (HDD), and a solid state drive (SSD).

[0033] In a sixth aspect, the present application provides a computing device program product, the computing device program product comprising computer instructions, when executed by a computing device, the computing device performs the method provided in the aforementioned first aspect or any possible implementation of the first aspect. The computer program product may be a software installation package, and when it is necessary to use the method provided in the aforementioned first aspect or any possible implementation of the first aspect, the computer program product may be downloaded and executed on a computing device. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1A A schematic diagram of the architecture of a system provided for this application;

[0035] Figure 1B A schematic diagram of the architecture of a system provided for this application;

[0036] Figure 2 A schematic diagram of a scenario provided for this application;

[0037] Figure 3 A schematic diagram of an API testing method provided for this application;

[0038] Figure 4A A schematic diagram of a method for determining dependencies between APIs provided in this application;

[0039] Figure 4B A schematic diagram of ODG provided for this application;

[0040] Figure 4C A schematic diagram of a method for generating a test prophet provided in this application;

[0041] Figure 4D A schematic diagram of a test case provided for this application;

[0042] Figure 4E A schematic diagram of a method for generating test cases provided in this application;

[0043] Figure 4FA schematic diagram of a method for updating ODG provided in this application;

[0044] Figure 4G A schematic diagram of a method for discovering new API dependencies provided by this application;

[0045] Figure 4H A schematic diagram of the structure of a code coverage feedback unit provided in this application;

[0046] Figure 5 A schematic diagram of an API testing method provided for this application;

[0047] Fig. 6A A schematic diagram of ODG provided for this application;

[0048] Figure 6B A schematic diagram of a test case provided for this application;

[0049] Figure 7 A schematic diagram of an API testing method provided for this application;

[0050] Figure 8 A schematic diagram of the structure of an API testing device provided for this application;

[0051] Figure 9-10 A schematic diagram of the structure of a computing device provided in this application. DETAILED DESCRIPTION

[0052] Before describing an application program interface testing method provided in an embodiment of the present application, the concepts involved in the embodiment of the present application are described first:

[0053] (1) API definition document

[0054] The API definition document includes the description information of the API, such as the resource identifier called by the API (such as the identifier of the element on the web page), the parameter definition of the API (such as the input parameter structure, the output parameter structure), the operation method of the resource in the API, and the parameter constraints (such as the data value range or value of the API parameter). One API definition document can describe one or more APIs.

[0055] Taking the RESTful API definition document as an example, this type of API definition document records the API method name, resource address, API request and API response message body (that is, the message format of API request and API response), and the status code carried in the API response. Among them, the resource address can be the uniform resource locator (URL) of the resource.

[0056] (2) Test cases

[0057] In the embodiment of the present application, the test case is a test case for testing the API, which describes the sequence of steps for executing the API operation and the parameter data instance of the API request initiated in each step, and the parameter data instance indicates the specific values ​​of each parameter in the API request.

[0058] (3) API traffic data

[0059] API traffic data records the API requests and corresponding API responses sent when nodes in a production environment or a quasi-production environment call APIs for interaction.

[0060] Among them, the API request is a request message sent by the sender (such as the client) to the receiver (such as the server), including the request address, request method, request message header, request parameters, request message body and other contents.

[0061] The API response is the feedback message returned by the receiver to the sender (client), including the response status code, response header, response body, etc.

[0062] (4) API operation

[0063] API operations refer to the operation methods of resources by calling APIs. API operations are divided into add, read, modify, and delete, and follow the Create, Read, Update, and Delete (CRUD) semantics, which describes the rules of creating, obtaining, modifying, and deleting resources. API operations are not limited to create, obtain, modify, and delete. An API can be understood as an API operation.

[0064] Taking the Restful API as an example, the operation for obtaining resources is GET. The operation for creating resources is to send PUT to a new resource (such as a resource URL), or to send POST to an existing resource (such as a resource URL).

[0065] The resource replacement operation can be sending a PUT to an existing resource. The resource modification operation (such as modifying some attributes of a resource) can be sending a PATCH to an existing resource. The resource deletion operation can be sending a DELETE to an existing resource.

[0066] (5) API operation relationship diagram

[0067] The API operation relationship graph is used to describe the dependency relationship between API operations. The embodiment of the present application does not limit the specific type of the API operation relationship graph, for example, it can be an operation dependency graph (ODG) or an operation transfer probability graph (OTPG).

[0068] ODG describes the dependencies between different API operations. OTPG can also describe the dependencies between operations in an API. The difference from ODG is that OTPG also records the transition probability between API operations, that is, the probability of executing another operation after one API operation is executed.

[0069] The dependency relationship between different API operations can be expressed as the dependency relationship when different API operations are executed on the same resource, and can also be expressed as the association between parameters of different API operations (such as input parameters or output parameters of the API).

[0070] For example, taking a RESTful API as an example, the dependencies between API operations can follow the static data in the API definition file and the resource CRUD semantic rules, such as the get resource operation (GET) depends on the create resource operation (PUT). It is also possible for the input parameter value of API A's resource operation to depend on the response parameter value of API B's resource operation, such as the input parameter petId in the interface operation GetPetById depends on the response parameter petId of the interface operation getPets.

[0071] The operation transfer probability graph is derived from API traffic data. The API traffic data can be cut off according to certain rules (such as users, days), and then the API operation transfer probabilities with parameter dependencies in all previous and subsequent API operations in the API traffic data are counted. For example, the parameters of Restful interfaces B, C, and D all depend on interface A, and A→B appears 700 times, A→C appears 200 times, and A→D appears 100 times. Then the transfer probability of A→B is 70%, the transfer probability of A→C is 20%, and the transfer probability of A→D is 10%.

[0072] (6) Testing the Prophet

[0073] The Test Prophet describes the predicted rules of the API request when calling the API and the API response corresponding to the API request. In the embodiment of the present application, the Test Prophet includes some or all of the following: the rules that the pre-predicted API request and the API response corresponding to the API request must follow, the distribution of the pre-predicted API input parameters and output parameters, and parameter constraints.

[0074] Based on the API definition document, we can determine the rules that the API requests and the corresponding API responses must follow (which can be called explicit test prophets).

[0075] Based on the API definition document, explicit test prophets can be derived. Taking the RestFul-style API as an example, under the premise of complying with the standard definition, determining whether the response status code (2XX, 4XX, 5XX) meets expectations is an explicit test prophet. For example, under normal input parameter conditions, the expected result of the status code should be 2XX; under abnormal input parameter conditions, the expected result of the status code should be 4XX, and if 5XX is returned, it is judged to be invalid. In addition, the input and output parameter structures (schemas) clearly defined in the API definition document, including the interface parameter name, parameter type, value range, enumeration value range, whether it is required, etc., can also be used as explicit test prophets.

[0076] Based on API traffic data, we can determine implicit test prophets such as the normal distribution of API input parameters and output parameters and constraints between parameters.

[0077] The distribution of API input and output parameters may include some or all of the following:

[0078] The range of the API output parameter can be set by the maximum and minimum values ​​of the output parameter.

[0079] The probability distribution of the API output parameters, that is, the probability of the output parameters taking different values.

[0080] The mapping relationship between API input parameters and output parameters. Taking input parameter A and output parameter B as an example, the mapping relationship between input parameter A and output parameter B can indicate the size relationship between input parameter A and output parameter B, such as input parameter A>output parameter B, input parameter A=output parameter B, input parameter A<output parameter B, and can also indicate that when input parameter A is equal to a first value or belongs to a first range, output parameter B is equal to a second value or belongs to a second range.

[0081] The association relationship between API input parameters and output parameters, taking input parameter A and output parameter B as an example, the association relationship between input parameter A and output parameter B can be that input parameter A and output parameter B appear at the same time, any one of them appears, or they cannot appear at the same time.

[0082] The method for generating application program interface test cases provided in the embodiment of the present application can be executed by an API test device. The API test device can be a hardware device, such as a server, a terminal computing device, etc., or a software device, specifically a software system running on a hardware computing device. The embodiment of the present application does not limit the location where the API test device is deployed. For example, Figure 1A As shown, the API testing device can run on a cloud computing device system (including at least one cloud computing device, such as a server, etc.), or on an edge computing device system (including at least one edge computing device, such as a server, a desktop computer, etc.), or on various terminal computing devices, such as laptops, personal desktop computers, etc.

[0083] The API testing device may also be logically composed of multiple modules. For example, the API testing device may include an acquisition module, a parsing module, a collection module, and a providing module. Each component of the API testing device may be deployed in different systems or servers. Figure 1B As shown, the various parts of the device can be respectively operated in the three environments of the cloud computing device system, the edge computing device system or the terminal computing device, or can be operated in any two of the three environments. The cloud computing device system, the edge computing device system and the terminal computing device are connected by a communication path, and can communicate and transmit data with each other. The application program interface API testing method provided in the embodiment of the present application is executed by the combined parts of the API testing device running in the three environments (or any two of the three environments).

[0084] The embodiments of the present application do not limit the deployment method of the API testing device. For example, the API testing device can be an application deployed in the cloud to provide cloud services to users. That is, the API testing device can be deployed in an edge computing device system or a cloud computing system, and users can obtain the cloud service from the API testing device through a client deployed on the user side. The API testing device can also be a terminal computing device deployed close to the user side. For example, the API testing device can be an application software on the terminal computing device, or a plug-in of an application software on the terminal computing device.

[0085] The following is a description of the application scenarios applicable to the relationship between the user, the API testing device and the client with reference to the accompanying drawings. Figure 2 .

[0086] Figure 2 In the figure, an exemplary interface is drawn that is displayed to the user by the client's display. The user can perform operations on the client through external input and output devices (such as a display, keyboard, mouse, etc.). For example, the user can create test tasks, upload API definition documents, upload API traffic data, etc. on the client. The user can also configure test cases on the client, such as configuring the test scenarios of the test cases, the blacklist or whitelist of APIs, the domain name of the service environment under test, and the adapted computing nodes (which can be referred to as environmental information) and other information.

[0087] The API testing device can monitor the user's operations on the client, generate test cases and test prophets according to the API files uploaded by the user and the configuration of the test cases, and trigger the service node to execute the test cases, collect the API requests and corresponding API responses initiated when executing the test cases, determine the matching status of the API requests and corresponding API responses initiated when executing the test cases and the test prophet, and provide the user with the matching status of the API requests and corresponding API responses initiated when executing the test cases and the test prophet, that is, execute the application program interface testing method provided in the embodiment of the present application.

[0088] In an embodiment of the present application, an API testing device can obtain an API definition document and / or API traffic data, generate a test case and a test prophet according to the user's configuration, execute the test case, and provide the user with the API request initiated during the execution of the test case and the corresponding API response and the test prophet's matching status. The method of the embodiment of the present application can automatically complete the generation of test cases and the execution of test cases, and human participation is no longer required, which can effectively improve the generation efficiency and execution efficiency of test cases. In addition, in an embodiment of the present application, the execution results of the test case can be fed back to the user, that is, the API request initiated when executing the test case and the corresponding API response and the test prophet's matching status, which is convenient for feedback on the execution of the test case and improves the user experience.

[0089] That is, the test case generation methods involved in the embodiments of the present application can be divided into three types: the first is to generate test cases based only on the API definition document. The second is to generate test cases based only on the API traffic data. The third is to generate test cases based on the API definition document and the API traffic data.

[0090] The following describes the API testing methods performed in these three ways:

[0091] First, the API testing device generates test cases based on the API definition document.

[0092] In the first mode, after obtaining the API definition document, the API testing device generates an API operation relationship diagram according to the API definition document, and generates a test case and a test prophet based on the API operation relationship diagram and the user's configuration information. The API testing device can also send the test case to the service node, triggering the service node to execute the test case, obtain the API request and the corresponding API response initiated by the service node in the process of executing the test case, and judge the API request and the corresponding API response to generate a test report. The test report can include the matching status of the API request and the corresponding API response with the test prophet. Optionally, the test report can also include test cases and / or coverage information of test cases.

[0093] The API testing device can also provide the test report to the user through the client. The user confirms the test report and can also select a test case according to the test report to generate a test case script so that the test case can be reused.

[0094] The following is combined with Figure 3 The application program interface testing method under this mode is described, and the method includes:

[0095] Step 301: The user sends an interface test request to the API test device through the client, and the interface test request carries an API definition document. Optionally, the interface test request may also carry user configuration information.

[0096] The configuration information includes some or all of the following:

[0097] Test scenarios of test cases, API blacklist or whitelist, API parameter blacklist or whitelist, environment information, whether test case coverage information is provided, project information, and component information.

[0098] Among them, the test scenario of the test case can indicate the scenario in which the test case is applied, such as the test scenario of the test case can indicate that the test case is applied to the scenario of testing normal API calls (including normal call sequences and compliant input parameters). The test scenario of the test case can also indicate that the test case is applied to the scenario of testing abnormal API calls, that is, the test case mainly detects the scenario test of abnormal API calls (including abnormal call sequences such as disorder and illegal input parameter requests).

[0099] The API blacklist refers to the list of APIs that the test case does not need to detect. On the contrary, the API whitelist refers to the list of APIs that the test case needs to detect.

[0100] The API parameter blacklist refers to the list of API parameters that the test case does not need to detect. Conversely, the API parameter whitelist refers to the list of API parameters that the test case needs to detect.

[0101] The environment information is used to indicate the service environment required to run the test case, and the tested service environment of the test case can be indicated by the tested service domain name or the Internet protocol (IP) address and port of the cluster node.

[0102] The coverage information of a test case is used to describe the coverage of the test case on the API, that is, to describe the API that the test case can detect, and / or the code or elements in the API (elements such as methods, classes, branches). The coverage information of a test case can include coverage at different granularities, such as coverage at the API granularity and coverage at the code granularity.

[0103] The coverage of API granularity includes but is not limited to: scenario coverage, API method coverage (API method coverage is used to indicate the number of APIs that can be covered by test cases), and API parameter coverage (API parameter coverage is used to indicate the API parameters that can be covered by test cases).

[0104] Among them, scenario coverage refers to the coverage of input generation, output parameter values ​​and API operation sequences when the users or test (beta) users covered by the test case actually call the API.

[0105] The coverage of code granularity includes but is not limited to: code coverage (code coverage is used to indicate the code of the API that can be covered by the test case), class coverage (class coverage is used to indicate the number of classes in the API that can be covered by the test case), method coverage (method coverage is used to indicate the number of methods in the API that can be covered by the test case), line coverage (line coverage is used to indicate the number of lines of code in the API that can be covered by the test case), branch coverage (branch coverage is used to indicate the number of branches in the API that can be covered by the test case). It should be noted that the above coverage is only an example. In specific applications, other types of coverage may also be included in the coverage information. Any coverage that can describe the coverage of the API by the test is applicable to the embodiments of the present application.

[0106] Project information refers to information such as the project number of the test object (such as the software service product version), and component information refers to the number and name information of the object being tested (microservice or embedded component).

[0107] The embodiment of the present application does not limit the execution method of step 301. For example, the user can create a test task on the client, upload an API definition document, and select or input configuration information. After the client detects the user's operation, it can send an interface test request to the API testing device.

[0108] In some scenarios, the interface test request may not carry the configuration information. The configuration information may be pre-configured on the API test device, that is, the configuration information may be default.

[0109] Step 302: After receiving the interface test request, the API testing device may parse the API definition document and generate an ODG and a test prophet.

[0110] In step 302, ODG and test prophet need to be generated. The generation methods of ODG and test prophet are described below respectively:

[0111] 1. ODG

[0112] The API testing device can parse the API definition document, analyze the parameters of the API defined in the API definition document, and determine the dependencies between the API parameters. The dependencies between the API parameters indicate the reference relationship between the API parameters. For example, the output parameter of API1 is the input parameter of API2, and the input of API2 needs to reference the output of API1. There is a dependency between the parameters of API2 and API1. The dependency between the parameters also indicates that there is a dependency between the API operations. Therefore, the API testing device can generate ODG based on the dependency between the API parameters.

[0113] In an embodiment of the present application, when generating ODG, the API testing device can call an API document parser and input the API definition document into the API document parser to generate a readable API intermediate expression. The information contained in the API intermediate expression and the API definition document can be equivalent. The difference lies in the format of the API intermediate expression and the API definition document. The format of the API intermediate expression is a format that is convenient for subsequent data processing. The method of converting the API definition document into the API intermediate expression enables API definition documents of different formats to generate an API intermediate expression with a unified data format after format conversion, which is convenient for subsequent data processing and also expands the application scope of the embodiment of the present application.

[0114] The embodiments of the present application do not limit the method of converting API intermediate expressions. Any conversion method that can unify the data format is applicable to the embodiments of the present application.

[0115] See also Figure 4AAfter the API testing device generates the API intermediate expression, ODG can be generated based on the API intermediate expression. When generating ODG based on the API intermediate expression, the different APIs included in the API intermediate expression can be analyzed, and the dependency relationship between the parameters in the API can be constructed through the complete matching of parameter names (i.e., the parameter names are exactly the same), substring matching (i.e., one parameter name is a substring of another parameter), word segmentation matching (i.e., the parameter stems are exactly the same), modification distance (i.e., the modification distance between the two parameter names is less than the default value) and create, read, update, and delete (CRUD) semantics (i.e., the rules for creating, obtaining, modifying, and deleting the same resource), and ODG can be generated based on the dependency relationship between the API parameters. Figure 4B Shown is a schematic diagram of ODG.

[0116] As shown in 4A, the establishment of dependency between API1 and API2 is used as an example. First, the parameters in API1 and API2 are matched. If there is no parameter match, based on CRUD semantic matching, determine whether there is a dependency relationship between API1 and API2. If the match is successful through CRUD semantic matching, add the dependency. For example, if API1 creates resource A and API2 modifies the resource A, there is a dependency between the creation operation of API1 and the modification operation of API2. Add the dependency between the creation operation of API1 and the modification operation of API2. If the match fails through CRUD semantic matching, if there is no operation on the same resource in API1 and API2, it means that there is no dependency between API and API2, and continue to match the next group of APIs.

[0117] If there is a parameter match, the parameters are further matched. Figure 4A Take parameter 1 and parameter 2 as an example, where parameter 1 determines the parameters of API 1 and parameter 2 is the parameter of API 2. Determine whether parameter 1 and parameter 2 have a complete match in parameter name, a substring match, a word segmentation match, and calculate the modification distance. If the match is successful, add dependencies, match and identify, and continue to match the next group of APIs.

[0118] 2. Test the Prophet

[0119] The API testing device can call the Oracle miner to analyze the API definition document and output the test oracle.

[0120] See also Figure 4C ,When Oracle miner analyzes the API definition document, it can perform ,static analysis, causal inference, and frequent item set distribution analysis on the API ,definition document or API intermediate expression, and ,combined with the Bayesian network, output the test ,prophet.

[0121] In the embodiment of the present application, a test prophet can be obtained by analyzing the API definition document, that is, the test prophet is more in line with the API definition document, and does not require manual configuration of the test prophet, which can effectively reduce human intervention, ensure the accuracy of the test prophet, and further ensure the generation of efficient test cases.

[0122] Step 303: The API testing device may generate a test case set according to the configuration information and ODG. The test case set includes at least one test case. Figure 4D The figure shows a schematic diagram of a test case.

[0123] See also Figure 4E If the configuration information indicates that the test case is applied to the scenario of normal API call, the API test device can traverse the ODG to generate a test case set according to the ODG. The API test device can define the execution sequence of the API operations according to the dependency relationship between the API operations in the ODG, and can also determine the parameter data instance of the API request when calling the API. The parameter data instance can be randomly configured by the API test device, or configured according to the parameter constraints recorded in the API definition document, so that the parameter data instance meets the parameter constraints.

[0124] If the configuration information indicates that the test case is applied to the scenario of normal API call, the API test device can cross-mutate the ODG according to the ODG, that is, change the dependency between the API operations defined by the ODG, and generate a test case set based on the cross-mutated ODG. The API test device can define the execution sequence of the API operations according to the dependency between the API operations in the cross-mutated ODG, and can also determine the parameter data instance of the API request when calling the API. The parameter data instance can be randomly configured by the API test device or configured according to a pre-configured mutator bundle. The mutator can indicate whether the API input parameter or output parameter is negated, whether to change the parameter type, specify that the numeric type parameter exceeds the upper and lower bounds, indicate whether the parameter is allowed to be missing, change the data structure in the API (copy subnodes, delete subnodes, modify subnodes), whether to allow special encoding formats as parameter values, whether to change parameter names, etc.

[0125] It should be noted that if the configuration information includes an API blacklist, the test cases in the test case set comply with the API blacklist in the configuration information. That is, the test cases in the test case set do not detect the APIs in the API blacklist. If the configuration information includes an API parameter blacklist, the test cases in the test case set comply with the API parameter blacklist in the configuration information. That is, the test cases in the test case set do not detect the API parameters in the API parameter blacklist. If the configuration information includes an API whitelist, the test cases in the test case set comply with the API whitelist in the configuration information. That is, the test cases in the test case set detect the APIs in the API whitelist. If the configuration information includes an API parameter whitelist, the test cases in the test case set comply with the API parameter whitelist in the configuration information. That is, the test cases in the test case set detect the API parameters in the API parameter whitelist.

[0126] Step 304: The API testing device triggers the service node to execute the test cases in the test case set, and obtains the API request and the corresponding API response initiated by the service node during the execution of the test cases. If the environment information is defined in the configuration information, the service node complies with the environment information.

[0127] The API testing device can send the test cases in the test case set to the service node, instructing the service node to execute the test case. When the service node executes the test case, the service node can call the API involved in the test case based on the test case, initiate an API request according to the parameter data instance indicating the API request in the test case, and interact with the service node under test. That is, the service node sends an API request to the service node under test and receives an API response from the service node under test. The API testing device can obtain the API request and the API response.

[0128] Step 305: The API testing device determines the target test case based on the API request initiated by the service node during the execution of the test case, the corresponding API response and the test prophet. The target test case is the test case in the test case set that does not match the test prophet, that is, the API request initiated by the target test case during execution and the corresponding API response do not match the test prophet.

[0129] See also Figure 4FThe API testing device can classify the API responses corresponding to the API requests into two categories according to the test prophet: one is a successful API response, which is an API response that complies with the test prophet, and the successful API response can include a 200 response or a 400 response. The other is a failed API response, which is an API response that does not comply with the test prophet, and the failed API response can be a 500 response.

[0130] Step 306: The API testing device updates the ODG according to the target test case, and the updated ODG can execute steps 303 to 305.

[0131] When the API testing device updates the ODG according to the target test case, it can use the target test case for iterative splitting, generate multiple sub-test cases according to the target test case, extract successful sub-test cases and failed sub-test cases, and a successful sub-test case refers to an API request and a corresponding API response in the process of executing the sub-test case as a successful API response, such as an API request that meets the test prophet, including a 200 response and a 400 response. A failed sub-test case refers to an API request and a corresponding API response in the process of executing the sub-test case as a failed API response, such as an API request that does not meet the test prophet, including a 500 response. Afterwards, multiple successful sub-test cases can be combined, multiple failed test sub-cases can be combined, and successful sub-test cases and failed sub-test cases can be combined, and the service node can be triggered to execute the combined test case for testing, and the dependency between API operations can be determined based on the test results.

[0132] See also Figure 4G As shown in the figure, the API test order in the target test case is to test API1 first, then API2, then API3, and finally API4. When the service node executes the target test case, the API responses of API1 and API2 are 200 responses, the API response of API3 is 400 response, and the API response of API4 is a failed API response that does not conform to the test prophet. Figure 4G0 is used to indicate a failed API response that does not conform to the test prophet. The target test case is divided into two sub-test cases, namely, a sub-test case for testing API1 and API4, and a sub-test case for testing API3 and API4. Among them, during the process of the service node executing the sub-test case for testing API1 and API4, the API response of API1 and API2 is a 200 response and a failed API response that does not conform to the test prophet (not a 200 response), and the sub-test case for testing API1 and API4 is a failed sub-test case. During the process of the service node executing the sub-test case for testing API3 and API4, the API response of API3 is a 200 response, and the API3 response of API4 is a failed API response that does not conform to the test prophet, and the sub-test case for testing API3 and API4 is a failed sub-test case. Continue to split the sub-test cases for testing API3 and API4, decomposing them into sub-test cases for testing API3 and sub-test cases for testing API4. During the process of the service node executing the sub-test case for testing API3, the API response of API3 is a 200 response, which is a successful sub-test case. During the process of the service node executing the sub-test case for testing API4, the API response of API4 is a 400 response, which is a successful sub-test case. The sub-test case for testing API4 is combined with the sub-test cases for testing API1 and API2, and the dependency between API operations is determined based on the result of the service node executing the combined test case. If the result of executing the combined test case indicates that the API response matches the test oracle, it means that there is a new dependency between API operations, which can be updated to ODG, otherwise it will not be updated.

[0133] The API testing device can directly collect and display the test case set to the user. The API testing device can also continue to perform one or more target test case determination processes. When the API testing device performs the next target test case set determination process, it can use the last determined target test case to update the ODG, and then perform steps 303 to 305 based on the updated ODG. A test case set will be generated in each target test case determination process. That is, the API testing device can iteratively perform multiple target test case determination processes, and the target test case determined in each iteration process can be used to update the ODG. After that, the updated ODG can be used for the next iteration, that is, the API testing device can perform step 306.

[0134] Step 306: the API testing device updates the ODG according to the target test case, and the API testing device uses the updated ODG to perform the next target test case, that is, uses the updated ODG to execute steps 303 to 305.

[0135] The API testing device may perform steps 303 to 306 multiple times, that is, the API testing device performs multiple iterations. The embodiment of the present application does not limit the number of iterations performed by the API testing device. The number of iterations may be pre-configured. If the number of iterations of the API testing device reaches the pre-configured number value, the API testing device may stop iterating.

[0136] The API testing device can also use the iteration time to constrain the number of iterations. If the time for multiple iterations of the API testing device reaches a preset time value, the API testing device can stop iterating and provide the user with the matching status of the API requests and API responses initiated during the execution of the test case in each iteration and the test prophet.

[0137] Step 307: The API testing device can summarize the API requests and API responses initiated during the execution of the test case in each iteration and generate a test report. The test report may include the matching status of the API requests and API responses initiated during the execution of the test case, and the test case set.

[0138] If the configuration information requires providing coverage information of test cases, the API testing device may also analyze the test case set to generate coverage information of the test cases in the test case set. The coverage information of the test cases may be used as part of the test report.

[0139] Step 308: The API testing device provides the test report to the user; if the configuration information requires the provision of coverage information of the test cases, the test also includes the coverage information of the test cases.

[0140] The following describes a method for generating coverage information of a test case. The API testing device may generate coverage information of a test case by means of black box feedback and / or gray box feedback.

[0141] Gray box feedback refers to the process in which the API testing device collects coverage information of test cases, such as class coverage, method coverage, module coverage, line coverage, branch coverage, etc., during the execution of test cases. The code coverage feedback unit (such as Java Coverage Agent) in the module provided in the API testing device can be used to collect coverage information of test cases.

[0142] The structure of the code coverage feedback unit is as follows Figure 4HAs shown, the code coverage feedback unit mainly includes three components: an instrumentation component, a coverage analysis component, and an information / control component. First, the code coverage feedback unit inserts probes inside the test case, and when the program execution touches the probe, the execution information will be recorded for the next step of analysis. For stability reasons, the code coverage feedback unit reuses the probe mechanism of Jacoco, but makes improvements when inserting probes, allowing selective instrumentation based on the detailed level of collection of coverage information provided by the user. Therefore, the number of inserted probes can be reduced, thereby increasing the execution speed of the program. After collecting the original probe information, the code coverage feedback unit needs to compare the content of the class file to generate detailed coverage information. In an embodiment of the present application, the code coverage feedback unit is allowed to cache a copy of the uninserted class content after the class of the program under test is loaded into the memory. Through this mechanism, the next step of analysis can be automatically performed after the original probe information is collected to generate the final coverage information without user intervention. Finally, the code coverage feedback unit can display the collected coverage information in a variety of ways. It allows the use of a calling interface to provide coverage information display, collection, and display services, that is, it allows self-query of coverage information.

[0143] Black box feedback is used to analyze API responses, such as analyzing the status code, response header, message body, etc. of the API response, and to generate coverage information at the API granularity (such as scenario coverage, API method coverage information, and API generation coverage information.

[0144] It should be noted that in an embodiment of the present application, in addition to being displayed to the user, the coverage information of the test cases can also be used in multiple rounds of iterations. During each round of iteration, the API testing device can generate coverage information for the test cases in the test case set generated during the round of iteration. The API testing device can further screen the test case set based on the coverage information of the test cases. The API testing device can mutate the screened test cases with the help of a genetic algorithm to generate a new set of test cases. This new set of test cases can be added to the next round of iterations, such as being added to the test case set generated during the next round of iterations as part of the test case set; continue to be sent to the service node, triggering the service node to execute the test case set.

[0145] For example, the API testing device may screen the test case set according to a preset objective function (which may be understood as a coverage condition). In the embodiment of the present application, the objective function may score the test case according to factors such as the coverage information of the test case. The API testing device may retain the test cases with scores higher than a preset threshold and discard the test cases with scores lower than the preset threshold.

[0146] Of course, in a possible scenario, the test case set may not be further screened based on the test case coverage information, but the test cases in the test case set in the current iteration may be mutated directly with the help of a genetic algorithm to generate a new set of test cases. This new set of test cases may be added to the next iteration as part of the test case set generated in the next iteration.

[0147] Since the test case set includes multiple test cases, regardless of whether the API test device performs multiple iterations, the number of API responses corresponding to the API requests obtained by the API test device may be large. In addition to classifying the API responses corresponding to the API requests into successful API responses and failed API responses, the API test device can also cluster multiple API responses in the failed API responses, classify API responses with the same failure cause or the same failure location into one category, and mark the failure cause or failure location. For example, the API test device can classify 500 responses into one category. In the case where the API response is inconsistent with the test prophet, it can be further classified according to the inconsistent location (that is, the failure location), and the APIs with the same failure location can be classified into one category.

[0148] To facilitate user viewing, the API testing device can display classified and clustered API responses to users when providing them with the matching status of API responses and test prophets during the execution of test cases in a test case set. For failed API responses, the API testing device can also display the failure reason or failure location of the API response when displaying the API response.

[0149] Optionally, the API testing device may also generate the test case script according to the user's selection.

[0150] The user can browse the test cases in the displayed test cases, and can also select one or more test cases therefrom, and instruct the API test device to generate the selected test case text and test case script through the client.

[0151] The test case text describes the test object, test preconditions, test steps, and test expected results in natural language. The test case script describes the test object, test method, test business parameter instance, and test environment parameter instance in program executable language. The test script can be automatically executed by the test execution machine Agent.

[0152] Second, the API testing device generates test cases based on API traffic data.

[0153] In the second method, after obtaining the API traffic data, the API testing device generates an operation transfer probability graph according to the API traffic data, and generates a test case and a test prophet based on the operation transfer probability graph. The API testing device can also send the test case to the service node, triggering the service node to execute the test case, obtain the API request and the corresponding API response initiated by the service node in the process of executing the test case, and analyze the API request and the corresponding API response according to the test prophet to generate the test report, which includes the matching status of the API request and the corresponding API response with the test prophet, and can also include the coverage information of the test case and / or the test case.

[0154] The API testing device can also provide the test report to the user through the client. The user confirms the test report and can also select a target test case according to the test report and generate a target test case script so that the target test case can be reused.

[0155] The following is combined with Figure 5 , an application interface test case generation method provided in an embodiment of the present application is described.

[0156] Step 501: The user sends an interface test request to the API test device through the client, and the interface test request carries API flow data. Optionally, the interface test request may also carry user configuration information.

[0157] The configuration information can be found in the above description, which will not be repeated here. It should be noted that in this scenario, the configuration information may also include sensitive information, which refers to user accounts, passwords, IP addresses and other personal privacy information that may be involved in the API traffic data.

[0158] The embodiment of the present application does not limit the execution method of step 501. For example, the user can create a test task on the client, upload API traffic data, and select or input the configuration information of the test case. After the client detects the user's operation, it can send an interface test request to the API test device.

[0159] Step 502: After receiving the test interface test request, the API test device can parse the API traffic data and generate an operation transition probability graph, a test oracle, and a parameter constraint dictionary. Fig. 6A As shown, this is a schematic diagram of the operation transition probability graph.

[0160] When parsing the API traffic data, the API testing device can also perform data cleansing on the API traffic data. For example, sensitive information in the API process data can be deleted. If the configuration information includes an API blacklist, the API testing device can delete the API requests and corresponding API responses when calling the APIs in the API blacklist in the API traffic data. If the configuration information includes an API whitelist, the API testing device can retain the API requests and corresponding API responses when calling the APIs in the API whitelist in the API traffic data. If the configuration information includes an API parameter blacklist, the API testing device can delete the parameters in the API parameter blacklist in the API traffic data. If the configuration information includes an API parameter whitelist, the API testing device can retain the parameters in the API parameter whitelist in the API traffic data.

[0161] Afterwards, an operation transition probability graph and a test prophet may be generated based on the cleaned API traffic data. In step 502, the operation transition probability graph and the test prophet need to be generated. The following describes the generation methods of the operation transition probability graph and the test prophet respectively:

[0162] 1. Operation transition probability diagram

[0163] The API testing device can analyze API traffic data and generate an operation transition probability graph through parameter matching (i.e., parameter value equality) and CRUD semantic analysis (i.e., creation, acquisition, modification, and deletion rules for the same resource).

[0164] The way in which the API testing device analyzes API traffic data to generate an operation transfer probability graph is similar to the way in which the API definition document is analyzed to generate an operation dependency graph. The API testing device can determine the dependencies between API parameters by analyzing API traffic data, using parameter matching and CRUD semantic analysis, and further determine the dependencies between API operations.

[0165] The API testing device can also determine the dependency probability between API operations by counting the number of occurrences of APIs with dependency relationships in the API traffic data.

[0166] 2. Test the Prophet

[0167] The API testing device can call the Oracle miner to analyze the API traffic data and output the test oracle.

[0168] When Oracle Miner analyzes API traffic data, it can perform static analysis, causal inference, and analysis of the distribution of frequent itemsets, and combine it with the Bayesian network to output a test oracle.

[0169] 3. Parameter Constraint Dictionary

[0170] The parameter constraint dictionary records the value range or value of the API parameter, and can also record the dependency relationship between parameters. For example, the dependency relationship between parameters can appear at the same time or at different times.

[0171] The API testing device can determine the distribution of values ​​of various API parameters and the occurrence of parameters based on API traffic data, and then generate a parameter constraint dictionary.

[0172] Step 503: The API testing device may generate a test case set according to the parameter constraint dictionary, configuration information, and operation transition probability graph. The test case set includes at least one test case. Figure 6B The above is a schematic diagram of a test case.

[0173] If the configuration information indicates that the test case is applied to a scenario where the API is called normally, the API testing device can traverse the operation transition probability graph according to the operation transition probability graph to generate a test case set. The API testing device can define the execution sequence of the API operations according to the dependency relationship between the API operations in the operation transition probability graph, and can also determine the parameter data instance of the API request when calling the API. The parameter data instance can be randomly configured by the API testing device, or can be configured according to a parameter constraint dictionary so that the parameter data instance conforms to the parameter constraint dictionary.

[0174] If the configuration information indicates that the test case is applied to a scenario where the API is called normally, the API testing device can generate a test case set based on the abnormal dependencies recorded in the operation transition probability graph (i.e., the dependencies found in the abnormal call situation). Optionally, the API testing device can also constrain the test case set. For example, the API testing device can also use a mutation operator to mutate the parameters involved in the test cases in the test case set. The embodiment of the present application does not limit the specific type of the mutation operator.

[0175] It should be noted that if the configuration information includes an API blacklist, the test cases in the test case set comply with the API blacklist in the configuration information. That is, the test cases in the test case set do not detect the APIs in the API blacklist. If the configuration information includes an API parameter blacklist, the test cases in the test case set comply with the API parameter blacklist in the configuration information. That is, the test cases in the test case set do not detect the API parameters in the API parameter blacklist. If the configuration information includes an API whitelist, the test cases in the test case set comply with the API whitelist in the configuration information. That is, the test cases in the test case set detect the APIs in the API whitelist. If the configuration information includes an API parameter whitelist, the test cases in the test case set comply with the API parameter whitelist in the configuration information. That is, the test cases in the test case set detect the API parameters in the API parameter whitelist.

[0176] Step 504: The API testing device triggers the service node to execute the test cases in the test case set, and obtains the API request and the corresponding API response initiated by the service node in the process of executing the test cases. If the service environment domain name is defined in the configuration information, the service node is the node in the service environment domain name. If the service environment node is defined in the configuration information, the service node can be the service environment node defined in the configuration information. For details, please refer to the relevant instructions of step 304, which will not be repeated here.

[0177] Step 505: The API testing device determines the target test case according to the API request initiated by the service node in the process of executing the test case, the corresponding API response and the test prophet, and the target test case is the test case in the test case set that does not match the test prophet. For details, please refer to the relevant description of step 305, which will not be repeated here.

[0178] The way in which the API testing device updates the operation transition probability graph according to the target test case is similar to the way in which the ODG is updated according to the target test case in step 306 , and the details can be found in the above description and will not be repeated here.

[0179] The API testing device can directly collect and display the test case to the user. The API testing device can also continue to perform one or more target test case determination processes. When the API testing device performs the next target test case determination process, it can use the target test case determined last time to update the operation transition probability graph, and then perform steps 303 to 305 based on the updated operation transition probability graph. In each target test case determination process, a test case set will be generated. That is, the API testing device can iteratively perform multiple target test case determination processes, and the target test case determined in each iteration process can be used to update the operation transition probability graph. After that, the updated operation transition probability graph can be used for the next iteration, that is, the API testing device can perform step 506.

[0180] Step 506: The API testing device updates the operation transition probability graph according to the target test case, and the updated operation transition probability graph can execute steps 503 to 505. For details, please refer to the relevant description of step 306, which will not be repeated here.

[0181] Step 507: The API testing device can summarize the API requests and API responses initiated during the execution of the test case in each iteration and generate a test report. The test report may include the matching of the API requests and API responses initiated during the execution of the test case, and the test case set. For details, please refer to the relevant description of step 307, which will not be repeated here.

[0182] Step 508: The API testing device provides the test report to the user; if the configuration information requires the provision of coverage information of the test case, the test also includes the coverage information of the test case. For details, please refer to the relevant description of step 308, which will not be repeated here.

[0183] The third type, the API testing device generates test cases based on the API definition document and API traffic data.

[0184] In the third mode, after obtaining the API definition document and API traffic data, the API testing device generates an API operation relationship diagram according to the API definition document and API traffic data, and generates a test case and a test prophet based on the API operation relationship diagram. The API testing device can also send the test case to the service node, triggering the service node to execute the test case, obtain the API request and the corresponding API response initiated by the service node in the process of executing the test case, and judge the API request and the corresponding API response according to the test prophet, and generate the test report, which includes the matching status of the API request and the corresponding API response with the test prophet, and can also include the coverage information of the test case.

[0185] The API testing device can also provide the test report to the user through the client. The user confirms the test report and can also select a test case according to the test report to generate a test case script so that the test case can be reused.

[0186] The following is combined with Figure 7 , an application interface test case generation method provided in an embodiment of the present application is described.

[0187] Step 701: The user sends an interface test request to the API test device through the client, and the interface test request carries API flow data and an API definition document. Optionally, the interface test request may also carry user configuration information.

[0188] The description of this configuration information can be found in the above description and will not be repeated here.

[0189] The embodiment of the present application does not limit the execution method of step 701. For example, the user can create a test task on the client and upload API traffic data, API definition documents, select or input configuration information. After the client detects the user's operation, it can send an interface test request to the API testing device.

[0190] Step 702: After receiving the interface test request, the API testing device can parse the API traffic data and the API definition document to generate an API operation relationship diagram, a test oracle, and a parameter constraint dictionary. The API operation relationship diagram is a diagram obtained by merging the operation dependency diagram and the operation transition probability diagram. In other words, the API testing device can parse the API traffic data to generate an operation transition probability diagram, and parse the API definition document to generate an operation dependency diagram. After the operation dependency diagram and the operation transition probability diagram are generated, the operation dependency diagram and the operation transition probability diagram are merged (i.e., a union is taken) to generate an API operation relationship diagram.

[0191] When parsing the API traffic data, the API test device can also perform data cleaning on the API traffic data. The method of data cleaning can be referred to the above description and will not be repeated here. The method of generating the operation transition probability graph, the operation dependency graph, the test oracle and the parameter constraint dictionary can be referred to the above description and will not be repeated here.

[0192] In this way, since the API operation relationship graph includes the information in the operation dependency graph and the operation transition probability graph, the information in the API operation relationship graph is richer, ensuring that more efficient test cases can be generated subsequently.

[0193] Step 703: The API testing device may generate a test case set according to the parameter constraint dictionary, the configuration information, and the API operation relationship diagram. The test case set includes at least one test case. The API testing device generates a test case set according to the configuration information and the API operation relationship diagram, as described in the relevant description of steps 303 and 503.

[0194] Step 704: The API testing device triggers the service node to execute the test cases in the test case set, and obtains the API request and the corresponding API response initiated by the service node in the process of executing the test cases. If the service environment domain name is defined in the configuration information, the service node is the node in the service environment domain name. If the service environment node is defined in the configuration information, the service node can be the service environment node defined in the configuration information. For details, please refer to the relevant instructions of step 304, which will not be repeated here.

[0195] Step 705: The API testing device determines the target test case according to the API request initiated by the service node in the process of executing the test case, the corresponding API response and the test prophet, and the target test case is the test case in the test case set that does not match the test prophet. For details, please refer to the relevant description of step 305, which will not be repeated here.

[0196] Step 706: The API testing device updates the API operation relationship diagram according to the target test case, and the updated API operation relationship diagram can execute steps 703 to 705. For details, please refer to the relevant description of step 306, which will not be repeated here.

[0197] The way in which the API testing device updates the API operation relationship diagram according to the target test case is similar to the way in which the ODG is updated according to the target test case in step 306. For details, please refer to the above description and no further details will be given here.

[0198] The API testing device can directly use the target test case as the final test case and display it to the user. The API testing device can also continue to perform the target test case determination process once or multiple times. When the API testing device executes the next target test case, it can use the target test case determined last time to update the API operation relationship diagram, and then execute steps 703 to 305 based on the updated API operation relationship diagram. That is, the API testing device can use the process of generating the final test case as an iterative process, and the target test case determined in each iteration process can be used to update the API operation relationship diagram. After that, the updated API operation relationship diagram can be used for the next iteration, that is, the API testing device can execute step 706.

[0199] Step 707: The API testing device can summarize the API requests and API responses initiated during the execution of the test case in each iteration and generate a test report. The test report may include the matching of the API requests and API responses initiated during the execution of the test case, and the test case set. For details, please refer to the relevant description of step 307, which will not be repeated here.

[0200] Step 708: The API testing device provides the test report to the user; if the configuration information requires the provision of coverage information of the test case, the test also includes the coverage information of the test case. For details, please refer to the relevant description of step 308, which will not be repeated here.

[0201] Based on the same inventive concept as the method embodiment, the present application embodiment also provides an API testing device, which is used to execute the method executed by the API testing device in the above method embodiment. Figure 8 As shown, the API testing device 800 includes an acquisition module 801, a parsing module 802, a generation module 803, a collection module 804, and a providing module 805. Specifically, in the API testing device, each module is connected through a communication path.

[0202] The acquisition module 801 is used to obtain the application program interface API file transmitted by the user. The API file includes an API definition document and / or API traffic data. The API definition document includes description information of the API. The API traffic data includes the API request initiated when calling the API and the API response corresponding to the API request.

[0203] The parsing module 802 is used to parse the API file, generate an API operation relationship diagram and a test prophet. The API operation relationship diagram is used to describe the dependency relationship of the API operation, and the test prophet is used to predict the rules of the API request and the corresponding API response when calling the API.

[0204] The generating module 803 is used to generate a first test case set according to the API operation relationship diagram.

[0205] The collection module 804 is used to trigger the service node to execute the test cases in the first test case set, and obtain the API requests and corresponding API responses initiated by the service node during the execution of the test cases in the first test case set.

[0206] Module 805 is provided, which is used to determine the matching status of the API requests and corresponding API responses initiated during the execution of the test cases in the first test case set with the test prophet, and provide the user with the matching status of the API requests and corresponding API responses initiated during the execution of the test cases in the first test case set with the test prophet.

[0207] As a possible implementation, the API testing device may perform multiple rounds of iterations, each round of iteration taking the output of the previous round of iteration as input, and the following is described by taking a subsequent round of iteration as an example.

[0208] In this round of iteration, the generation module 803 can determine the target test case from the first test case set according to the test prophet, wherein the API request and API response initiated by the target test case during execution do not match the test prophet; thereafter, the parsing module 802 updates the API operation relationship diagram according to the target test case, and generates a second test case set according to the updated API operation relationship diagram. Thereafter, the collection module 804 can trigger the service node to execute the test cases in the second test case set, and obtain the API request and API response initiated by the service node in the process of executing the test cases in the second test case set. The providing module 805 can determine the matching of the API request and API response initiated by the service node in the process of executing the test cases in the second test case set with the test prophet according to the test prophet, and provide the user with the matching of the API request and the corresponding API response initiated in the process of executing the test cases in the second test case set with the test prophet.

[0209] As a possible implementation, when the API file includes API traffic data, the parsing module 802 may generate an API operation relationship diagram and a parameter constraint dictionary when parsing the API file of the API, and the parameter constraint dictionary includes the value range of the API parameters.

[0210] As a possible implementation, the parameters of the API request defined by the test cases in the first test case set conform to the parameter constraint dictionary.

[0211] As a possible implementation, the acquisition module 801 may also acquire configuration information of the user, where the configuration information is used to indicate the rules that the test case needs to follow, and the configuration information includes part or all of the following information:

[0212] Test scenarios of test cases, API blacklist or whitelist, API parameter blacklist or whitelist, and whether to provide test case coverage information.

[0213] As a possible implementation, the test cases in the first test case set conform to the configuration information.

[0214] As a possible implementation, the providing module 805 may also determine the coverage information of the test cases in the first test case set, where the coverage information of the test cases describes the coverage of the API by the test cases; and provide the coverage information of the test cases to the user.

[0215] As a possible implementation, the API testing device may also perform another iteration method, in which the API testing device may also perform multiple rounds of iterations, each round of iteration using the output of the previous round of iteration as input, and a subsequent round of iteration is used as an example for explanation below.

[0216] In this round of iteration, the generation module 803 can determine a candidate test case set from the first test case set, wherein the coverage information of the test cases in the candidate test case set meets the preset coverage condition; based on the candidate test case set, a derived test case set is generated. Afterwards, the collection module 804 can trigger the service node to execute the test cases in the derived test case set, and obtain the API requests and API responses initiated by the service node in the process of executing the test cases in the derived test case set. The provision module 805 can determine the matching of the API requests and API responses initiated by the service node in the process of executing the test cases in the derived test case set with the test prophet according to the test prophet, generate a test report, and display the test report.

[0217] The division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present application may be integrated into a processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0218] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a terminal device (which can be a personal computer, mobile phone, or network device, etc.) or a processor (processor) to execute all or part of the steps of the method of each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0219] This application also provides Fig. 9 The computing device 900 shown in FIG. 900 includes a bus 901 , a processor 902 , a communication interface 903 , and a memory 904 . The processor 902 , the memory 904 , and the communication interface 903 communicate with each other via the bus 901 .

[0220] The processor 902 may be a central processing unit (CPU). The memory 904 may include a volatile memory, such as a random access memory (RAM). The memory 904 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a HDD or a SSD. The memory stores executable code, and the processor 902 executes the executable code to execute the aforementioned Figure 3 , 5 , 7. The memory 904 may also include other software modules required for running processes such as an operating system (such as multiple modules in the API testing device 800). The operating system may be LINUX TM ,UNIX TM ,WINDOWS TM wait.

[0221] The present application also provides a computing device system, wherein the computing device system includes at least one Fig.10The computing device 1000 shown in the figure includes a bus 1001, a processor 1002, a communication interface 1003 and a memory 1004. The processor 1002, the memory 1004 and the communication interface 1003 communicate with each other through the bus 1001. At least one computing device 1000 in the computing device system communicates with each other through a communication path.

[0222] The processor 1002 may be a CPU. The memory 1004 may include a volatile memory, such as a random access memory. The memory 1004 may also include a non-volatile memory, such as a read-only memory, a flash memory, a HDD or a SSD. The memory 1004 stores executable code, and the processor 1002 executes the executable code to execute the aforementioned Figure 3 , 5 , 7, or any part or all of the methods described in the preceding paragraphs. The memory may also include an operating system or other software modules required for running the process. The operating system may be LINUX TM ,UNIX TM ,WINDOWS TM wait.

[0223] At least one computing device 1000 in the computing device system establishes communication with each other through a communication network, and any one or multiple modules in the API testing device 800 are run on each computing device 1000.

[0224] The descriptions of the processes corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process, please refer to the relevant descriptions of other processes.

[0225] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes computer program instructions, which, when loaded and executed on a computer, generate all or part of the embodiments according to the present invention. Figure 3 , 5 , the process or function described in 7.

[0226] The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line, or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., an SSD).

[0227] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. An application program interface API testing method, characterized in that: The method comprises: Acquire an API file transmitted by a user, wherein the API file includes an API definition document and / or API traffic data, wherein the API definition document includes description information of the API, and the API traffic data includes an API request when calling the API and an API response corresponding to the API request; Parse the API file to generate an API operation relationship diagram and a test prophet, wherein the API operation relationship diagram is used to describe the dependency relationship of API operations, and the test prophet is the prediction rule of API request and corresponding API response when calling the API; Performing cross-mutation on the API operation relationship graph, and generating a first test case set according to the cross-mutated API operation relationship graph; Triggering the service node to execute the test cases in the first test case set, and obtaining the API request and the corresponding API response initiated by the service node in the process of executing the test cases in the first test case set; The user is provided with the matching status of the API requests and corresponding API responses initiated during the execution of the test cases in the first test case set and the test prophet.

2. The method according to claim 1, characterized in that The method further comprises: Determining a target test case from the first test case set according to the test prophet, wherein an API request and an API response initiated by the target test case when executed do not match the test prophet; Update the API operation relationship diagram according to the target test case, and generate a second test case set according to the updated API operation relationship diagram; Triggering the service node to execute the test cases in the second test case set, and obtaining API requests and API responses initiated by the service node in the process of executing the test cases in the second test case set; The user is provided with the matching status of the API requests and corresponding API responses initiated during the execution of the test cases in the second test case set and the test prophet.

3. The method according to claim 1, characterized in that When the API file includes the API traffic data, parsing the API file to generate an API operation relationship diagram includes: The API file is parsed to generate the API operation relationship diagram and parameter constraint dictionary, wherein the parameter constraint dictionary includes the value range of the API parameters.

4. The method according to claim 3, characterized in that The parameters of the API requests defined by the test cases in the first test case set conform to the parameter constraint dictionary.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Obtain the user's configuration information, where the configuration information is used to indicate the rules that the test case must follow, and the configuration information includes part or all of the following information: Test scenarios of test cases, API blacklist or whitelist, API parameter blacklist or whitelist, and whether to provide test case coverage information.

6. The method according to claim 5, characterized in that The test cases in the first test case set conform to the configuration information.

7. The method according to any one of claims 1 to 4 and 6, characterized in that: The method further comprises: Determine coverage information of the test cases in the first test case set, where the coverage information of the test cases describes coverage of the API by the test cases; The coverage information of the test case is provided to the user.

8. The method according to claim 7, characterized in that The method further comprises: Determine a candidate test case set from the first test case set, wherein coverage information of the test cases in the candidate test case set meets a preset coverage condition; Based on the candidate test case set, generating a derived test case set; The service node is triggered to execute the test cases in the derived test case set, and an API request and an API response initiated by the service node in the process of executing the test cases in the derived test case set are obtained.

9. An API testing device, characterized in that: The testing device comprises an acquisition module, a parsing module, a generation module, a collection module and a providing module; The acquisition module is used to acquire the API file transmitted by the user, wherein the API file includes an API definition document and / or API flow data, wherein the API definition document includes description information of the API, and the API flow data includes an API request initiated when calling the API and an API response corresponding to the API request; The parsing module is used to parse the API file, generate an API operation relationship diagram and a test prophet, the API operation relationship diagram is used to describe the dependency relationship of the API operation, and the test prophet is the prediction rule of the API request and the corresponding API response when calling the API; The generating module is used to perform cross-mutation on the API operation relationship graph, and generate a first test case set according to the cross-mutation; The collection module is used to trigger the service node to execute the test cases in the first test case set, and obtain the API request and the corresponding API response initiated by the service node in the process of executing the test cases in the first test case set; The providing module is used to provide the user with the matching status of the API request and the corresponding API response initiated during the execution of the test cases in the first test case set and the test prophet.

10. The testing device according to claim 9, characterized in that: The generating module is further configured to determine a target test case from the first test case set according to the test prophet, wherein an API request and an API response initiated by the target test case when executed do not match those of the test prophet; The parsing module is further used to update the API operation relationship diagram according to the target test case, and generate a second test case set according to the updated API operation relationship diagram; The collection module is further used to trigger the service node to execute the test cases in the second test case set, and obtain the API request and API response initiated by the service node in the process of executing the test cases in the second test case set; The providing module is also used to provide the user with the matching status of the API requests and corresponding API responses initiated during the execution of the test cases in the second test case set and the test prophet.

11. The testing device according to claim 9, characterized in that: When the API file includes the API traffic data, the parsing module, when parsing the API file and generating the API operation relationship diagram, is specifically used to: The API file is parsed to generate the API operation relationship diagram and parameter constraint dictionary, wherein the parameter constraint dictionary includes the value range of the API parameters.

12. The testing device according to claim 11, characterized in that: The parameters of the API requests defined by the test cases in the first test case set conform to the parameter constraint dictionary.

13. The testing device according to any one of claims 9 to 12, characterized in that: The acquisition module is further used for: Obtain the user's configuration information, where the configuration information is used to indicate the rules that the test case must follow, and the configuration information includes part or all of the following information: Test scenarios of test cases, API blacklist or whitelist, API parameter blacklist or whitelist, and whether to provide test case coverage information.

14. The testing device according to claim 13, characterized in that: The test cases in the first test case set conform to the configuration information.

15. The testing device according to any one of claims 9 to 12 and 14, characterized in that: The providing module is further used to determine the coverage information of the test cases in the first test case set, where the coverage information of the test cases describes the coverage of the API by the test cases; and provide the coverage information of the test cases to the user.

16. The testing device according to claim 15, characterized in that: The generating module is further configured to: determine a candidate test case set from the first test case set, wherein the coverage information of the test cases in the candidate test case set meets a preset coverage condition; and generate a derived test case set based on the candidate test case set; The collection module is also used to trigger the service node to execute the test cases in the derived test case set, and obtain the API request and API response initiated by the service node in the process of executing the test cases in the derived test case set.

17. A computing device, characterized in that The computing device includes a processor and a memory; The memory is used to store computer program instructions; The processor executes and calls the computer program instructions in the memory to perform the method according to any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium, characterized in that When the non-transitory computer-readable storage medium is executed by a computing device, the computing device executes the method of any one of claims 1 to 8 above.

19. A computing device program product, characterized in that The computing device program product comprises computer instructions, which when executed by a computing device, cause the computing device to perform the method according to any one of claims 1 to 8.

Images