An interface automation testing method and system

Abstract

The application relates to the technical field of computer network communication, in particular to an interface automatic testing method and system, which comprises the following steps: an interface automatic framework is built; interface information in a business process is recorded to form an interface document, and corresponding state identifiers are called from the interface automatic framework; the interface document is analyzed based on a mature large language model to obtain test cases and test scripts conforming to a predetermined template style in the interface automatic framework, and test identifiers are generated based on the state identifiers corresponding to the interface document; and the test scripts are run based on the test cases and matching timing rules, and an interface automatic report is generated. The application can improve the interface automatic testing effect.

Description

Technical Field

[0001] This application relates to the field of computer network communication technology, and in particular to an interface automated testing method and system. Background Technology

[0002] With the increasing complexity of software systems and the shortening of development cycles, manual test case construction methods can no longer meet the testing needs of modern software systems. Automated testing, as an effective solution, can significantly improve testing efficiency and ensure software quality.

[0003] However, currently, a large number of test cases and test scripts in automated testing lack a unified framework template. Furthermore, the testing process cannot be autonomously optimized and adjusted for different states of the interface, resulting in poor overall automated testing performance and low efficiency. Summary of the Invention

[0004] To improve the effectiveness of automated interface testing, this application provides an automated interface testing method and system.

[0005] Firstly, this application provides an automated interface testing method, which adopts the following technical solution:

[0006] An automated interface testing method includes the following steps:

[0007] Build an API automation framework;

[0008] The proxy pointing tool is invoked and an interface is selected. The interface is recorded and the full-link interface information is generated based on the recording results to form an interface document. The corresponding status identifier is retrieved from the interface automation framework. The status identifier includes an abandoned identifier and a priority identifier.

[0009] Based on the large language model, the interface document is analyzed according to the configuration scenario-based use case requirements to obtain test cases and test scripts that conform to the predetermined template style in the interface automation framework, and test identifiers are generated based on the status identifiers corresponding to the interface document.

[0010] The test script is run based on the timing rules matched with the test cases, and an interface automation report is generated.

[0011] In some embodiments, the process includes retrieving a proxy pointing tool and selecting an interface, recording the interface, generating end-to-end interface information based on the recording results to form an interface document, and retrieving the corresponding status identifier from the interface automation framework, including the following steps:

[0012] The interface information includes at least the interface path, interface request header, interface request body, response header, and response body;

[0013] Confirm the fillable and required parameters corresponding to the interface request body, and determine the return parameter information of the response body under different states to generate the interface document;

[0014] Based on the interface automation framework, configure abandonment and priority indicators for the interface documents.

[0015] In some embodiments, the interface document is analyzed based on a large language model to obtain test cases and test scripts that conform to the predetermined template style in the interface automation framework, including the following steps:

[0016] Retrieve test case styles and test script templates within the interface automation framework;

[0017] Configure scenario-based use case requirements, which include valid parameter scenarios, boundary value scenarios, special character scenarios, and parameter missing scenarios;

[0018] Based on the scenario-based use case requirements, the interface document is extracted and combined to generate test cases that conform to the test case style, and all test cases inherit the priority identifier in the interface document;

[0019] Extract the request information and expected response results from the test cases to write code and generate the test script that conforms to the test script template.

[0020] In some embodiments, generating a test identifier based on the status identifier corresponding to the interface document includes the following steps:

[0021] Determine whether the abandoned identifier exists in the interface document;

[0022] If a skip flag is found, a skip flag is added to the test case. If a skip flag is found before the test case is executed, the test script corresponding to that test case will skip execution.

[0023] In some embodiments, the test script is run based on timing rules matched to the test cases, and an interface automation report is generated, including the following steps:

[0024] Update the test cases and test scripts to the code management platform;

[0025] When the matched timing rule is event-triggered, if the test case and / or the test script in the code management platform are updated, the test script is executed based on the priority identifier in the corresponding interface document and the test result is obtained.

[0026] When the matching timing rule is periodically triggered, the test script is executed periodically based on the periodic time and the priority identifier in the corresponding interface document, and the test results are obtained.

[0027] The interface automation report is generated based on the results of each or several tests. The interface automation report includes basic exceptions, time-consuming exceptions, and interface success rate.

[0028] In some embodiments, the interface automation report is generated based on the results of each or several tests, specifically including:

[0029] The test cases to be executed this time are determined based on the priority identifier and the skip identifier, and the total number of test cases is counted;

[0030] Determine whether the assertion parameters of the test script match the expected results under different scenario-based use case requirements. If they match, the test is considered successful; if they do not match, the test is considered to have failed and is defined as a basic exception.

[0031] The execution time corresponding to the test script is obtained, and the difference is calculated with the expected time in the previous stable run result. If the difference is not greater than the preset time, the test is considered successful. If the difference is greater than the preset time, the test is considered to have failed and is defined as an abnormal time consumption.

[0032] The success rate of the interface is calculated based on the ratio of the total number of exceptions to the total number of test cases.

[0033] In some embodiments, when running the test script based on the timing rules matched with the test cases, the following steps are further included:

[0034] The hierarchical retry rules are obtained in the interface automation framework, and the hierarchical retry rules include immediate retries and timed retries.

[0035] When the aforementioned basic anomaly exists, the assertion parameters are retrieved to extract the cause of the anomaly, and the corresponding hierarchical retry rule is matched based on the cause of the anomaly.

[0036] When a corresponding immediate retry is performed, the test script is retried at the first time interval until the preset number of times. If the retry is successful, the basic exception is updated to occasional success, the total number of corresponding exceptions is reduced, and it is recorded in the interface automation report.

[0037] When a scheduled retry is performed, a corresponding scheduled task is set based on the priority identifier, and the test script is retried once when the scheduled task is triggered at the second time. The retry result is recorded in the interface automation report.

[0038] In some embodiments, when the underlying exception exists and the immediate retry or the timed retry fails, the following steps are further included:

[0039] Obtain the test cases corresponding to the basic exception and create a development work order;

[0040] The development work order is assigned and the developer waits for feedback on the repair of the development interface. The status label of the development work order is then updated to the trigger label.

[0041] The immediate retry is initiated based on the trigger tag, and the retry result is awaited. If the retry result is successful, the status tag is updated to a verification tag and recorded in the interface automation report. If the retry result fails, the development work order is reassigned to the development interface.

[0042] In some embodiments, generating the interface automation report based on the results of each or several tests further includes the following steps:

[0043] Set up response body tracking points. When the test results are obtained, capture the complete response body based on the response body tracking points and associate it with the test case ID to generate a query index to the log file.

[0044] When the test result corresponds to a basic anomaly or a time-consuming anomaly, the query index is used to query and retrieve the corresponding log file.

[0045] Secondly, this application provides an automated interface testing system, which adopts the following technical solution:

[0046] An automated interface testing system is provided to implement the above method.

[0047] The technical solutions provided by the embodiments of this application have the following technical effects:

[0048] The core technology stack of the framework is built around four major stages: "framework support, interface acquisition, test case generation, and execution feedback." It utilizes recording tools to automatically process interface information throughout the entire process, avoiding the omission of key parameters when manually compiling interface data. A mature large language model is introduced, leveraging its ability to analyze interface document parameter rules and response logic to automatically generate test cases and scripts that conform to the framework template. This solves the problems of low efficiency and uneven scenario coverage in manual test case design. At the continuous execution and result feedback level, an automated mechanism based on the framework's timed rules constructs a closed loop of "code submission - test execution - report generation," replacing the inefficient traditional manual test triggering model. Attached Figure Description

[0049] Figure 1 This is a schematic diagram illustrating the steps of an interface automated testing method provided in this embodiment. Detailed Implementation

[0050] To better understand the purpose, technical solutions, and advantages of this application, it has been described and illustrated below with reference to the accompanying drawings and embodiments. However, those skilled in the art should understand that this application can be implemented without these details. In some cases, to avoid obscuring various aspects of this application due to unnecessary description, well-known methods, processes, systems, components, and / or circuits already described at a higher level will not be elaborated upon. It will be apparent to those skilled in the art that various modifications can be made to the embodiments disclosed in this application, and the general principles defined in this application can be applied to other embodiments and application scenarios without departing from the principles and scope of this application. Therefore, this application is not limited to the illustrated embodiments, but conforms to the broadest scope consistent with the scope of protection claimed in this application.

[0051] It should be noted that the descriptions of these embodiments are for the purpose of aiding understanding the present invention, but do not constitute a limitation thereof. Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0052] In the description of this application, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0053] In the description of this application, the terms "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples.

[0054] like Figure 1 As shown in the figure, this application discloses an automated interface testing method, including the following steps:

[0055] S100, build an interface automation framework.

[0056] By combining existing mature technologies, such as Java + TestNG + Allure, an interface automation framework can be built. This framework will include various templates, test rules, and test evaluation criteria needed for subsequent automated testing. It also supports identifying the status information and importance of interfaces and test cases, and configuring several types of tags.

[0057] Java is one of the development languages ​​used in testing environments. Its advantages lie in its strong cross-platform compatibility and mature enterprise-level project ecosystem (such as the HTTP request library HttpClient and the interface parsing library fastJson), which can adapt to the interface types of most backend services.

[0058] TestNG is responsible for the management and execution of subsequent test cases, supporting test case grouping, priority sorting, parallel execution, assertion analysis, and more.

[0059] Allure is used to generate visual test reports, such as displaying test case execution success rates, and also supports annotation of failed test cases and log appending.

[0060] S200 records interface information in the business process to form interface documentation and retrieves the corresponding status identifiers from the interface automation framework.

[0061] By using proxy tools, you can record interface information in all business processes without having to manually write interface call code. Simply point the client's network proxy to the tool to automatically capture all interfaces in the business process. At the same time, the recording results include the "request-response full-link information" of the interface, avoiding the omission of key parameters (such as authentication information such as tokens and cookies) when manually organizing interfaces.

[0062] At the same time, the interface automation framework analyzes the data status and importance corresponding to the interface information to configure the corresponding status labels for the interface documentation.

[0063] The status labels further include abandoned and priority labels, the meanings and effects of which will be explained in detail later.

[0064] S300 analyzes interface documents based on a large language model to obtain test cases and test scripts that conform to the predefined template style in the interface automation framework, and generates test identifiers based on the status identifiers corresponding to the interface documents.

[0065] By leveraging a mature large language model (LLM) to analyze API documentation, matching test cases and scripts are generated. Both the generated test cases and scripts must meet the template requirements supported by the API automation framework. When generating test cases and scripts using the LLM, the system first receives structured input from the API documentation as the core input source. Precise prompts constrain the LLM's generation logic to meet the template and rule requirements supported by the API automation framework, automatically generating compliant test cases. Finally, executable test scripts are obtained through script API construction, test case method conversion, and HTTP request logic writing.

[0066] Meanwhile, under the API automation framework, the generated test cases and test scripts will be synchronously bound to and inherit the status identifiers corresponding to the API documentation to obtain test identifiers. Subsequently, when executing the scripts, the execution priority and whether to execute are selected based on the test identifiers.

[0067] S400 runs test scripts based on test case matching and timing rules, and generates automated interface reports.

[0068] Match the script execution time rules to the content of the test cases, and run the test script based on the time rules. Analyze and determine whether the script execution is successful based on continuously recorded interface information, and generate an interface automation report based on the execution statistics based on the success and failure results.

[0069] Based on the above technical solution, the core technology stack of the framework is constructed around the four major stages of the entire solution process: "framework support - interface acquisition - test case generation - execution feedback". The recording tool automatically records the entire process of interface information, avoiding the omission of key parameters when manually compiling interface data. A large language model is introduced, and its ability to analyze interface document parameter rules and response logic is used to automatically generate test cases and scripts that conform to the framework template. This solves the problems of low efficiency and uneven scenario coverage in manual test case design. At the level of continuous execution and result feedback, based on the automated mechanism of the timing rules under the framework, a closed loop of "code submission - test execution - report generation" is built to replace the inefficient mode of traditional manual triggering of tests.

[0070] In other embodiments, recording interface information in the business process to form interface documentation and retrieving corresponding status identifiers from the interface automation framework includes the following steps:

[0071] S210, retrieve the proxy pointing tool and select the interface, record the interface, and generate full-link interface information based on the recording results.

[0072] The interface information across the entire chain is mainly divided into several elements, namely interface path, interface request header, interface request body, response header, and response body.

[0073] An API path is an address segment used by a client (such as an app or webpage) to accurately locate the target API when making a request to the server. It needs to be combined with the server domain name (such as https: / / api.xxx.com) to complete the full request URL and is the basic location information for API calls.

[0074] It is used to uniquely identify a specific interface on the server (such as "user login interface" or "order payment interface"), ensuring that the request is sent to the correct business processing module.

[0075] The API request header is metadata that describes the request attributes, authentication, and data format when the client sends a request. It does not contain core business data and is only used by the server to identify the basic rules of the request. It is used to inform the server of the data format of the request (such as JSON / form), provide authentication information, ensure API access restrictions, and pass additional configurations for the request.

[0076] The API request body is the core data sent by the client to the server, containing the parameters required for business logic. It is the key to the API's business functionality, used to transmit the critical parameters required for the API to complete its business, determine the API's processing logic, and allow manual differentiation between required and optional parameters to meet the subsequent API documentation integrity requirements.

[0077] Response headers are information returned by the server after processing a request, describing the response attributes, data format, and status configuration. They correspond to the request headers and are used to inform the client of the format of the response data, additional status, and to convey the server's configuration information.

[0078] The response body is the data carrier containing the business processing results returned by the server after processing the request. It is the core basis for verifying whether the function is normal in interface testing. It is used to return the processing results and corresponding return data (such as returning a token when logging in successfully, returning an error message when failing, etc.), and is the key to generating test cases and writing assertions in subsequent LLM.

[0079] S220, Confirm the fillable and required parameters corresponding to the interface request body, determine the return parameter information of the response body under different states to generate the interface document.

[0080] The API request body contains required parameters that will cause an API error if missing, as well as optional parameters that do not affect the core functionality if missing. For example, in a payment API, payAmount is a required parameter, while remark is an optional parameter.

[0081] Once the fillable and required parameters are determined, the corresponding test execution scenarios can be set based on whether the required parameters are missing when designing test cases.

[0082] At the same time, it is also necessary to determine the return parameter information of the response body in different states in order to analyze whether the execution is successful or not and the reason for failure. Therefore, the return parameter information is essentially composed of two parts: one is the result parameter and the other is the reason parameter. Specifically, when the payment interface is successful, it returns 200+orderId; when the balance is insufficient, it returns 403+msg: "Insufficient balance"; and when the parameter is incorrect, it returns 400+msg: "Incorrect amount format".

[0083] S230 configures abandoned and priority flags for interface documents based on the interface automation framework.

[0084] The API automation framework supports configuration for abandoning and prioritizing APIs.

[0085] Prioritize interfaces based on their importance, such as P0 corresponding to core interfaces (e.g., login, ECG data collection), P1 corresponding to secondary interfaces (e.g., personal information query), and P2 corresponding to non-core interfaces (e.g., help service query).

[0086] When generating test cases and test scripts, the priority identifier corresponding to the interface document will be inherited and bound to generate the test order strategy.

[0087] For some interfaces that have been decommissioned or are no longer used in older versions, the automated testing framework will configure a deprecated tag in the recorded interface documentation for that interface. For deprecated interfaces, the test cases corresponding to those interfaces need to be automatically marked as non-running in the future, so that the test scripts can skip execution directly and avoid invalid execution.

[0088] In other embodiments, the interface documentation is analyzed based on a large language model to obtain test cases and test scripts that conform to the predefined template style in the interface automation framework, including the following steps:

[0089] S310 retrieves test case styles and test script templates within the interface automation framework.

[0090] A unified test case format includes test case ID, interface URL, request method, request parameters, header parameters, expected response information, and priority. A unified script interface includes encapsulation of HTTP requests, response parsing, and assertion logic. This avoids confusion in test case formats written by different testers and different test interfaces, and reduces repetitive coding.

[0091] S320, configure scenario-based use case requirements.

[0092] When generating test cases, it is necessary to match the functional completeness requirements of the test cases and cover multiple key scenarios to ensure the effectiveness and accuracy of the subsequently generated test scripts. Specifically:

[0093] Valid parameter scenarios: verifying the correctness of the interface under "normal input", such as the login interface passing in the correct username and password.

[0094] Boundary value scenario: Verify whether the extreme values ​​of parameters are handled correctly to determine whether the handling of boundary parameters meets expectations (such as extreme value parameters taking effect normally, and parameters exceeding the extreme value returning reasonable prompts), so as to avoid device data abnormalities or functional failures due to improper handling of parameter boundaries.

[0095] Special character scenarios: Verify the interface's compatibility with abnormal inputs, such as parameters containing spaces or special symbols, to avoid interface crashes or device data parsing failures due to improper character processing.

[0096] Missing Required Parameter Scenarios: Verify the interface's tolerance for "missing required parameters" and determine whether the interface can correctly process business logic with valid parameters. For example, in a real-time ECG data upload interface, required parameters include deviceID (unique device identifier), sampleRate (sampling rate), and ecgDate (ECG waveform data).

[0097] S330 extracts and combines the content of the interface document according to the requirements of the scenario-based test cases to generate test cases that conform to the test case style, and all test cases inherit the priority identifier in the interface document.

[0098] First, the LLM large language model obtains structured input from the interface documentation information. It parses the interface path, request method, and request header in the interface documentation to clarify the basic rules for calling the interface. The LLM large model focuses on identifying the required parameters, optional parameters, and parameter attributes (type, value range) given in the request body. At the same time, it extracts the scenario-based test case requirements and return rules from the response body. The LLM large model captures the priority identifier and instrument identifier in the interface documentation. The priority identifier will be directly used to inherit and bind with the generated test cases, while the abandonment identifier will be used to indicate whether the subsequently generated test scripts need to be executed.

[0099] The generation logic of LLM is constrained by precise prompt words. The prompt words mainly include three core logics: test case style rules, script template rules, and test scenario coverage rules.

[0100] The test case style rules explicitly require that test cases must include "test case ID, associated interface information (path / method), test scenario description, input parameters (required + optional combination), expected response result (response code + response body fields), and priority", and the format must be compatible with the framework.

[0101] The script template rules explicitly require that the scripts conform to Java + TestNG syntax and include TestNG annotations (such as @Test specifying test cases, @Priority specifying priority), Allure annotations (such as @Feature annotating interface modules, @Story annotating test scenarios), HTTP request encapsulation (using OkHttp to send requests), and response assertion logic (verifying response codes and expected fields).

[0102] The test scenario coverage rule requires test cases to cover all of the above-mentioned test scenarios.

[0103] LLM combines the required parameters and valid values ​​of optional parameters in the interface documentation to generate test cases for "fully valid parameter input"; LLM identifies the boundary of parameter value range and generates test cases for boundary extreme value input; LLM generates parameter input test cases containing special characters / abnormal formats.

[0104] S340: Extract request information and expected response results from test cases to write code and generate test scripts that conform to the test script template.

[0105] When setting up the script, the interface automation structure is built into Java classes, including TestNG class annotations and class attributes; each test case is transformed into an independent test method, with the method name corresponding to the test case ID, and annotations are added to bind priority and Allure annotations are used to describe the scenario; inside the method, an HTTP request (such as the JSON request body of a POST request) is constructed based on the test case's "input parameters", the OkHttp utility class encapsulated by the framework is called to send the request, and the time consumption is counted through instrumentation; based on the expected response result of the test case, TestNG assertion code is written, such as verifying that the response code is 200 and verifying that the response body contains a token field.

[0106] In other embodiments, generating a test identifier based on the status identifier corresponding to the interface document includes the following steps:

[0107] S350, determine whether there is an abandoned identifier in the interface document.

[0108] S360, if it exists, add a skip flag to the test case. When a test case is filtered out before execution, the test script corresponding to that test case will skip execution.

[0109] For deprecated interfaces, the corresponding test cases need to be automatically marked as non-running. The framework automatically filters out the associated test cases by reading the deprecation flags in the interface documentation and marks them as skipped. Since there is a one-to-one correspondence between scripts and test cases, once a test case is marked as skipped, the corresponding script will be automatically skipped by the framework, preventing invalid code from consuming execution resources.

[0110] In other embodiments, timing rules are matched based on test cases to run test scripts and generate interface automation reports, including the following steps:

[0111] S410 updates test cases and test scripts to the code management platform.

[0112] Submit test scripts and test cases to code management platforms such as Git to achieve version control (such as rolling back to historical script versions) and team collaboration (multiple people jointly maintain test cases).

[0113] Meanwhile, when running test scripts, the corresponding timing rules can be matched according to the data characteristics of the test cases. The timing rules mainly include two triggering methods to cover different scenarios: event triggering and periodic triggering.

[0114] S420: When the matching timing rule is event-triggered, if the test cases and / or test scripts in the code management platform are updated, the test scripts are executed and the test results are obtained based on the priority identifier in the corresponding interface document.

[0115] In event-triggered events, scripts / test cases run automatically after each update and execute tests to quickly verify whether the new interface is working correctly.

[0116] Among them, the events corresponding to the event triggers include update events led by testers, where testers modify existing test cases and test scripts and submit the updated test cases and scripts to the code management platform; and update events associated with developers, where developers submit scripts related to new interfaces (such as basic call scripts for new interfaces) to the code management platform. Although this update behavior is initiated by the developers, it belongs to the scenario of adding test scripts and is also classified as a test script update event.

[0117] When the script is updated, the changed files are committed to the specified code repository via Git commands. It is specified that "when a specific branch of the code repository is updated, an HTTP request message will be automatically sent to the preset interface". After receiving the trigger message, the preset interface automated test task will be automatically started and the test will be executed based on the framework requirements.

[0118] At the same time, when executing scripts, they need to be performed in order of priority.

[0119] S430: When the matched timing rule is periodic triggering, the test script is executed periodically based on the periodic time and the priority identifier in the corresponding interface document, and the test results are obtained.

[0120] When the corresponding period is triggered, the test script is executed based on the preset period time, such as automatic execution every week or every month. Furthermore, it can also be run during off-peak business periods according to the server load.

[0121] At the same time, when running test scripts through periodic triggering, they also need to be executed in order of priority.

[0122] S440 generates an interface automation report based on the results of each or several tests. The interface automation report includes basic exceptions, time-consuming exceptions, and interface success rate.

[0123] Generate visual reports that include success rate, time distribution, and reasons for failure through the integrated Allurev reporting tool.

[0124] In other embodiments, an automated interface report is generated based on the results of one or more tests, specifically including:

[0125] S441 determines the test cases to be executed this time based on priority identifiers and skip identifiers, and counts the total number of test cases.

[0126] First, test cases that are not executed are filtered out based on the skip flag. These test cases are not actually executed and will not be included in the success rate statistics. Then, the number of test cases that need to be executed in this test is determined based on the priority flag to calculate the total number of test cases.

[0127] S442 determines whether the assertion parameters of the test script match the expected results under different scenario-based use case requirements. If they match, the test is considered successful; if they do not match, the test is considered to have failed and is defined as a basic exception.

[0128] The analysis checks whether the assertion parameters in the test script match the expected results. The comparison of assertion parameters includes basic assertions and core assertions.

[0129] Under the basic assertion, perform response code matching: verify that the HTTP response code returned by the interface is consistent with the expectation. For example, if the interface documentation specifies that a 400 error is returned, then the test case assertion needs to verify whether the response code is 400 (abnormal parameter). If the response code does not match, the test case fails.

[0130] Under the core assertion, the response body parameter matching is performed: verify that the key fields of the response body (such as business status and return parameters) are consistent with the expectations. For example, if the expected response body of the valid parameter test case of the payment interface contains "success":true and "tradeNo" (transaction number), then the script assertion needs to verify whether "the response body contains "success":true" and "tradeNo is not empty". If either condition is not met, the test case fails.

[0131] When the assertion parameters match, the test is considered successful and recorded as the number of successful tests. If they do not match, the test is considered a failure and the test is defined as a basic exception.

[0132] S443: Obtain the execution time corresponding to the test script and calculate the difference between it and the expected execution time in the previous stable run result. If the difference is not greater than the preset time, the test is considered successful. If the difference is greater than the preset time, the test is considered to have failed and is defined as an abnormal execution time.

[0133] At the same time, we need to pay attention not only to whether the interface runs successfully, but also to the time it takes.

[0134] First, the time taken for a single script test of an interface cannot exceed a threshold. If it does, the script is considered to have failed and is recorded as a time-consuming exception.

[0135] On the other hand, when the time taken in a single execution does not exceed the threshold, it is necessary to further compare whether the time taken in this execution is significantly different from the time taken in the previous stable execution. If the difference is significant, the test will also be considered a failure and defined as an abnormal time taken.

[0136] It should be noted that the time consumption in this embodiment is represented as the time period starting from the moment the test script sends a request to the target interface through an HTTP tool (such as HttpClient) encapsulated by the framework, and ending at the moment when the script fully receives the response data returned by the interface and completes basic parsing.

[0137] S444 calculates the interface success rate based on the ratio of the total number of exceptions to the total number of test cases.

[0138] The test success rate is represented by the result of dividing the number of test cases that were actually executed successfully by the total number of test cases that were actually executed.

[0139] In other embodiments, existing automated testing frameworks only focus on the success rate of test scripts and test cases in common technical items and basic scenarios, but ignore the misjudgment of test results caused by occasional failures of the interface (such as request timeouts due to network fluctuations). Therefore, when running test scripts based on test cases and matching timing rules, the following steps are also included:

[0140] S450 retrieves tiered retry rules from the interface automation framework. Tiered retry rules include immediate retries and timed retries.

[0141] S460: When a basic exception exists, the assertion parameters are retrieved to extract the cause of the exception, and the corresponding hierarchical retry rule is matched based on the cause of the exception.

[0142] S470: When corresponding to immediate retry, the test script is retried at the first time interval until the preset number of times. If the retry is successful, the basic exception is updated to occasional success, the corresponding total number of exceptions is reduced and recorded in the interface automation report.

[0143] S480: When a scheduled retry is performed, a corresponding scheduled task is set based on the priority identifier, and the test script is retried once when the scheduled task is triggered at the second time. The retry result is recorded in the interface automation report.

[0144] First, the tiered retry rules are configured as follows: the first tier is immediate retry, and the second tier is timed retry.

[0145] In real-time retries, if the script execution fails due to "network timeout, connection refused" in the response body, several quick retries will be performed automatically with short intervals, such as 2 quick retries at intervals of 3. If the retry is successful, it will be marked as an occasional success. This reduces the total number of exceptions to eliminate the impact of misjudgment. On the other hand, the occasional success information needs to be updated in the interface automation report, such as indicating in the report "the reason is network fluctuation, and it was successful after retry".

[0146] During scheduled retries, if the script fails to execute due to "interface returns 500, internal server failure", it will automatically trigger "retry in 10 minutes" to avoid temporary server failures affecting the test results. At the same time, the retry results and retry reasons will also be updated to the interface automation report.

[0147] In other embodiments, when a basic anomaly exists and immediate retry or timed retry fails, the following steps are also included:

[0148] S490: Obtain the test cases corresponding to the basic exceptions and create a development work order.

[0149] S491: Assign the development work order and wait for feedback on the repair of the development interface, then update the status label of the development work order to the trigger label.

[0150] S492 initiates an immediate retry based on the trigger tag and waits for the retry result. If the retry result is successful, the status tag is updated to the verification tag and recorded in the interface automation report. If the retry result fails, the development work order is reassigned to the development interface.

[0151] When test failures occur, existing solutions can often only indicate the cause of the failure, but cannot specify who will fix it, when to fix it, or how to verify it after the fix, which can easily lead to failed test cases being shelved for a long time.

[0152] Therefore, in this embodiment of the application, a management module such as Jira is configured to automatically create a repair work order when there are abnormal test cases that fail to retry. The processing logic of "failed test case work order creation" is added to the corresponding test script. If the test fails, the basic information corresponding to the failed test case is retrieved, such as the interface name, failure reason, and the link of the interface automation report. The repair work order is created and assigned to the developer by calling the JIRA API. The assignment logic can adopt strategies such as point-to-point assignment (assigning to the developer) and proximity assignment (selecting an idle development port).

[0153] Meanwhile, after the developers fix the failed test cases, they will update the status of the fix ticket to "fixed" and can also add a note about the fix version.

[0154] The test cases corresponding to the repaired work order are re-entered into the testing phase to trigger the immediate retry phase of the rapid response, and wait for the rerun result. If the rerun result is successful, the test cases corresponding to the repaired work order are updated to the "verified" status. They still correspond to one exception total, but will be used as actual usable test cases in the future.

[0155] If the retry of the repaired test case still fails, the reason for the failure will be generated and packaged into the repair work order to be sent back to the development interface to form a closed loop.

[0156] In other embodiments, generating an automated interface report based on the results of each or several tests further includes the following steps:

[0157] S445, set up response body tracking points. When test results are obtained, capture the complete response body based on the response body tracking points and associate it with the test case ID to generate a query index to the log file.

[0158] S446, when the test result corresponds to a basic exception or a time-consuming exception, the query index is used to query and retrieve the corresponding log file.

[0159] In existing technologies, when generating interface automation reports, the reports will indicate the reasons for test failures. However, they do not include the specific response body details and server logs associated with the test failure. Testers need to manually find the response body and log in to the server to view the corresponding operation logs for the error.

[0160] In this embodiment of the application, a response body log point is added to the interface automation framework. Regardless of whether the test case succeeds or fails, the complete response body will be automatically captured and saved while the test script is running, and it will be bound to the test case ID and stored in the log file.

[0161] When testing system platform interfaces using the aforementioned interface automation framework, and interfacing with the corresponding platform, if a test case fails during testing, the framework automatically queries the platform's server logs by looking up the index (test case ID + interface path + request time), thus eliminating the need for manual cross-platform log viewing by personnel.

[0162] This application also discloses an automated interface testing system for implementing the above-described method.

[0163] The implementation principle is as follows:

[0164] The core technology stack of the framework is built around four major stages: "framework support, interface acquisition, test case generation, and execution feedback." It utilizes recording tools to automatically process interface information throughout the entire process, avoiding the omission of key parameters when manually compiling interface data. A mature large language model is introduced, leveraging its ability to analyze interface document parameter rules and response logic to automatically generate test cases and scripts that conform to the framework template. This solves the problems of low efficiency and uneven scenario coverage in manual test case design. At the continuous execution and result feedback level, an automated mechanism based on the framework's timed rules constructs a closed loop of "code submission - test execution - report generation," replacing the inefficient traditional manual test triggering model.

[0165] It should be understood that although the steps in the flowcharts in the accompanying drawings are shown sequentially as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise expressly stated herein, there is no strict order in which these steps are performed, and they may be performed in other orders.

[0166] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. An interface automation testing method, characterized in that, The method comprises the following steps: building an interface automation framework; calling an agent pointing tool and selecting an interface, recording the interface, generating full-link interface information based on the recording results, and confirming parameter information to form an interface document, calling state identifiers including abandonment identifiers and priority identifiers from the interface automation framework, and configuring the corresponding abandonment identifiers and priority identifiers for the interface document based on the interface automation framework; analyzing the interface document based on a large language model to obtain test cases and test scripts that conform to the predetermined template style in the interface automation framework, generating test identifiers based on the corresponding state identifiers of the interface document, and capturing the priority identifiers and abandonment identifiers in the interface document, wherein the priority identifiers will be directly used for inheritance binding with the generated test cases, and the abandonment identifiers are used to determine whether the test scripts generated thereafter need to be executed, if the interface document contains the abandonment identifier, a skip identifier is added to the test case, and when the test case is executed, the test script corresponding to the test case is skipped; Specifically; calling test case styles and test script templates in the interface automation framework; configuring scenario-based case requirements, which include valid parameter scenarios, boundary value scenarios, special character scenarios, and parameter missing scenarios; extracting and combining the contents of the interface document according to the scenario-based case requirements to generate test cases that conform to the test case styles, and all test cases inherit the priority identifiers in the interface document; extracting request information and expected response results in the test cases to generate test scripts that conform to the test script templates; running the test scripts based on the test cases matching timing rules, and generating an interface automation report, which includes basic exceptions that do not conform to the expected response results, time-consuming exceptions with a large difference between the actual time-consuming and the expected time-consuming of the last stable running result, and interface running success rate; The method further comprises the following steps: obtaining layered retry rules in the interface automation framework, including immediate retry and timed retry; when the basic exception exists, calling assertion parameters to extract the exception reason, and matching the corresponding layered retry rules based on the exception reason; when immediate retry is required, the test script is executed every first time interval until a preset number of times, if the retry is successful, the basic exception is updated to an occasional success, the corresponding total number of exceptions is reduced, and the interface automation report is recorded; when timed retry is required, set the corresponding timing task based on the priority identifier, and retry the test script once at the second time of the timing task, and record the retry result to the interface automation report; if the basic exception exists and does not pass the immediate retry or the timed retry, the method further comprises the following steps: acquiring the test case corresponding to the basic exception and creating a development work order; assigning the development work order and waiting for the repair feedback of the development interface, updating the status label of the development work order to a trigger label; starting the instant retry based on the trigger label and waiting for the retry result, if the retry result is successful, updating the status label to a verification label and recording to the interface automation report, if the retry result fails, reassigning the development work order to the development interface.

2. The interface automation testing method of claim 1, wherein, calling a proxy pointing tool and selecting an interface, recording the interface and generating full-link interface information based on the recording result to form an interface document, calling the corresponding state identifier from the interface automation framework, including the following steps: The interface information at least includes interface path, interface request header, interface request body, response header and response body; Confirming the fillable parameters and required parameters corresponding to the interface request body, determining the return parameter information of the response body under different states to generate an interface document; Configuring the discard identifier and priority identifier for the interface document based on the interface automation framework.

3. The interface automation testing method of claim 2, wherein, Generating a test identifier based on the state identifier corresponding to the interface document, including the following steps: Determine whether the discard identifier exists in the interface document; If it exists, add a skip identifier to the test case, and when the test case is executed, filter out the test case with the skip identifier.

4. The interface automation testing method of claim 3, wherein, Based on the test case, match the timing rule to run the test script, and generate an interface automation report, including the following steps: Update the test case and the test script to the code management platform; When the matching timing rule is event triggered, if the test case and / or the test script in the code management platform are updated, execute the test script based on the priority identifier in the corresponding interface document and obtain the test result; When the matching timing rule is periodic triggering, execute the test script periodically based on the periodic time and the priority identifier in the corresponding interface document and obtain the test result; Generate the interface automation report based on the test result of each time or several times.

5. The interface automation testing method of claim 4, wherein, Based on the test result of each time or several times, the interface automation report is generated, which specifically further includes: Determine the test case to be executed this time based on the priority identifier and the skip identifier, and count the total number of cases; Determine whether the assertion parameters of the test script match the expected results under different scenario requirements, if they match, the test is successful, if they do not match, the test is failed and defined as a basic exception; Obtain the execution time corresponding to the test script, and calculate the difference value with the expected time in the last stable running result, if the difference value is not greater than the preset time length, the test is successful, if the difference value is greater than the preset time length, the test is failed and defined as a time-consuming exception; Calculate the interface running success rate based on the ratio of the total number of exceptions to the total number of cases.

6. The interface automation testing method of claim 5, wherein, Based on the test result of each time or several times, the interface automation report is generated, which further includes the following steps: setting a response body burying point, capturing a complete response body based on the response body burying point and associating a use case ID to generate a query index to a log file when the test result is obtained; when the test result corresponds to a basic exception or a time-consuming exception, the query index is used to query and obtain the corresponding log file.

7. An interface automation test system, characterized by, for implementing the method as claimed in any one of claims 1 to 6.

Images