An automated testing system and method for server interfaces
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 百信信息技术有限公司
- Filing Date
- 2026-03-06
- Publication Date
- 2026-06-30
Smart Images
Figure CN122309352A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of software automated testing technology, specifically a server interface automated testing system and method. Background Technology
[0002] In the software development lifecycle, server interface testing is a crucial step in verifying system functionality, performance, and stability. Currently, common interface testing methods primarily rely on testers manually writing test scripts or using automated testing tools to compare fixed request parameters with expected responses based on interface documentation. These methods require testers to have a deep understanding of the business logic and design test cases for each interface individually. For complex microservice architectures, with numerous interface calls and data dependencies between services, manually maintaining test cases is extremely labor-intensive and struggles to cover complex scenarios involving combined interface calls.
[0003] Existing technical solutions typically treat each interface as an independent unit for testing, with test case construction relying on the interface's input / output specifications or the tester's pre-set parameters. This approach has drawbacks: it struggles to dynamically and accurately reflect the actual logical call relationships at the code level, especially when business code is frequently updated; the lack of synchronization between documentation and code can lead to invalid or incomplete test cases. Furthermore, the generated test data is often isolated and based on individual interface contracts, lacking a coherent framework from the perspective of the overall business process. This makes it difficult to effectively simulate real, continuous end-to-end user operation scenarios, hindering the discovery of deep-seated defects caused by specific call sequences or data transfer between interfaces. Summary of the Invention
[0004] This invention aims to solve at least one of the technical problems existing in the prior art;
[0005] Therefore, this invention proposes an automatic testing method for server interfaces, comprising:
[0006] The test task configuration unit receives externally input server interface test configuration instructions, which include address information of the set of interfaces to be tested, expected response data samples, and parameters for setting the scale of concurrent requests.
[0007] In response to the server interface test configuration command, the latest version of the business logic code associated with the set of interfaces to be tested is obtained from the server version management library, and the code dependency analysis tool is called to perform syntax parsing on the latest version of the business logic code to generate an interface function call relationship topology diagram.
[0008] Based on the scale of the concurrent requests, parameters are set to dynamically schedule and create multiple logically isolated test execution containers in the test resource pool, and the runtime environment of the interface test framework is injected into each test execution container.
[0009] Based on the interface function call relationship topology and the expected response data sample, the test case generation engine automatically constructs an independent test request message sequence for each interface to be tested. The independent test request message sequence contains multiple request data packets covering different input conditions.
[0010] The independent test request message sequence is distributed to the multiple logically isolated test execution containers according to a preset scheduling strategy, and each test execution container is driven to send the request data packet in the independent test request message sequence to the interface to be tested assigned to it.
[0011] Furthermore, a code dependency analysis tool is invoked to perform syntax parsing on the latest version of the business logic code, generating an interface function call relationship topology diagram, including:
[0012] The code dependency analysis tool is used to traverse the latest version of the source code directory of the business logic code and identify all function definition nodes marked as interface entry points;
[0013] Lexical analysis is performed on the function body of each function definition node to extract the names of internal service functions called within the function body, database operation statements, and call links to external interfaces.
[0014] Establish a directed call chain starting from the function definition node, with the internal service function name, database operation statement, and call link to external interface as intermediate nodes or endpoints;
[0015] The directed call chains of all function definition nodes are summarized, and duplicate nodes and chains are merged to form a global interface function call relationship topology graph. Each node in the interface function call relationship topology graph is accompanied by a code location identifier.
[0016] Furthermore, based on the interface function call relationship topology diagram and the expected response data sample, the test case generation engine automatically constructs an independent test request message sequence for each interface under test, including:
[0017] Extract a node of the interface to be tested from the interface function call relationship topology graph, and trace back its input parameter data structure definition based on the code position identifier of the node of the interface to be tested;
[0018] From the expected response data sample, match the standard response data format corresponding to the interface node under test;
[0019] Based on the input parameter data structure definition, a parameter filling template that satisfies the data type constraints but has random content values is generated, and a test assertion template is defined based on the standard response data format.
[0020] By combining boundary value generation strategy and abnormal input strategy, multiple different sets of parameter value combinations are created, and each set of parameter value combinations is filled into the parameter filling template to form a single test request data packet;
[0021] The individual test request data packet is bound to the corresponding test assertion template as a test case. All test cases generated for the interface node under test are arranged in a preset order to form the independent test request message sequence for the interface under test.
[0022] Further, the independent test request message sequence is distributed to the multiple logically isolated test execution containers according to a preset scheduling strategy, including:
[0023] Read the number of test execution containers and the resource quota information for each test execution container;
[0024] Based on the dependency relationships between nodes in the interface function call relationship topology diagram, the sequence of independent test request messages is sorted topologically to ensure that the test cases of the dependent interfaces are executed before the test cases of the interfaces that depend on them.
[0025] The test case queue, which has been topologically sorted, is divided into multiple sub-task packages according to the load balancing algorithm. Each sub-task package contains multiple test cases, either consecutive or non-consecutive.
[0026] Each subtask package is bound to a test execution container, and the task execution instruction and the corresponding subtask package data are sent to the bound test execution container.
[0027] Monitor the task status of each test execution container, and when it is detected that a test execution container has completed its sub-task package ahead of schedule, allocate a new test case to the test execution container from the unallocated task queue.
[0028] Furthermore, it also includes steps for collecting test results and comparing differences:
[0029] While each test execution container executes its assigned test cases, the actual response data packets returned by the interface of the server under test are captured in real time.
[0030] The captured actual response data packet is compared in real time with the test assertion template bound in the independent test request message sequence;
[0031] Record the comparison results. If the actual response data packet satisfies all the assertion conditions defined in the test assertion template, then mark the test case as passed; otherwise, mark it as failed and record the specific content of the actual response data packet.
[0032] The test case records marked as failing from all test execution containers are aggregated, and their corresponding request data packets, expected assertions and actual response data are extracted to generate a detailed report of test failure cases.
[0033] By associating the request data packets in the detailed report of the test failure cases with the latest version of the business logic code, the error can be located in the line of code or function definition node that caused the error.
[0034] Furthermore, the request data packets in the detailed report of the test failure cases are correlated with the latest version of the business logic code to trace back to the line of code or function definition node that caused the error, including:
[0035] The target interface address and call parameters are parsed from the request data packets in the detailed report of the test failure cases.
[0036] Based on the target interface address, locate the corresponding function definition node in the interface function call relationship topology diagram;
[0037] Based on the code location identifier of the function definition node, the specific source code file and function entry point are located in the latest version of the business logic code;
[0038] Using code instrumentation tools, dynamic trace probes are implanted at the source code file and function entry points, and a separate debugging request is re-initiated using the call parameters that caused the failure.
[0039] The dynamic tracking probe captures snapshots of internal variables and execution paths during function execution, up to the point where an exception occurs, thereby pinpointing the specific line of code causing the error.
[0040] Furthermore, it also includes regression comparison analysis of historical test data:
[0041] Establish a test case historical execution database to store the request data packets, expected assertions, actual response data, and execution results for each test case in each automated test.
[0042] When a new automated test task is started, historical test cases that intersect with the current set of interfaces to be tested are retrieved from the test case history execution database.
[0043] The actual response data of the retrieved historical test cases is used as a benchmark, and compared field by field with the newly obtained actual response data of the same test cases in this test.
[0044] Calculate the difference index between the new and old actual response data, and mark historical test cases whose difference index exceeds the allowable threshold;
[0045] The marked cases and their differences are correlated with the server code change logs corresponding to this test by time correlation analysis, and a regression test impact analysis report is generated.
[0046] Furthermore, the marked cases and their differences are correlated with the server code change logs corresponding to this test through time-related analysis to generate a regression test impact analysis report, including:
[0047] Retrieve all code commit records submitted to the server version management repository from the last successful test to the current test, forming the server code change log;
[0048] Analyze the server code change log to extract the file paths and function names modified in each code commit;
[0049] The interfaces associated with the marked cases are mapped to specific function names through the interface function call relationship topology diagram;
[0050] Compare the function names associated with the marked cases with the function names modified in the code commit records to find the code commits that have overlap.
[0051] The report integrates overlapping code commit records, corresponding code modifications, and detailed reports of associated test failure cases, organizing them in the order of code commits to form the regression test impact analysis report. The report clearly identifies code changes that are suspected of causing regression problems.
[0052] Furthermore, it also includes the automatic isolation and cleanup of the test environment and test data:
[0053] A dedicated virtual network namespace and temporary file storage volume are created for each automated test task in the test resource pool;
[0054] The multiple logically isolated test execution containers created by the dynamic scheduling are deployed in the dedicated virtual network namespace;
[0055] All temporary data and log files generated during the execution of all test cases are written to the aforementioned temporary file storage volume;
[0056] Once all test cases for this automated test task have been executed and the test results have been collected, the environment cleanup process will be triggered.
[0057] The environment cleanup process is forcibly terminated and all test execution containers created in this task are deleted. Subsequently, the dedicated virtual network namespace is released and the temporary file storage volume is emptied.
[0058] Furthermore, the present invention also includes an automatic server interface testing system, the system comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor, when executing the computer program, implements the steps of the automatic server interface testing method described above.
[0059] Compared with the prior art, the beneficial effects of the present invention are:
[0060] By using code dependency analysis tools to parse the business logic code, a topology diagram of interface function call relationships is generated. This technology works directly on the source code, enabling the static extraction and structured presentation of the actual call chains and data dependencies between interfaces. This process shifts the testing focus from potentially outdated or missing documentation to the code itself, ensuring precise synchronization between the test model and the system implementation logic. When business code is updated, the system can automatically reconstruct the topology, providing immediate feedback on the scope and path of the changes' impact. This allows testing activities to closely follow development iterations, enabling rapid and accurate assessment and analysis of the impact of changes.
[0061] Based on the generated interface function call topology and expected response data samples, the test case generation engine automatically constructs independent test request message sequences covering different input conditions. This technology guides the combination of test data and the orchestration of requests based on the actual logical relationships between interfaces, and the generated test sequences inherently simulate the execution flow of business functions. This enables automated testing to go beyond the functional verification of isolated interfaces, constructing continuous test scenarios with context state propagation and data dependencies, thereby effectively detecting integration logic defects that are only exposed during cross-interface calls, specific execution sequences, or abnormal data flow. Attached Figure Description
[0062] Figure 1 This is a flowchart illustrating the steps of an automatic server interface testing method according to the present invention.
[0063] Figure 2 A flowchart generated from the topology diagram of interface function call relationships;
[0064] Figure 3 A flowchart for constructing a sequence of independent test request messages;
[0065] Figure 4 A dual-axis trend monitoring chart showing the pass rate and number of failed test cases for interface tests;
[0066] Figure 5 A chart analyzing the average time and error distribution at each stage of environmental cleanup. Detailed Implementation
[0067] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0068] See Figure 1 The test task configuration unit receives external server interface test configuration instructions, which include address information of the set of interfaces under test, expected response data samples, and concurrent request scale settings. In response to these configuration instructions, the system retrieves the latest version of the business logic code associated with the set of interfaces under test from the server version management library and calls a code dependency analysis tool to perform syntax parsing on this latest version of the business logic code, thereby generating an interface function call relationship topology diagram. Based on the concurrent request scale settings, the system dynamically schedules and creates multiple logically isolated test execution containers in the test resource pool and injects the interface test framework's runtime environment into each test execution container. The test case generation engine automatically constructs an independent test request message sequence for each interface under test based on the interface function call relationship topology diagram and expected response data samples. This sequence contains multiple request data packets covering different input conditions. The system distributes the independent test request message sequence to multiple logically isolated test execution containers according to a preset scheduling strategy, driving each test execution container to send the request data packets from the independent test request message sequence to its assigned interface under test.
[0069] See Figure 2 In one embodiment of the present invention, the step of generating the interface function call relationship topology graph includes source code scanning and node identification. A code dependency analysis tool performs a full traversal of the source code directory containing the latest version of the business logic code. The scanning rules of the code dependency analysis tool identify public methods marked with specific annotations in the source code files and identify these methods as function definition nodes. The information of the function definition node includes the method name, the full path of its class, and its start and end line numbers in the source code file.
[0070] In some embodiments, lexical analysis and syntax parsing are performed on the function body of each identified function definition node. The parsing process extracts the names of other methods explicitly called within the function body as internal service function names, extracts string sequences starting with specific SQL keywords as database operation statements, and extracts remote call links initiated through HTTP clients or RPC frameworks as call links to external interfaces. The extracted internal service function names, database operation statements, and call links to external interfaces are recorded as call targets.
[0071] It can be understood that the process of establishing a directed call chain starts with a function definition node as the starting node, and uses the extracted internal service function names, database operation statements, and calls to external interfaces as target nodes. A directed edge is established between the starting node and each target node, thus forming a directed call chain from the interface entry point to its internal dependencies. For an internal service function name, if it is itself an identified function definition node, then that node in the topology graph is both the starting point of other chains and the end point of the current chain.
[0072] In practice, the aggregation and merging operations aggregate the directed call chains generated by all function definition nodes. During the aggregation process, nodes with names and code location identifiers that are exactly the same are considered duplicate nodes and merged. Similarly, directed edges with the same start node and target node are considered duplicate links and merged. The merged nodes and edges form a global directed graph structure, namely the interface function call relationship topology graph. Each node in the interface function call relationship topology graph is associated with a code location identifier, which uniquely points to a specific location in the source code. Its format follows the convention of "file path:start line number-end line number".
[0073] In some embodiments, to evaluate the structural complexity and potential test concerns of function definition nodes, a static complexity weight can be calculated for each node in the interface function call relationship topology graph. The calculation of the static complexity weight considers the number of lines of code in the function body corresponding to the node, the number of conditional branches within the function body, and the total number of target nodes called by the function body. Static Complexity Weight The calculation formula is as follows:
[0074] Where: characters The number of lines of code representing the function body, characters Represents the number of conditional branch statements within the function body, in characters. This represents the name of the internal service function called by the function body, the database operation statements, and the total number of target nodes for calls to external interfaces, in characters. , , It is a predefined adjustment coefficient used to balance the influence of different factors on the complexity weight. This static complexity weight value can be attached to the attribute of the corresponding node.
[0075] The generated API call topology graph is a data structure containing a set of nodes and a set of edges. Each element in the node set stores a type identifier, a code location identifier, a node name, and an optional static complexity weight. Each element in the edge set stores the starting node index, the target node index, and the edge type. The API call topology graph is serialized and stored in a standard graph data format for use by the subsequent test case generation and scheduling module.
[0076] See Figure 3 In one embodiment of the present invention, the test case generation engine generates an independent test request message sequence based on the interface function call relationship topology diagram and the expected response data sample. The test case generation engine extracts a test interface node from the interface function call relationship topology diagram. This test interface node has a code location identifier attribute. Based on the code location identifier attribute of the test interface node, the test case generation engine locates the corresponding function definition in the source code, thereby tracing back to the complete input parameter data structure definition of the function. The input parameter data structure definition includes the parameter name, data type, whether it is required, default value, and value range constraints. The test case generation engine retrieves data from the expected response data sample. The expected response data sample is a structured data collection that stores the mapping relationship between interface paths and standard response data formats. The test case generation engine obtains the corresponding standard response data format by matching the interface path mapped to the test interface node with the keys in the expected response data sample. The standard response data format defines the fields that the response body should contain, their data types, and nesting relationships.
[0077] In some embodiments, the test case generation engine generates a parameter filling template based on the input parameter data structure definition. The parameter filling template is a data template completely isomorphic to the input parameter structure, with its field types strictly consistent with the input parameter data structure definition. However, the content values of string type fields are replaced with random strings conforming to length and character set rules, and the content values of numeric type fields are replaced with random numeric values conforming to the data type within the defined range. The test case generation engine defines a test assertion template based on the standard response data format. The test assertion template consists of a series of assertion rules, including verifying the response status code, verifying the integrity of the response body structure, and verifying the data type and value range of specific fields.
[0078] Understandably, the test case generation engine combines boundary value generation and exception input strategies to create multiple different combinations of parameter values. The boundary value generation strategy generates regular values (equal to the minimum, slightly less than the minimum, equal to the maximum, slightly greater than the maximum, and within the normal range) for fields with clearly defined value ranges or length constraints in the input parameter data structure definition. The exception input strategy generates strings that do not conform to the data type definition, values exceeding the extreme values of the constraint range, null values, or parameter values that are intentionally missing or have deliberately omitted required fields. The test case generation engine then fills each combination of parameter values into the corresponding fields of the parameter filling template, thus forming a single test request data packet with a complete structure and containing specific parameter values.
[0079] In practice, a single test request data packet is bound to a corresponding test assertion template. This binding operation generates a test case record, which simultaneously stores the address of the interface under test, the request method, the request header information, the single test request data packet, and the test assertion template. The test case generation engine generates all preset combinations of test case records for the currently processed interface node under test. The test case generation engine arranges the test case records in a preset order, which can be ascending order of parameter combination complexity, boundary value priority, or random order. The sequence of test case records after arrangement constitutes an independent test request message sequence for the current interface under test. The independent test request message sequence is stored in memory or persistent storage as a list, and each element in the list is an independent test case record.
[0080] In some embodiments, the number of test cases generated for the interface node under test is determined by a combination factor. Control, Combination Factor The calculation is related to the number of boundary conditions and outliers of the input parameters. Combination factor The calculation formula is as follows:
[0081] Where: characters Represents input parameters with well-defined boundary values, characters Representing the The number of boundary values for each parameter, character Representative regarding the first The number of additional special values for each parameter. Character This represents the number of independent parameter combinations applicable to the abnormal input strategy, character Representing the The number of outliers that can be selected from the group of outlier parameter combinations. The product term is used to estimate the total number of multi-parameter outlier combinations. The test case generation engine uses the combination factor. The scale control limits the maximum number of test case records generated for each interface under test.
[0082] The scheduling module reads the total number of test execution containers. The resource quota information for each test execution container includes the number of CPU cores allocated, the maximum memory allocation, and the reserved network bandwidth. The scheduling module performs topological sorting on the sequence of independent test request messages based on the dependencies between nodes in the interface function call relationship topology graph. The topology sorting algorithm identifies the interface paths requested in the test cases and searches for the corresponding nodes in the interface function call relationship topology graph. If a directed path exists from node A to node B, it means that the interface function corresponding to node B depends on the interface function corresponding to node A. During sorting, it must be ensured that all test cases corresponding to node A are in the sequence before all test cases corresponding to node B. The sorting process generates a linear test case queue, which guarantees that test cases of dependent interfaces are executed before test cases of interfaces that depend on them.
[0083] It's understandable that the scheduling module divides the topology-sorted test case queue into multiple sub-task packages using a load balancing algorithm. The load balancing algorithm uses the resource quota information of the test execution containers as the weighting basis; test execution containers with higher resource quotas are allocated more test cases. This splitting operation generates... Each subtask package contains multiple test case records, either consecutively or non-consecutively, from the test case queue. Non-consecutive allocation allows for mixing test cases from different interfaces to achieve better resource utilization. The scheduling module binds each subtask package to a test execution container, recording the subtask package's identifier and the target test execution container's network address or identifier. The scheduling module sends a task execution command and the corresponding subtask package data to the bound test execution container. The task execution command contains the command to begin execution, and the subtask package data is a serialized list of test case records.
[0084] In implementation, the scheduling module monitors the task status of each test execution container. This monitoring is achieved by periodically sending status query requests to the test execution containers or receiving heartbeats and progress information proactively pushed by the containers. When the scheduling module detects that a test execution container has completed its assigned sub-task package ahead of schedule, it checks the unassigned task queue in the central task queue. This unassigned task queue contains all test case records that have not been assigned to any test execution container and are not currently being executed. The scheduling module selects one or more new test case records from the unassigned task queue and assigns them to idle test execution containers for continued execution. The allocation strategy can be sequential or dynamically calculated based on the current load of the test execution containers.
[0085] In one embodiment of the present invention, the steps of test result collection and difference comparison are performed synchronously with the task execution of the test execution container. After each test execution container sends a test request data packet to its assigned interface under test, it immediately starts the network listening and data capture module. The module captures the network data packets returned by the interface under test in real time. The captured network data packets are parsed into structured actual response data packets, which contain response status codes, response header information, and response body content. The assertion execution engine in the test execution container compares the captured actual response data packets with the test assertion templates bound to the currently executing test case records in real time. The comparison operation is performed one by one according to the assertion rules defined in the test assertion templates. For example, it checks whether the response status code of the actual response data packet is equal to the expected status code, parses the response body of the actual response data packet and verifies whether its JSON structure conforms to the predefined pattern, and checks whether the values of specific fields are within the numerical range specified by the assertion rules.
[0086] In some embodiments, the assertion execution engine records detailed results for each comparison. If the actual response data packet satisfies all assertion conditions defined in the test assertion template, the assertion execution engine marks the execution result of the current test case as passed and generates a log record containing a timestamp, case identifier, and "pass" status. If the actual response data packet fails to satisfy any one or more assertion conditions in the test assertion template, the assertion execution engine marks the execution result of the current test case as failed, and in addition to recording the timestamp and case identifier, the log record must also fully record the specific content of the actual response data packet that caused the assertion failure, including but not limited to the original response status code, response header information, and response body string. After completing the execution of all test cases within a subtask package, the test execution container summarizes all log records containing pass and fail markers and uploads them to a centralized test result collection server.
[0087] As you can understand, a centralized test result collection server receives result data from all test execution containers. The report generation module within this server aggregates and organizes all test case records marked as failing. The report generation module extracts key information from each failing test case record. This key information includes the entire content of the request data packet that triggered the test case, the entire content of the test assertion template bound to the test case, and the entire content of the recorded actual response data packet. The report generation module organizes this key information according to a predefined template, generating a detailed issue entry for each failing test case. All issue entries are then compiled into a structured detailed test failure report. This detailed test failure report is stored in the form of a document or database record, where each issue entry clearly links to the input, expected output, and actual output.
[0088] In practice, after the detailed report of test failure cases is generated, the correlation backtracking and localization process is immediately initiated. The diagnostic module parses the target interface address and call parameters from the problem entries in the detailed report of test failure cases. The target interface address is usually represented in the form of a Uniform Resource Identifier, and the call parameters are a set of key-value pairs extracted from the request data packet. Based on the target interface address, the diagnostic module performs a search operation in the pre-loaded interface function call relationship topology graph. The search operation matches the interface path and target interface address in the node attributes, thereby locating the corresponding function definition node. The diagnostic module obtains the code location identifier of the function definition node. The code location identifier points to the specific file path and line number range in the source code repository. Based on the code location identifier, the diagnostic module checks out the latest version of the business logic code in the local or remote server version management repository, and locates the specific source code file and function entry point in the latest version of the code repository.
[0089] In some embodiments, the diagnostic module uses a code instrumentation tool to implant dynamic tracing probes at the located source code files and function entry points. The code instrumentation tool can be a bytecode enhancement tool or a source code-level instrumentation tool. The implantation locations of the dynamic tracing probes include at least the function entry point, the decision points of all conditional branches within the function, the starting points of all loops within the function, and the positions before and after function calls to other internal service functions or database operation statements. After implantation, the diagnostic module uses the call parameters that caused the failure extracted from the detailed report of the test failure cases to reconstruct a debug request message that is completely identical to the original failure request. This debug request message is then sent to the interface of the server under test through an independent debug executor. The debug executor runs in single-threaded synchronous mode and does not involve concurrency or container scheduling.
[0090] The diagnostic module captures internal state information during function execution using implanted dynamic tracing probes. These probes record snapshots of internal variables at each execution point, including the current values of all local variables, input parameters, and key object attributes. Simultaneously, the probes record the execution path, a sequence of probe identifiers executed in order. Debugging requests continue until an uncaught exception interrupts execution or the function completes normally. The diagnostic module analyzes the captured internal variable snapshots and execution path. By comparing the state of the last successful internal variable snapshot before the exception point with the state of the exception point or assertion failure point, the module can pinpoint the specific line of code causing the actual response to deviate from expectations. This pinpointing can be achieved by analyzing whether the execution path deviates from the expected branch based on the standard response data format, or by analyzing whether the value of a key variable in the internal variable snapshot has undergone an illogical change after a specific line of code. The diagnostic module appends the pinpointed erroneous line of code, along with its context code, as the location result to the issue entry in the corresponding test failure case detailed report. The entire diagnostic process consumes [amount missing]. An approximate estimate can be made, and the estimation formula is shown below:
[0091] Where: characters This represents the base time overhead required for code instrumentation and dynamic tracing probe implantation processes, in characters. Represents the average time cost of executing a single line of code, in characters. Represents the total number of probes activated during the diagnostic process, characters Representing the The latency factor introduced by an activated probe during execution, which captures snapshots of internal variables and serializes the data.
[0092] In one embodiment of the present invention, regression comparison analysis of historical test data relies on a persistent test case historical execution database. After each automated test task is completed, the system stores detailed information of all test cases in that task into the test case historical execution database. The stored detailed information includes a unique identifier for each test case, the complete content of the request data packet, the complete content of the test assertion template, the complete content of the actual response data packet returned by the interface of the server under test, the execution result of the test case, and the timestamp of the test execution. The test case historical execution database is stored using a relational database or a document database, and an index based on the interface path and test case identifier is established to accelerate subsequent retrieval.
[0093] When a new automated test task is initiated, the system retrieves historical test cases from the test case history execution database that intersect with the current set of interfaces under test. The retrieval operation matches the path of the interface under test, identifying all test case records executed for the same interface path in historical tests. The system uses the actual response data of the retrieved historical test cases as baseline data, and simultaneously obtains newly acquired actual response data corresponding to the same interface and the same test case identifier from the execution results of the current test. The comparison between the old and new actual response data is performed at the field level. The system parses the response data into structured data objects, such as JSON objects, and recursively traverses all leaf node fields of the object for field-by-field comparison. The comparison operation not only compares whether the field values are strictly equal, but also calculates the percentage difference of numerical fields and the edit distance or similarity for string fields.
[0094] In some embodiments, the system calculates a difference index between the old and new actual response data for each comparison field. It is a normalized numerical value used to quantify the degree of deviation between the old and new versions of response data. For simple data types, the difference metric is... The results can be directly based on value comparisons; for complex objects or arrays, the difference index... It can be a measure of the difference between all its internal fields. The system performs weighted aggregation. It maintains an overall difference metric for each test case. This metric is typically a measure of the difference across all fields. The maximum or average value. The system will use the difference index. With a predefined allowable threshold When comparing, the overall difference index of test cases Exceeding the allowed threshold At that time, the system marks the test case as a suspected regression case. Allowable threshold. Different configurations can be made for different interfaces or field types. Information on flagged regression suspicious cases is stored in a temporary analysis list, which includes case identifiers, details of the differing fields, and the degree of difference index. The value.
[0095] Understandably, when starting a new automated test task or after a task completes, the system proactively retrieves the code change history from the server version control repository. The system queries all code commit records submitted to the server version control repository between the time of the last successful test and the start time of the current test. These commit records collectively constitute the server code change log. Each record in the server code change log includes a commit hash, commit author, commit time, commit information, and a detailed list of file modifications. The system analyzes the server code change log, parsing each commit to reveal the modified file paths and function names within those files. For functions that have been renamed or deleted, the system records their original names and final status.
[0096] In practice, the system maps the interface paths associated with marked suspected regression cases to specific function names through an interface function call relationship topology graph. This mapping process involves finding the function definition node corresponding to the interface path in the topology graph and reading the node's name attribute. The system then compares the list of function names associated with marked suspected regression cases with the list of function names modified in each code commit record extracted from the server code change log. The comparison operation uses exact string matching or containment relationship matching to find function names that intersect between the two lists. Code commit records with intersection are considered changes potentially related to the regression issue.
[0097] The system integrates overlapping code commit records, corresponding code modifications, and detailed reports of associated test failure cases. The integration process is organized according to the chronological order of code commits or logical connections. The system generates a regression test impact analysis report, which clearly identifies code changes suspected of causing regression issues. The report typically includes a summary table clearly listing each suspicious code commit, the affected interface or function, the associated suspected regression case identifier, and a brief description of the code modification. See Table 1 for an example report excerpt.
[0098] Table 1: Summary of Regression Test Impact Association Analysis
[0099]
[0100] In some embodiments, the process of generating a regression test impact analysis report may further include a risk assessment of the changes. The risk assessment calculates a risk score based on factors such as the scale of the code modification, the centrality of the modified functions in the interface function call relationship topology graph, and the frequency of historical modifications. Risk Score The calculation formula is as follows:
[0101] Where: characters This represents the ratio of the number of lines of code modified in this commit to the total number of lines of code, used to quantify the scale of the changes. (Character) This represents the sum of the out-degree and in-degree of the modified function node in the interface function call relationship topology graph, indicating the tightness of its connection. (Character) It is the eigenvector centrality value calculated based on graph theory, reflecting the global importance of a node in the topological graph. (Character) This represents the number of times the function node has been modified in recent times. Risk Score It will be added to the corresponding entry in the regression test impact analysis report to provide a quantitative reference for judging the likelihood of regression problems.
[0102] Understandably, the final regression test impact analysis report is presented in document or webpage format. In addition to the correlation analysis table, the report includes links to detailed code differences, specific data comparisons of failure responses, and suggested next steps, such as code review or targeted supplementary testing. Once generated, the system can automatically send the report to the relevant developers or testers.
[0103] See Figure 4 In the interface testing quality monitoring system, the dual-axis linkage trend of test pass rate and number of failure cases intuitively reflects the stability fluctuations of the interface service over the past 30 days. The left vertical axis represents the test pass rate (0.70–1.00 range), the right vertical axis corresponds to the number of failure cases (0–15 range), and the horizontal axis is the test date sequence. From the coupling relationship between the curve and the bar chart, we can observe the following: Negative correlation: The test pass rate (blue line) and the number of failure cases (pink bars) show a significant negative correlation overall. For example, on dates such as January 6th, January 15th, and January 21st, when the number of failure cases reaches its peak, the test pass rate simultaneously drops to a temporary low, reflecting the direct drag effect of failure cases on the overall pass rate. Fluctuation cycle characteristics: The pass rate curve shows periodic fluctuations in the 0.75–0.95 range, while the number of failure cases appears in a pulse-like manner in the discrete range of 2–15, indicating that the interface testing quality is driven by events such as code commits and dependency changes, exhibiting non-steady-state characteristics. Quality Inflection Point Identification: For example, a sharp drop in the pass rate to 0.75 on January 6th, coupled with a surge in the number of failed cases to 15, can be identified as a critical quality inflection point. This requires analysis of the regression test impact report to pinpoint the related code submissions. Conversely, on dates like January 3rd and January 28th, the pass rate rebounds to above 0.93, and the number of failed cases falls below 4, reflecting the effectiveness of targeted fixes or optimizations. Monitoring Value: The dual-axis trend chart provides real-time quality feedback for the interface automation testing system. It can be linked with the historical test case database, and through correlation analysis between the difference index and code change logs, closed-loop management from "phenomenon monitoring" to "root cause identification" can be achieved.
[0104] In one embodiment of the present invention, the automatic isolation and cleanup mechanism for the test environment and test data is activated during the initialization phase of each automatic test task. After receiving the test task configuration instruction, the resource manager creates a set of exclusive resource units for this task in the test resource pool. The resource manager calls the operating system interface or container orchestration platform interface of the underlying infrastructure to create a dedicated virtual network namespace. The virtual network namespace provides an independent network protocol stack for all network communication within this task, including independent network device interfaces, routing tables, firewall rules, and port number allocation space. The resource manager also creates a temporary file storage volume, which can be a memory-based temporary file system or a logical volume allocated based on shared storage, and mounts it to a mount point dedicated to this task.
[0105] The resource manager deploys all dynamically scheduled, logically isolated test execution containers within a dedicated virtual network namespace. This deployment ensures that each test execution container's network mode is set to use this newly created virtual network namespace. Upon startup, the internal network configurations of each test execution container, such as IP addresses and routing, are allocated and managed within this virtual network namespace. This ensures complete network layer isolation between test execution containers from different automated test tasks, preventing IP address and port conflicts or network traffic interference. All output data generated by the test execution containers during operation, including temporary log files generated by the testing tools, cached data downloaded during testing, and temporary files written during test case execution, are configured to be written to a specific subdirectory within the temporary file storage volume. Each test execution container has an independent subdirectory within the temporary file storage volume named after its container identifier for data storage.
[0106] In some embodiments, log files generated by the test execution container during test case execution are written to a temporary file storage volume following a uniform naming format. The log file naming format includes the container identifier, timestamp, and log level. Temporary data files, such as intermediate state files generated by the test framework or data snapshots created during test case execution, are also stored in the temporary file storage volume. The system ensures the correctness of data write locations by deploying a file monitoring agent within each test execution container. The file monitoring agent captures and redirects all file operations pointing to the default system temporary directory to the corresponding subdirectory of the temporary file storage volume. The system maintains a resource mapping table that records the virtual network namespace identifier created for this automated test task, the temporary file storage volume path, and the mapping between the identifiers and network addresses of all test execution containers.
[0107] Understandably, the system continuously monitors the execution status of all test cases in this automated testing task. When the system detects that all test cases confirmed by the test result collection server have been completed and all test result data has been collected, the system triggers an environment cleanup process. This environment cleanup process is a high-privilege background service. It reads the resource mapping table and, based on the list of test execution container identifiers recorded in the resource mapping table, sends forced termination and deletion commands to the container orchestration platform or container runtime. The container runtime first sends a termination signal to each test execution container and waits for a predetermined graceful stop time window. If the test execution container does not stop itself within the time window, the container runtime sends a forced termination signal. After the test execution container process stops, the container runtime deletes its corresponding container instance and releases the runtime resources occupied by the container.
[0108] In practice, after all test execution containers have been confirmed deleted, the environment cleanup process performs cleanup of network and storage resources. The cleanup process sends instructions to the operating system or network management service to deactivate the virtual network namespace dedicated to this task. This deactivation includes deleting all virtual network devices created within the virtual network namespace, clearing routing and firewall rules, and ultimately destroying the virtual network namespace instance. Subsequently, the cleanup process performs a cleanup operation on the temporary file storage volume. This cleanup operation traverses all files and directories under the temporary file storage volume's mount point and performs recursive deletion. To ensure data security, before deleting files, the cleanup process may optionally calculate the hash value of important log files and archive them to persistent storage. Archiving is only performed on diagnostic files marked as needing to be saved during the test result collection phase.
[0109] In some embodiments, the total resource consumption cost of automated testing tasks A model can be used to estimate the total resource utilization cost, thus aiding resource planning. The calculation formula is as follows:
[0110] Where: characters The unit price for resource usage in a virtual network namespace per unit of time, in characters. Represents the total lifespan of a virtual network namespace from creation to destruction. (Character) The unit cost of computation per unit time for a single central processing unit core, character Represents the total number of test execution containers created in this task, character. This represents the average runtime from the start of the first test execution container to the forced termination of the last test execution container. The unit price representing a unit of memory capacity per unit of time, character This represents the average memory allocation across all test execution containers. (Character) The unit price per unit capacity for temporary storage, characters This represents the maximum storage capacity occupied by the temporary file storage volume before it is emptied. This model quantifies the main computing resource costs consumed throughout the entire lifecycle of the environment, from environment creation and test execution to environment cleanup.
[0111] After completing all resource release and data clearing operations, the environment cleanup process removes the record for this task from the resource mapping table and sends a cleanup completion notification to the task scheduling system. The system maintains a cleanup audit log, which records in detail the environment creation and destruction time, resource types and quantities involved, and the status of the cleanup operation for each automated test task. The cleanup audit log is used to track resource usage and investigate potential resource leaks, ensuring the long-term stable operation of the test resource pool.
[0112] See Figure 5 In the implementation of the environment cleanup mechanism of the server interface automated testing system, the graph accurately quantifies the time consumption characteristics and data dispersion of each stage of the end-to-end cleanup process after the test task is completed. In the specific analysis, the execution link of the environment cleanup process is broken down into five core stages: container termination, container deletion, network namespace destruction, storage volume clearing, and audit log recording. The vertical axis presents the average time consumption of each stage in seconds, while the error bars intuitively represent the time fluctuation range under different test task loads (labeled as "mean ± standard deviation"). From a timing perspective, the average time for the container termination phase is 12±3 seconds. As the initial stage of the cleanup process, this stage experiences some time fluctuations due to the need to wait for the container's graceful shutdown window. The container deletion phase reduces the time to 8±2 seconds, making it the stage with the lowest time and best stability throughout the entire process, demonstrating the high efficiency of destroying instances during container runtime. The average time for the network namespace destruction phase is 15±4 seconds. Due to the involvement of multiple levels of operations such as deleting virtual network devices and clearing routing and firewall rules, the time consumption increases significantly and the fluctuation range widens. The storage volume clearing phase reaches the peak time of the entire process at 20±5 seconds. This stage requires recursively traversing and deleting all test data in the temporary storage volume, which is the core performance bottleneck of the cleanup process. The audit log recording phase reduces the time to 5±1 seconds, only completing the structured archiving of cleanup actions, with short time consumption and minimal fluctuations. The quantitative results provide key parameter support for the cost estimation model of the test resource pool. The average time taken to clear the storage volume is directly related to the calculation of the runtime of the test execution container. The standard deviation of the time taken in each stage reserves reasonable time redundancy for resource scheduling in the environment cleanup process, ensuring the stability of test environment isolation and cleanup, and avoiding resource pool leakage problems caused by cleanup timeout.
[0113] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.
Claims
1. A method for automatically testing server interfaces, characterized in that, The method includes: The test task configuration unit receives externally input server interface test configuration instructions, which include address information of the set of interfaces to be tested, expected response data samples, and parameters for setting the scale of concurrent requests. In response to the server interface test configuration command, the latest version of the business logic code associated with the set of interfaces to be tested is obtained from the server version management library, and the code dependency analysis tool is called to perform syntax parsing on the latest version of the business logic code to generate an interface function call relationship topology diagram. Based on the scale of the concurrent requests, parameters are set to dynamically schedule and create multiple logically isolated test execution containers in the test resource pool, and the runtime environment of the interface test framework is injected into each test execution container. Based on the interface function call relationship topology and the expected response data sample, the test case generation engine automatically constructs an independent test request message sequence for each interface to be tested. The independent test request message sequence contains multiple request data packets covering different input conditions. The independent test request message sequence is distributed to the multiple logically isolated test execution containers according to a preset scheduling strategy, and each test execution container is driven to send the request data packet in the independent test request message sequence to the interface to be tested assigned to it.
2. The automatic testing method for server interfaces according to claim 1, characterized in that, The code dependency analysis tool is invoked to perform syntax parsing on the latest version of the business logic code, generating an interface function call relationship topology diagram, including: The code dependency analysis tool is used to traverse the latest version of the source code directory of the business logic code and identify all function definition nodes marked as interface entry points; Lexical analysis is performed on the function body of each function definition node to extract the names of internal service functions called within the function body, database operation statements, and call links to external interfaces. Establish a directed call chain starting from the function definition node, with the internal service function name, database operation statement, and call link to external interface as intermediate nodes or endpoints; The directed call chains of all function definition nodes are summarized, and duplicate nodes and chains are merged to form a global interface function call relationship topology graph. Each node in the interface function call relationship topology graph is accompanied by a code location identifier.
3. The automatic testing method for server interfaces according to claim 2, characterized in that, Based on the interface function call relationship topology and the expected response data sample, the test case generation engine automatically constructs an independent test request message sequence for each interface under test, including: Extract a node of the interface to be tested from the interface function call relationship topology graph, and trace back its input parameter data structure definition based on the code position identifier of the node of the interface to be tested; From the expected response data sample, match the standard response data format corresponding to the interface node under test; Based on the input parameter data structure definition, a parameter filling template that satisfies the data type constraints but has random content values is generated, and a test assertion template is defined based on the standard response data format. By combining boundary value generation strategy and abnormal input strategy, multiple different sets of parameter value combinations are created, and each set of parameter value combinations is filled into the parameter filling template to form a single test request data packet; The individual test request data packet is bound to the corresponding test assertion template as a test case. All test cases generated for the interface node under test are arranged in a preset order to form the independent test request message sequence for the interface under test.
4. The automatic testing method for server interfaces according to claim 3, characterized in that, Distributing the independent test request message sequence to the multiple logically isolated test execution containers according to a preset scheduling strategy includes: Read the number of test execution containers and the resource quota information for each test execution container; Based on the dependency relationships between nodes in the interface function call relationship topology diagram, the sequence of independent test request messages is sorted topologically to ensure that the test cases of the dependent interfaces are executed before the test cases of the interfaces that depend on them. The test case queue, which has been topologically sorted, is divided into multiple sub-task packages according to the load balancing algorithm. Each sub-task package contains multiple test cases, either consecutive or non-consecutive. Each subtask package is bound to a test execution container, and the task execution instruction and the corresponding subtask package data are sent to the bound test execution container. Monitor the task status of each test execution container, and when it is detected that a test execution container has completed its sub-task package ahead of schedule, allocate a new test case to the test execution container from the unallocated task queue.
5. The automatic testing method for server interfaces according to claim 4, characterized in that, It also includes steps for collecting test results and comparing differences: While each test execution container executes its assigned test cases, the actual response data packets returned by the interface of the server under test are captured in real time. The captured actual response data packet is compared in real time with the test assertion template bound in the independent test request message sequence; Record the comparison results. If the actual response data packet satisfies all the assertion conditions defined in the test assertion template, then mark the test case as passed; otherwise, mark it as failed and record the specific content of the actual response data packet. The test case records marked as failing from all test execution containers are aggregated, and their corresponding request data packets, expected assertions and actual response data are extracted to generate a detailed report of test failure cases. By associating the request data packets in the detailed report of the test failure cases with the latest version of the business logic code, the error can be located in the line of code or function definition node that caused the error.
6. The automatic testing method for server interfaces according to claim 5, characterized in that, The request data packets in the detailed report of the test failure cases are correlated with the latest version of the business logic code to trace back to the line of code or function definition node that caused the error, including: The target interface address and call parameters are parsed from the request data packets in the detailed report of the test failure cases. Based on the target interface address, locate the corresponding function definition node in the interface function call relationship topology diagram; Based on the code location identifier of the function definition node, the specific source code file and function entry point are located in the latest version of the business logic code; Using code instrumentation tools, dynamic trace probes are implanted at the source code file and function entry points, and a separate debugging request is re-initiated using the call parameters that caused the failure. The dynamic tracking probe captures snapshots of internal variables and execution paths during function execution, up to the point where an exception occurs, thereby pinpointing the specific line of code causing the error.
7. The automatic testing method for server interfaces according to claim 6, characterized in that, It also includes regression comparison analysis of historical test data: Establish a test case historical execution database to store the request data packets, expected assertions, actual response data, and execution results for each test case in each automated test. When a new automated test task is started, historical test cases that intersect with the current set of interfaces to be tested are retrieved from the test case history execution database. The actual response data of the retrieved historical test cases is used as a benchmark, and compared field by field with the newly obtained actual response data of the same test cases in this test. Calculate the difference index between the new and old actual response data, and mark historical test cases whose difference index exceeds the allowable threshold; The marked cases and their differences are correlated with the server code change logs corresponding to this test by time correlation analysis, and a regression test impact analysis report is generated.
8. The automatic testing method for server interfaces according to claim 7, characterized in that, The process involves performing a time-correlation analysis on the marked cases and their differences, along with the corresponding server code change logs for this test, to generate a regression test impact analysis report, including: Retrieve all code commit records submitted to the server version management repository from the last successful test to the current test, forming the server code change log; Analyze the server code change log to extract the file paths and function names modified in each code commit; The interfaces associated with the marked cases are mapped to specific function names through the interface function call relationship topology diagram; Compare the function names associated with the marked cases with the function names modified in the code commit records to find the code commits that have overlap. The report integrates overlapping code commit records, corresponding code modifications, and detailed reports of associated test failure cases, organizing them in the order of code commits to form the regression test impact analysis report. The report clearly identifies code changes that are suspected of causing regression problems.
9. The automatic testing method for server interfaces according to claim 8, characterized in that, It also includes automatic isolation and cleanup of the test environment and test data: A dedicated virtual network namespace and temporary file storage volume are created for each automated test task in the test resource pool; The multiple logically isolated test execution containers created by the dynamic scheduling are deployed in the dedicated virtual network namespace; All temporary data and log files generated during the execution of all test cases are written to the aforementioned temporary file storage volume; Once all test cases for this automated test task have been executed and the test results have been collected, the environment cleanup process will be triggered. The environment cleanup process is forcibly terminated and all test execution containers created in this task are deleted. Subsequently, the dedicated virtual network namespace is released and the temporary file storage volume is emptied.
10. An automated server interface testing system, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the automatic server interface testing method as described in any one of claims 1 to 9.