System testing method and device, electronic equipment and storage medium

Abstract

The invention provides a system testing method and device, electronic equipment and a storage medium. The method comprises the steps that standard HTTP flow data generated when a target system runs and standard link tracking data associated with the standard HTTP flow data are acquired, the standard HTTP flow data comprise an HTTP request, and the standard link tracking data represent the processing track of the target system on the HTTP request; testing a to-be-tested system according to the HTTP request, and obtaining simulated HTTP flow data generated during operation of the to-be-tested system and simulated link tracking data associated with the simulated HTTP flow data; and determining a test result of the to-be-tested system according to the standard HTTP flow data, the standard link tracking data, the simulated HTTP flow data and the simulated link tracking data. According to the method, the performance of the to-be-tested system is tested based on the data of the HTTP flow data and the link tracking data, and the system testing efficiency and the testing result precision can be effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of software testing, and particularly relates to a system testing method and device, electronic equipment and storage medium. BACKGROUND

[0002] The system playback verification method is a software testing method, and the core of the system playback verification method is that real user network traffic (for example, HTTP requests and corresponding responses) in a production environment is captured and recorded, and the recorded HTTP requests are resent (that is, request playback) in a testing environment isolated from the production environment, and by comparing the original responses with new responses generated after the playback, it is verified whether the functions provided by the system after code iteration or environment change are correct.

[0003] The traditional system playback verification method mainly focuses on the comparison of HTTP requests and responses, and although most functional problems can be found, the internal calls in the distributed system, the dependency relationship between services, the link delay and the resource consumption and other deep-seated problems are often difficult to solve. With the popularity of the microservice architecture, single-point errors can spread through complex call chains. Therefore, the traditional system playback verification method cannot meet the high-precision and high-efficiency quality assurance requirements of the system. SUMMARY

[0004] The present application provides a system testing method, device, electronic equipment and storage medium to solve the defect that the traditional system playback verification method cannot meet the high-precision and high-efficiency quality assurance requirements of the system.

[0005] The present application provides a system testing method, comprising the following steps: Obtaining standard HTTP traffic data generated by a target system during running and standard link tracking data associated with the standard HTTP traffic data, the standard HTTP traffic data comprising HTTP requests, and the standard link tracking data representing a processing track of the HTTP requests by the target system; According to the HTTP requests, a testing system is tested, and simulated HTTP traffic data generated by the testing system during running and simulated link tracking data associated with the simulated HTTP traffic data are obtained, the testing system being an upgraded target system; According to the standard HTTP traffic data, the standard link tracking data, the simulated HTTP traffic data and the simulated link tracking data, a test result of the testing system is determined.

[0006] According to the system testing method provided by the present application, the standard HTTP traffic data generated by the target system during running and the standard link tracking data associated with the standard HTTP traffic data are obtained, comprising: Obtain standard HTTP traffic data generated during the runtime of the target system, and determine the link identifier in the standard HTTP traffic data; Identify all data units carrying the link identifier; Based on the data unit, obtain standard link tracing data associated with the standard HTTP traffic data.

[0007] According to the system testing method provided in this application, determining the test result of the system under test based on the standard HTTP traffic data, the standard link tracing data, the simulated HTTP traffic data, and the simulated link tracing data includes: The standard HTTP traffic data is compared with the simulated HTTP traffic data to obtain a first comparison result; The standard link tracing data is compared with the simulated link tracing data to obtain a second comparison result; Based on the first comparison result and the second comparison result, the test result of the system under test is determined.

[0008] According to the system testing method provided in this application, the step of comparing the standard link tracing data with the simulated link tracing data to obtain a second comparison result includes: The link structure, data unit attributes, and events in the standard link tracing data are compared sequentially with the link structure, data unit attributes, and events in the simulated link tracing data to obtain a second comparison result; The link structure includes the number of data units, the parent-child relationship between data units, and the link depth. The data unit attributes include service name, operation name, time consumption, status code, error information, and custom attributes. The event includes event type and event attributes.

[0009] According to the system testing method provided in this application, standard link tracing data associated with the standard HTTP traffic data is obtained, including: Obtain initial link tracing data associated with the standard HTTP traffic data; Remove the target field from the initial link tracing data; The link tracing data after removing the target field is determined as the standard link tracing data.

[0010] According to the system testing method provided in this application, comparing the standard HTTP traffic data with the simulated HTTP traffic data to obtain a first comparison result includes: The HTTP response data in the standard HTTP traffic data is compared with the HTTP response data in the simulated HTTP traffic data to obtain the first comparison result.

[0011] According to the system testing method provided in this application, the step of obtaining simulated HTTP traffic data generated during the runtime of the system under test and simulated link tracing data associated with the simulated HTTP traffic data includes: Simulated HTTP traffic data generated during the runtime of the system under test was obtained using a network packet capture tool. Simulated link tracing data associated with the simulated HTTP traffic data is obtained using the OpenTelemetry software development kit.

[0012] This application also provides a system testing apparatus, including: The first acquisition module is used to acquire standard HTTP traffic data generated by the target system during runtime and standard link tracing data associated with the standard HTTP traffic data. The standard HTTP traffic data includes HTTP requests, and the standard link tracing data represents the processing trajectory of the target system for the HTTP requests. The second acquisition module is used to test the system under test according to the HTTP request, and to acquire simulated HTTP traffic data generated by the system under test during runtime and simulated link tracing data associated with the simulated HTTP traffic data; The determination module is used to determine the test results of the system under test based on the standard HTTP traffic data, the standard link tracing data, the simulated HTTP traffic data, and the simulated link tracing data.

[0013] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a system testing method as described above.

[0014] This application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a system testing method as described above.

[0015] This application also provides a computer program product, including a computer program that, when executed by a processor, implements a system testing method as described in any of the above.

[0016] The method of this application first obtains standard HTTP traffic data and associated standard link tracing data generated by the target system during runtime. The standard HTTP traffic data includes HTTP requests. Next, the system under test is tested based on the HTTP requests, and simulated HTTP traffic data and associated simulated link tracing data generated by the system under test during runtime are obtained. The test results of the system under test are determined based on the standard HTTP traffic data, standard link tracing data, simulated HTTP traffic data, and simulated link tracing data. This application, when conducting system testing, no longer considers only HTTP traffic data, but also considers both HTTP traffic data and associated link tracing data. Since link tracing data represents the system's processing trajectory of HTTP requests in the HTTP traffic data, it can effectively record deep-level information such as internal calls, service dependencies, link latency, and resource consumption in the distributed system. Therefore, this application tests the performance of the system under test based on these two dimensions of data, which can effectively improve system testing efficiency and the accuracy of test results. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a system testing method according to an embodiment of this application.

[0019] Figure 2 This is a schematic diagram illustrating the implementation process of a system testing method according to an embodiment of this application.

[0020] Figure 3 This is a schematic diagram illustrating the data flow during the system testing process as shown in the embodiments of this application.

[0021] Figure 4 This is a structural block diagram of a system testing device shown in an embodiment of this application.

[0022] Figure 5 This is a schematic diagram of the physical structure of an electronic device as shown in an embodiment of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] The following is combined with Figures 1-3 This application describes a system testing method.

[0025] This application provides a system testing method. The subject of this method is an electronic device such as a system testing equipment, computer, or server, or a system testing device installed in the electronic device. The system testing device can be implemented by software, hardware, or a combination of both, and can be configured according to actual needs.

[0026] Figure 1 This is a flowchart illustrating a system testing method according to an embodiment of this application. Figure 1 As shown, the method of this application includes the following steps: Step 101: Obtain standard HTTP traffic data and standard link tracing data associated with the target system during runtime. The standard HTTP traffic data includes HTTP requests, and the standard link tracing data represents the target system's processing trajectory for HTTP requests.

[0027] In this embodiment, the target application, the device running the target application, and a series of supporting services provided to ensure the normal operation of the target application are collectively referred to as the target system.

[0028] The target system can be understood as a system that is providing real services to users. Therefore, the target system operates in a production environment.

[0029] Since the target system provides real services to users, the HTTP traffic data generated by the target system during runtime and the link tracing data associated with the HTTP traffic data can be used as a reference standard when testing the system under test. Therefore, in this embodiment, the HTTP traffic data generated by the target system during runtime is called standard HTTP traffic data, and the link tracing data associated with the standard HTTP traffic data is called standard link tracing data.

[0030] In this embodiment, HTTP traffic data includes HTTP requests and HTTP responses. An HTTP request includes a request header and a request body. The request header includes metadata about the request, such as client type, desired data format, cookies, etc. The request body includes the actual data to be sent to the server (e.g., the target system). An HTTP response includes a status code, response headers, and a response body. The status code informs the client of the processing result of the HTTP request. The response header includes metadata about the response, such as server type, format of the returned content, caching strategy, etc. The response body includes the actual content returned by the server to the client, such as the HTML code of a webpage, the binary data of an image, or a piece of data in JSON format.

[0031] In this embodiment, the standard HTTP traffic data may include one or more HTTP requests, and each HTTP request has a corresponding HTTP response. Therefore, the standard HTTP traffic data includes at least one set of standard HTTP request-response data.

[0032] In this embodiment, standard HTTP traffic data and standard link tracing data can be stored in the database in advance, and when executing step 101, the standard HTTP traffic data and standard link tracing data can be read directly from the database, thereby speeding up the system testing efficiency.

[0033] In this embodiment, each HTTP request corresponds to a link tracing data set, which completely records the processing trajectory of the HTTP request within the system. The link tracing data set consists of a set of smallest units of work, called spans, each with a precise timestamp. All spans belonging to the same HTTP request are grouped into a complete call chain through parent-child relationships. Hereafter, a span (a smallest unit of work) will be referred to as a data unit. In this embodiment, each set of standard HTTP request-response data in the standard HTTP traffic data has associated standard link tracing data. For example, for the first set of standard HTTP request-response data, the associated standard link tracing data records the complete processing trajectory of the HTTP request within the first set of standard HTTP request-response data within the target system.

[0034] In practice, any tool can be used to collect standard HTTP traffic data and standard link tracing data generated during the runtime of the target system; this embodiment does not impose any specific restrictions on this.

[0035] It is specifically noted that the system testing method of this application can be applied to at least HTTP and HTTPS (HTTP over TLS / SSL) scenarios. HTTPS is still the HTTP protocol at the application layer, but a security encryption layer (TLS / SSL) is added at the transport layer. Its application layer concepts such as request headers, request bodies, status codes, and methods (GET / POST), regardless of whether the traffic is encrypted, are essentially HTTP protocol concepts. Therefore, the HTTP traffic data, HTTP requests, and HTTP responses mentioned in the various embodiments of this application can be HTTP traffic data, HTTP requests, and HTTP responses in either the HTTP scenario or the HTTPS scenario. Step 102: Test the system under test according to the HTTP request, obtain the simulated HTTP traffic data generated by the system under test during runtime and the simulated link tracing data associated with the simulated HTTP traffic data. The system under test is the upgraded target system.

[0036] In this embodiment, the application to be tested (the upgraded target application), the device running the application to be tested, and a series of supporting services provided to ensure the normal operation of the application to be tested are collectively referred to as the system to be tested. The system to be tested is equivalent to the upgraded target system. Since it is uncertain whether it can operate normally, it needs to be tested first.

[0037] The system under test is a system that simulates providing services to users. Therefore, the operating environment of the system under test is the test environment. In this embodiment, the rest of the target system, except for the target application, should be as similar as possible to the rest of the system under test, except for the application under test; that is, the test environment should be as similar as possible to the production environment. For example, hardware resources (CPU, memory specifications), software versions (operating system, database, cache, and other middleware versions), network configurations (firewall rules, network latency, etc.), dependent services, and configuration parameters (application startup configuration, function switches, etc.) should be the same to ensure high accuracy of the test results.

[0038] In this embodiment, HTTP requests are used to test the system under test, which can also obtain simulated HTTP traffic data generated by the system under test during runtime, and simulated link tracing data associated with the simulated HTTP traffic data. The principle for collecting simulated HTTP traffic data and simulated link tracing data is the same as the principle for collecting standard HTTP traffic data and standard link tracing data.

[0039] In this embodiment, each HTTP request has a corresponding HTTP response. Therefore, the simulated HTTP traffic data also includes at least one set of simulated HTTP request-response data, and each set of simulated HTTP request-response data has associated simulated link tracing data. In specific implementations, any tool can be used to collect the simulated HTTP traffic data and simulated link tracing data generated by the system under test during runtime; this embodiment does not impose specific limitations on this.

[0040] Step 103: Determine the test results of the system under test based on standard HTTP traffic data, standard link tracing data, simulated HTTP traffic data, and simulated link tracing data.

[0041] In this embodiment, after obtaining standard HTTP traffic data, standard link tracing data, simulated HTTP traffic data, and simulated link tracing data, the test results of the system under test can be obtained by comprehensively analyzing these data.

[0042] The method of this application first obtains standard HTTP traffic data and associated standard link tracing data generated by the target system during runtime. The standard HTTP traffic data includes HTTP requests. Next, the system under test is tested based on these HTTP requests, and simulated HTTP traffic data and associated simulated link tracing data generated by the system under test during runtime are obtained. The test results of the system under test are determined based on the standard HTTP traffic data, standard link tracing data, simulated HTTP traffic data, and simulated link tracing data. This application, when testing the system under test, no longer considers only HTTP traffic data, but also considers both HTTP traffic data and associated link tracing data. Since link tracing data represents the system's processing trajectory of HTTP requests in the HTTP traffic data, it can effectively record deep-level information such as internal calls, service dependencies, link latency, and resource consumption in the distributed system. Therefore, this application tests the performance of the system under test based on these two dimensions of data, which can accurately identify whether there are problems in the target application, thereby effectively improving software testing efficiency and the accuracy of test results.

[0043] In conjunction with the above embodiments, in one implementation, step 102 may include: Use network packet capture tools to obtain simulated HTTP traffic data generated during the runtime of the system under test; Simulated link tracing data associated with simulated HTTP traffic data can be obtained using the OpenTelemetry software development kit.

[0044] In this embodiment, any network packet capture tool can be deployed on the system under test to capture simulated HTTP traffic data. Suitable network packet capture tools include, but are not limited to, tcpdump, Wireshark, Nginx / Apache, mitmproxy, and Goreplay.

[0045] In practice, Goreplay is advantageous because it can capture data by listening to network interfaces without requiring any monitoring or logging logic in the application code. It operates at the network layer and can capture complete, unmodified raw HTTP packets (the request headers, request bodies, response headers, response bodies, status codes, and other information obtained are real interactive data). Therefore, using Goreplay to capture simulated HTTP traffic data can ensure the accuracy of the captured data and thus improve the accuracy of system test results.

[0046] Similarly, this application can also deploy any of the network packet capture tools mentioned above on the target system to capture standard HTTP traffic data.

[0047] In this embodiment, the OpenTelemetry software development tool (OpenTelemetry SDK) can be deployed on the system under test to obtain simulated link tracing data.

[0048] Specifically, by integrating the OpenTelemetry SDK into the system under test, it can automatically associate operations across services, databases, and other components to generate complete link tracing data containing accurate time consumption, call topology, and rich attributes. This can provide an important basis for efficiently and accurately locating performance bottlenecks and diagnosing complex errors.

[0049] Similarly, this application can also deploy the OpenTelemetry SDK on the target system to accurately capture standard link tracing data.

[0050] In conjunction with the above embodiments, in one implementation, step 101 may include: Step 1011: Obtain standard HTTP traffic data generated by the target system during runtime, and determine the link identifier in the standard HTTP traffic data.

[0051] In this embodiment, each set of standard HTTP request-response data corresponds to a set of standard link trace data. When the OpenTelemetry SDK obtains each set of standard link trace data, it generates a unique trace ID and writes the trace ID into the response header of the HTTP request in the corresponding set of standard HTTP request-response data, thereby establishing the association between the standard HTTP request-response data and the standard link trace data.

[0052] Similarly, when the OpenTelemetry SDK acquires each simulated link tracing data, it generates a unique link identifier (Trace ID) and writes the link identifier into the response header of the corresponding set of simulated HTTP request-response data, thereby establishing the association between the simulated HTTP request-response data and the simulated link tracing data.

[0053] Therefore, by performing step 1011, the link identifier in each set of HTTP request-response data in the standard HTTP traffic data can be determined.

[0054] Step 1012: Identify all data units carrying link identifiers.

[0055] In this embodiment, a link tracing data set consists of multiple data units (spans) carrying the same link identifier. During step 1012, all data units can be divided into multiple data sets according to their different link identifiers, with each data set uniquely corresponding to one link identifier. Next, the data set with the same link identifier as in step 1011 is identified, and this data set is determined as all data units carrying link identifiers.

[0056] Step 1013: Based on the data unit, obtain the standard link tracing data associated with the standard HTTP traffic data.

[0057] In this embodiment, after finding data units belonging to the same link identifier, the parent-child call relationship between them is reconstructed based on the Parent Span ID recorded by each data unit, and finally a visual link with a complete topology is formed, which is the standard link tracing data.

[0058] Similarly, the principle of obtaining simulated HTTP traffic data and simulated link tracing data associated with simulated HTTP traffic data is the same as the principle of obtaining standard HTTP traffic data and standard link tracing data associated with standard HTTP traffic data described above, and will not be repeated here in this embodiment.

[0059] In this embodiment, link identifiers are used to associate HTTP traffic data with link tracing data, enabling a precise one-to-one mapping between the system's external behavior (HTTP interactions) and its internal execution trajectory (call chain). Based on this association, this embodiment can accurately attribute any external response problem to a specific internal performance bottleneck or error root cause, greatly improving the efficiency and accuracy of problem localization.

[0060] In conjunction with the above embodiments, in one implementation, step 103 may include: Step 1031: Compare the standard HTTP traffic data with the simulated HTTP traffic data to obtain the first comparison result.

[0061] In this embodiment, by comparing standard HTTP traffic data with simulated HTTP traffic data, inconsistencies between the two can be obtained, resulting in the first comparison result.

[0062] Step 1032: Compare the standard link tracing data with the simulated link tracing data to obtain the second comparison result.

[0063] In this embodiment, by comparing standard link tracing data with simulated link tracing data, inconsistencies between the two can be obtained, resulting in a second comparison result.

[0064] Step 1033: Determine the test results of the system to be tested based on the first comparison results and the second comparison results.

[0065] In this embodiment, by combining HTTP traffic and link tracing data from two dimensions, comprehensive verification from post-event results to process details can be achieved, accurately revealing deep-seated problems such as performance bottlenecks and logical deviations within the system, and significantly improving the depth and accuracy of automated testing as well as the efficiency of locating the root causes of complex errors.

[0066] In conjunction with the above embodiments, in one implementation, step 1032 may include: The link structure, data unit attributes, and events in the standard link tracing data are compared sequentially with the link structure, data unit attributes, and events in the simulated link tracing data to obtain a second comparison result. The link structure includes the number of data units, the parent-child relationship between data units, and the link depth. The data unit attributes include the service name, operation name, time consumption, status code, error message, and custom attributes. The events include the event type and event attributes.

[0067] In this embodiment, the comparison between standard link tracing data and simulated link tracing data can be divided into the following three aspects: First, a comparison of the link structures.

[0068] This comparison primarily focuses on the macro-level topology of the HTTP request processing flow, aiming to identify significant changes in code logic branches and service dependencies. Specifically, it compares the following fields and metrics: (1) Number of data units (total number of spans, i.e., total number of spans) Comparison field: Counts the spans under a trace.

[0069] Purpose: Changes in the number of data units directly reflect increases or decreases in execution paths. For example, an additional Span in a new version might mean an additional database query or RPC call; conversely, it indicates that some logic has been skipped or optimized.

[0070] (2) Parent-child relationship between data units.

[0071] Comparison fields: Span ID and Parent Span ID for each Span.

[0072] Objective: By analyzing the parent-child relationships of all Spans, a complete call chain tree structure can be reconstructed. Comparing the call trees of the old and new versions reveals whether the order and hierarchy of service calls have changed. For example, previously service A directly called service C, but now service A calls service B, and then B calls C, which represents a significant structural change.

[0073] (3) Link depth.

[0074] Comparison field: Calculate the maximum depth of the call chain tree.

[0075] Second, compare the attributes of the data units.

[0076] After confirming that the macroscopic structure of the link is similar or the corresponding span is found, it is necessary to compare the detailed attributes of each data unit (span) at the micro level in order to discover subtle changes in functional logic, performance and error states.

[0077] (1) Core identity information.

[0078] Comparison fields: Service name, Operation name (Span name).

[0079] Objective: To ensure that the comparison is between two versions of the Span that perform the same or similar operations.

[0080] (2) Performance indicators.

[0081] Comparison field: Duration (calculated from the start and end timestamps of the Span).

[0082] Purpose: To locate performance issues. By comparing the time taken and setting reasonable thresholds (e.g., absolute value exceeding 50ms or percentage exceeding 20%), performance degradation points can be automatically identified.

[0083] (3) Status and error messages.

[0084] Compare the following fields: status code (usually Ok, Error, Unset in OTel) and error message.

[0085] Purpose: To check if an operation was successful. For example, if a Span in an internal RPC call was Ok in an older version but becomes Error in a newer version, then an internal error is directly identified.

[0086] (4) Configuration attributes of key components.

[0087] Comparison fields: For example, for database connection pool attributes, comparison fields can include maximum number of connections, number of idle connections, connection timeout, etc.

[0088] Objective: To verify whether the configuration of critical dependent components remains consistent across different environments or versions. For example, a comparison revealed that the maximum number of database connections in the test environment is significantly lower than that in the production environment. This avoids distorted performance evaluation results caused by configuration issues in the test environment (rather than the code itself), allowing for timely correction of such configuration errors before deployment and preventing hidden problems such as resource contention and connection timeouts caused by improper configuration from being introduced into the production environment.

[0089] (5) Custom and semantic attributes.

[0090] Compare fields: All key-value pairs in the Attributes collection (including custom attributes).

[0091] Objective: To conduct a comprehensive comparison of the most flexible and representative items that best reflect changes in business logic. Third, event comparison.

[0092] This type of comparison focuses on records at specific points in time that occur within the duration of a single span. It can be understood as a structured log attached to the span, used to diagnose more granular problems.

[0093] (1) Number and name of events.

[0094] Comparison fields: the number of all events within a span and their respective names. The event name is the event type.

[0095] Purpose: To compare whether any unexpected events have occurred. For example, a new version may include an event called cache.miss, which could indicate a change or invalidation of the caching logic.

[0096] (2) Event attributes.

[0097] Comparison field: The collection of key-value pairs of Attributes associated with each event.

[0098] Purpose: To provide contextual information about an event. For example, for an exception event, its properties may contain detailed exception.stacktrace information, and comparing this information can help pinpoint the root cause of the error.

[0099] In this embodiment, by comprehensively analyzing the link structure, data unit attributes, and events from multiple dimensions, a deep analysis of the differences in the internal behavior of the system under test can be achieved from macro to micro. This can greatly improve the accuracy of automated testing in discovering hidden and complex problems and significantly improve the efficiency of locating the root cause of errors.

[0100] In conjunction with the above embodiments, in one implementation, obtaining standard link tracing data associated with standard HTTP traffic data based on data units may include: Based on the data unit, obtain the initial link tracing data associated with the standard HTTP traffic data; Remove the target field from the initial link tracing data; The link tracing data after removing the target field is determined as standard link tracing data.

[0101] In this embodiment, after finding data units belonging to the same link identifier, the parent-child call relationship between them can be reconstructed based on the Parent Span ID recorded by each data unit to obtain initial link tracing data. Then, the target field of the initial link tracing data is removed to obtain standard link tracing data.

[0102] The target field is a predefined non-critical field that can be set according to actual needs. It is a field that will inevitably change in different executions or different environments, such as timestamp, hostname, IP address, service instance ID, or dynamically generated business IDs (such as order number, session ID).

[0103] In one implementation, filtering rules can be implemented using configured regular expressions and managed through a configuration file. For example, for the timestamp field, regular expression 1 can be configured to match and filter all time-related fields in the data unit attribute or event; for the hostname, IP address, and service instance ID fields, regular expression 2 can be configured to match and filter out hostnames, IP addresses, and service instance IDs that inevitably differ between production and test environments; for dynamic business IDs, regular expression 3 can be configured to match and filter dynamic business IDs that are regenerated in each request. Regular expressions 1 through 3 can be set according to actual needs. In this embodiment, in addition to removing target fields from the standard link tracing data, it is also necessary to remove target fields from the simulated link tracing data. Similarly, the process of obtaining initial simulated link tracing data and optimizing the initial simulated link tracing data are based on the same principles as the process of obtaining standard link tracing data and optimizing the standard simulated link tracing data described above. In this embodiment, during the link tracing data comparison process, removing non-critical fields can improve the accuracy of the comparison analysis results, ensuring that automated testing can accurately focus on substantial system behavior differences caused by code changes. Specifically, by configuring ignore rules to exclude uncertain data that will inevitably change (such as timestamps) and environment-specific information (such as hostnames), false alarms caused by randomness or environmental differences can be fundamentally eliminated, making test results more stable and thus greatly improving the efficiency of accurately locating hidden errors.

[0104] In one implementation, combining the above embodiments, when acquiring standard HTTP traffic data, initial HTTP traffic data can be acquired first, and then non-critical fields can be removed to obtain standard HTTP traffic data. Simultaneously, the same operation of removing non-critical fields can be performed on simulated HTTP traffic data. This avoids interference from unimportant items in the comparison analysis results during the HTTP traffic data comparison process, effectively improving the accuracy of the test results. Non-critical fields can be set according to actual needs, such as timestamps, dynamic IDs (e.g., business IDs), or other variable fields. Since the replay request is executed at a completely different time point than the original request, the system under test will inevitably embed the latest time when generating a new response. Therefore, any time-related field values ​​are dynamically changing and can be ignored. A dynamic ID is a unique identifier generated for a specific resource, transaction, or session during system runtime to ensure that each generated value is unique; it is also dynamically changing and can be ignored.

[0105] In one implementation, in conjunction with the above embodiments, step 1031 may include: The HTTP response data in the standard HTTP traffic data is compared with the HTTP response data in the simulated HTTP traffic data to obtain the first comparison result.

[0106] The HTTP response data in the HTTP traffic data includes status codes, response headers, response bodies, etc. By comparing the status codes, response headers, response bodies, etc. of the HTTP response data in the standard HTTP traffic data with the status codes, response headers, response bodies, etc. of the HTTP response data in the simulated HTTP traffic data, the first comparison result can be obtained.

[0107] In this embodiment, comparing the HTTP response data in standard HTTP traffic data with the HTTP response data in simulated HTTP traffic data can quickly and intuitively identify functional regressions, interface contract violations, and high-level system errors caused by code changes.

[0108] Next, a complete embodiment will be used to describe the system testing method of this application in detail. This embodiment includes the following steps: Step S1: Collect standard data from the target system.

[0109] Step S11: Collect HTTP traffic data. In the production environment, use the network packet capture tool goreplay to capture complete HTTP request and response data packets in real time, including request headers, request bodies, response headers, response bodies, and status codes.

[0110] Step S12: Collect OTel tracing data. Integrate the OpenTelemetry SDK into the production environment service to collect distributed tracing data, including key information such as Span ID, Trace ID, Parent Span ID, service name, operation name, duration, attributes, and events. Write the Trace ID generated by OTel into the corresponding HTTP response header.

[0111] Step S13: Data Association and Storage: Associate the collected HTTP traffic data and OTel link tracing data (associate via Trace ID) to obtain multiple sets of HTTP traffic data-OTel link tracing data. Store these multiple sets of HTTP traffic data-OTel link tracing data in a unified collection table in MongoDB, and record the unique key of each set of HTTP traffic data-OTel link tracing data as RecordId.

[0112] Step S2: Collect the simulation data generated by the system under test.

[0113] Step S21: Replay Environment (Test Environment) Preparation: Prepare a test environment that is as consistent as possible with the production environment, including service deployment, configuration, dependent services, etc.

[0114] Step S22: HTTP Request Simulation and Sending: Read the HTTP request from MongoDB, add headers such as ReplayId and RecordId, and then simulate the HTTP request and send it to the service under test in the system under test. In this step, injecting RecordId as a header into the simulated HTTP request establishes a connection between the simulated HTTP request and the actual HTTP request, enabling data traceability. ReplayId serves as the batch identifier for the replay task. Simulated HTTP requests belonging to the same replay task will carry the same ReplayId, while requests belonging to different replay tasks will carry different ReplayIds. Therefore, ReplayId enables the differentiation and management of different replay tasks.

[0115] Step S23: OTel Trace Data Acquisition. Ensure that the service in the test environment also integrates the OpenTelemetry SDK to generate new OTel trace data during playback. Also, ensure that the Trace ID generated by OTel is written to the HTTP response header.

[0116] Step S24: Data Association and Storage. The HTTP traffic data and OTel link tracing data obtained from the playback are associated (by ReplayId and Trace ID generated by the playback) and then stored in the playback table in MongoDB.

[0117] Step S3: Compare playback results.

[0118] Step S31: HTTP Response Comparison. Compare the HTTP response of the replay request with the status code, response headers, response body, etc., of the original HTTP response. For variable fields such as timestamps and dynamic IDs, ignore rules can be configured to avoid false positives.

[0119] Step S32: OTel link data comparison. (1) Link structure comparison: Compare the number of Spans, parent-child relationships between Spans, link depth, etc. of the OTel link tracing data collected in the production environment with the OTel link tracing data generated during playback. (2) Span attribute comparison: Compare the service name, operation name, time consumption, status code, error message, and custom attributes of the same or similar Spans. For indicators such as time consumption, a threshold or percentage deviation can be set to tolerate it. (3) Event comparison: Compare the event types and event attributes recorded in the Span.

[0120] Step S4: Visualization and Reporting of Differences. Visualize the comparison results, for example, by using a Gantt chart to compare the timeline of the link, generate a detailed comparison report, point out the differences in HTTP responses and OTel link tracing data, and mark the type and extent of the differences.

[0121] Figure 2 This is a schematic diagram illustrating the implementation process of a system testing method according to an embodiment of this application. (Refer to...) Figure 2 The implementation process of system testing methods may include: Step 1: Production Environment Data Collection. When an online business service processes a real user request, an HTTP traffic collection proxy (such as Goreplay) deployed in the production environment will fully capture its HTTP request and response data. Simultaneously, the OTel SDK integrated into the business service will record the complete processing trajectory of this request within the system, generating a trace data entry with a unique link identifier (e.g., TraceID-A). The system will inject TraceID-A into the HTTP response header, storing the associated HTTP traffic volume and OTel trace data (i.e.,...). Figure 2 The recorded data is stored in the record and assigned a unique record identifier (e.g., RecordID-123).

[0122] Step 2: Test Environment Replay. Once the new version of the service is deployed to the test environment, the traffic replay unit in the replay and verification engine (system test equipment) reads the HTTP request corresponding to RecordID-123 from storage. This request is then sent to the target service to simulate an identical online access. After processing the request, the target service generates a new HTTP response, and its built-in OTel SDK also generates a completely new trace data (e.g., TraceID-B) containing internal execution details.

[0123] Step 3: Comparative Analysis and Reporting. After the playback result collector captures new HTTP responses and TraceID-B link data, it sends them to the comparative analyzer. The comparative analyzer simultaneously retrieves the original data (original responses and TraceID-A links) from storage for a two-dimensional comparison. Finally, the reporting and alerting module outputs a detailed analysis report.

[0124] Figure 3 This is a schematic diagram illustrating the data flow during the system testing process as shown in the embodiments of this application. Figure 3 The data acquisition and storage process shown includes: Step 1: Concurrent Data Acquisition. Suppose a user initiates an HTTP request to create an order. When the request arrives at the server, it triggers two parallel acquisition paths: (1) OTel Link Data Capture (Internal Link Capture): The OpenTelemetry SDK integrated in HTTP service A immediately starts a new tracing and generates a globally unique link identifier TraceID (e.g., T-123). When the request continues to call service B and the database internally, the SDK creates a span for each operation (e.g., RPC call, database query). These spans share the same link identifier Trace ID: T-123, eventually forming a complete OTel link tracing data in the backend. (2) HTTP Traffic Data Capture (External Traffic Capture): HTTP traffic capture tools (e.g., Goreplay) record the original message of this HTTP request completely at the network layer.

[0125] Step 2: Association and Unified Storage. Before HTTP service A completes processing and returns the final HTTP response to the user, the application injects the internally generated Trace ID: T-123 into the response header. Next, the system integrates this HTTP traffic data (request + response) with the complete OTel link tracing data corresponding to T-123 into a telemetry record and stores it in a MongoDB collection table. This telemetry record is assigned a unique key RecordId (e.g., ...). Figure 3 The Object_id in the database stores common link identifiers, HTTP requests, HTTP responses, and link tracing data details, providing high-quality data for subsequent replay comparisons.

[0126] The system testing solution proposed in this application is applicable to microservice version upgrade verification scenarios. In this scenario, if it is necessary to verify whether upgrading the order service of an e-commerce system from version v1.0 to version v1.1 introduces functional regression or performance degradation, the steps are as follows: Step 1: Baseline Data Acquisition. In the production environment, monitor the running v1.0 legacy order service. When a user initiates a real operation such as creating an order, collect their complete HTTP request / response traffic and simultaneously capture the complete OTel link tracing data triggered internally by that request. This data, which includes external performance and internal trajectory, is correlated and stored as a baseline dataset to verify the correctness of the new version's behavior.

[0127] Step 2: Replay and two-dimensional comparison.

[0128] The new version 1.1 order service was deployed to a test environment consistent with the production environment. The system read the collected HTTP request traffic and automatically replayed the new version service. During the replay, the system not only recorded the HTTP responses returned by the new version, but also generated corresponding new OTel link tracing data. The comparative analysis engine performed two-dimensional verification: on the one hand, it compared the old and new HTTP responses to ensure that the external interface contract and functional logic were correct; on the other hand, it compared the old and new OTel links.

[0129] Step 3: In-depth analysis and problem identification.

[0130] The analysis report showed that although all HTTP responses matched exactly, OTel's process flow comparison revealed a significant increase of 120ms in the average processing time for order creation requests in the new version. Further comparison of the span attribute automatically pinpointed the performance degradation to a database operation span named db:update_stock. This allowed developers to focus on optimizing inventory update-related code or SQL statements, resolving a potential performance bottleneck before deployment to production.

[0131] This application discloses a system replay verification method. This method captures HTTP traffic and OpenTelemetry (OTel) link data from the production environment and performs precise replay in the test environment. During replay, not only are HTTP responses compared and verified, but more importantly, the OTel link data generated during replay is simultaneously collected and compared. By comparing the differences in OTel link data before and after replay, as well as between the production and test environments, this application can more comprehensively and meticulously discover subtle changes in system behavior, potential performance issues, and code logic regressions, thereby significantly improving testing efficiency and accuracy.

[0132] The system testing scheme in this application can also be applied to database migration verification scenarios. In this scenario, if it is necessary to verify whether the business functions and query performance of a financial system meet the requirements after migrating its core transaction database from Oracle to PostgreSQL, the process is as follows: Step 1: Baseline data collection before migration.

[0133] Before the formal database migration, this application monitors the systems in the production environment that connect to the old Oracle database. Business traffic involving complex database queries, such as "generating monthly reports" and "querying transaction history," is collected. The system fully captures the HTTP traffic of these operations and their corresponding Otel link tracing data. This data records detailed performance metrics such as the execution time of each SQL query, forming a baseline for evaluating the migration effectiveness.

[0134] Step 2: Post-migration playback and depth comparison.

[0135] A test environment connecting to the new PostgreSQL database was set up, and the application was deployed. Next, the replay and verification engine used the collected HTTP traffic to initiate automated replays of the new environment. The comparative analyzer, while verifying the HTTP responses to ensure the correctness of data migration and business functions, focused on a deep comparison of the spans related to database operations in the old and new OTel links.

[0136] Step 3: Performance bottleneck identification and analysis.

[0137] The comparison report shows that although the HTTP response content is completely identical, the link analysis module found that the number of database spans marked as "slow queries" in the new link increased by 40%. To achieve automated performance bottleneck localization, this embodiment supports flexible setting of time-consuming threshold rules. Specifically, relative thresholds (e.g., a target percentage increase in replay time exceeding the baseline time, such as 200%) or absolute thresholds (e.g., a target increase in replay time exceeding the baseline time by more than 100ms) can be set. By further drilling down the span attributes, the system can use the aforementioned time-consuming threshold rules to accurately locate one or more specific database operation spans whose performance deviates significantly from the baseline. For example, the system accurately locates a specific SELECT query operation whose average time surges from 80ms (baseline time) before the migration to 400ms (replay time). This change (replay time increasing by 320ms compared to the baseline time, or replay time exceeding 400% of the baseline time) will immediately trigger an alarm at the preset relative threshold (200%) or absolute threshold (100ms). Analysis revealed that the query used Oracle-specific query optimization hints, which were not only ineffective in PostgreSQL but also degraded the execution plan. Based on this clear analysis, the database administrator (DBA) could immediately perform targeted SQL optimizations, successfully resolving the serious performance issues caused by database heterogeneity before deployment.

[0138] This application deeply integrates OpenTelemetry's distributed link tracing data into HTTP traffic replay verification. Through multi-dimensional comparison of link data, it can uncover deep-seated system behavior changes that traditional methods cannot reach, thus providing more powerful and accurate system regression testing and performance verification capabilities. In summary, this application has the following significant advantages over existing technologies: First, it not only verifies the functionality of HTTP interfaces, but also delves into the internal behavior and link details of distributed systems, which can improve the comprehensiveness of testing.

[0139] Second, through detailed comparison of OTel link data, performance degradation, hidden errors, resource contention, and inter-service communication problems that are difficult to detect using traditional methods can be discovered.

[0140] Third, the automated comparison of HTTP and OTel data can significantly reduce the time spent on manual analysis and debugging.

[0141] Fourth, before problems affect users, potential regressions and defects in the system can be identified early through playback verification.

[0142] Fifth, introducing OTel into the replay verification process has facilitated the understanding and observability of the system's internal behavior.

[0143] Sixth, the comparison results can provide data support for system optimization, capacity planning, and architecture adjustment.

[0144] The following describes a system testing apparatus provided in this application. The system testing apparatus described below and the system testing method described above can be referred to in correspondence.

[0145] Figure 4 This is a structural block diagram of a system testing apparatus shown in an embodiment of this application. (Refer to...) Figure 4 The system testing apparatus 400 of this application may include: The first acquisition module 401 is used to acquire standard HTTP traffic data generated during the runtime of the target system and standard link tracing data associated with the standard HTTP traffic data. The standard HTTP traffic data includes HTTP requests, and the standard link tracing data represents the processing trajectory of the target system for the HTTP requests. The second acquisition module 402 is used to test the system under test according to the HTTP request, and to acquire the simulated HTTP traffic data generated by the system under test during operation and the simulated link tracing data associated with the simulated HTTP traffic data. The system under test is the upgraded target system. The determination module 403 is used to determine the test results of the system under test based on the standard HTTP traffic data, the standard link tracing data, the simulated HTTP traffic data, and the simulated link tracing data.

[0146] According to the system testing apparatus 400 provided in this application, the first acquisition module 401 is specifically used for: acquiring standard HTTP traffic data generated during the runtime of the target system, and determining the link identifier in the standard HTTP traffic data; determining all data units carrying the link identifier; and acquiring standard link tracing data associated with the standard HTTP traffic data based on the data units.

[0147] According to the system testing apparatus 400 provided in this application, the determining module 403 is specifically used for: comparing the standard HTTP traffic data with the simulated HTTP traffic data to obtain a first comparison result; comparing the standard link tracing data with the simulated link tracing data to obtain a second comparison result; and determining the test result of the system to be tested based on the first comparison result and the second comparison result.

[0148] According to the system testing apparatus 400 provided in this application, the determining module 403 is specifically used to: compare the link structure, data unit attributes, and events in the standard link tracing data with the link structure, data unit attributes, and events in the simulated link tracing data in sequence to obtain a second comparison result; wherein, the link structure includes the number of data units, the parent-child relationship between data units, and the link depth; the data unit attributes include service name, operation name, time consumption, status code, error message, and custom attributes; and the events include event type and event attributes.

[0149] According to the system testing apparatus 400 provided in this application, the first acquisition module 401 is specifically used for: acquiring initial link tracing data associated with the standard HTTP traffic data; removing the target field from the initial link tracing data; and determining the link tracing data after removing the target field as the standard link tracing data.

[0150] According to the system testing device 400 provided in this application, the determining module 403 is specifically used to: compare the HTTP response data in the standard HTTP traffic data with the HTTP response data in the simulated HTTP traffic data to obtain a first comparison result.

[0151] According to the system testing apparatus 400 provided in this application, the first acquisition module 401 is specifically used for: acquiring simulated HTTP traffic data generated during the operation of the system under test by using a network packet capture tool; and acquiring simulated link tracing data associated with the simulated HTTP traffic data by using the OpenTelemetry software development kit.

[0152] Figure 5 This is a schematic diagram of the physical structure of an electronic device as illustrated in an embodiment of this application. Figure 5As shown, the electronic device may include a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a system testing method.

[0153] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0154] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute a system testing method provided by the above methods.

[0155] In another aspect, this application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform a system testing method provided by the methods described above.

[0156] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0157] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A system testing method, characterized in that, include: Acquire standard HTTP traffic data generated during the runtime of the target system and standard link tracing data associated with the standard HTTP traffic data. The standard HTTP traffic data includes HTTP requests, and the standard link tracing data represents the processing trajectory of the target system for the HTTP requests. The system under test is tested according to the HTTP request, and the simulated HTTP traffic data generated by the system under test during runtime and the simulated link tracing data associated with the simulated HTTP traffic data are obtained. The system under test is the upgraded target system. The test results of the system under test are determined based on the standard HTTP traffic data, the standard link tracing data, the simulated HTTP traffic data, and the simulated link tracing data.

2. The method according to claim 1, characterized in that, The acquisition of standard HTTP traffic data generated during the runtime of the target system and standard link tracing data associated with the standard HTTP traffic data includes: Obtain standard HTTP traffic data generated during the runtime of the target system, and determine the link identifier in the standard HTTP traffic data; Identify all data units carrying the link identifier; Based on the data unit, obtain standard link tracing data associated with the standard HTTP traffic data.

3. The method according to claim 1, characterized in that, The step of determining the test results of the system under test based on the standard HTTP traffic data, the standard link tracing data, the simulated HTTP traffic data, and the simulated link tracing data includes: The standard HTTP traffic data is compared with the simulated HTTP traffic data to obtain a first comparison result; The standard link tracing data is compared with the simulated link tracing data to obtain a second comparison result; Based on the first comparison result and the second comparison result, the test result of the system under test is determined.

4. The method according to claim 3, characterized in that, The step of comparing the standard link tracing data with the simulated link tracing data to obtain a second comparison result includes: The link structure, data unit attributes, and events in the standard link tracing data are compared with the link structure, data unit attributes, and events in the simulated link tracing data to obtain a second comparison result. The link structure includes the number of data units, the parent-child relationship between data units, and the link depth. The data unit attributes include service name, operation name, time consumption, status code, error information, and custom attributes. The event includes event type and event attributes.

5. The method according to claim 2, characterized in that, The step of obtaining standard link tracing data associated with the standard HTTP traffic data based on the data unit includes: Based on the data unit, obtain initial link tracing data associated with the standard HTTP traffic data; Remove the target field from the initial link tracing data; The link tracing data after removing the target field is determined as the standard link tracing data.

6. The method according to claim 3, characterized in that, The step of comparing the standard HTTP traffic data with the simulated HTTP traffic data to obtain a first comparison result includes: The HTTP response data in the standard HTTP traffic data is compared with the HTTP response data in the simulated HTTP traffic data to obtain the first comparison result.

7. The method according to any one of claims 1-6, characterized in that, The acquisition of simulated HTTP traffic data generated during the runtime of the system under test and simulated link tracing data associated with the simulated HTTP traffic data includes: Simulated HTTP traffic data generated during the runtime of the system under test was obtained using a network packet capture tool. Simulated link tracing data associated with the simulated HTTP traffic data is obtained using the OpenTelemetry software development kit.

8. A system testing device, characterized in that, include: The first acquisition module is used to acquire standard HTTP traffic data generated by the target system during runtime and standard link tracing data associated with the standard HTTP traffic data. The standard HTTP traffic data includes HTTP requests, and the standard link tracing data represents the processing trajectory of the target system for the HTTP requests. The second acquisition module is used to test the system under test according to the HTTP request, and to acquire simulated HTTP traffic data generated by the system under test during runtime and simulated link tracing data associated with the simulated HTTP traffic data. The system under test is the upgraded target system. The determination module is used to determine the test results of the system under test based on the standard HTTP traffic data, the standard link tracing data, the simulated HTTP traffic data, and the simulated link tracing data.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements a system testing method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it performs a system test as described in any one of claims 1 to 7.

Images