A method and device for playing back traffic double sending based on network layer recording packets

Abstract

The application discloses a kind of based on network layer recording message and realizes the playback method and system of flow double launch, method includes: debugging message recording engine basic environment, real-time capture and record network communication message in network layer, analyze and process static log message, set relevant parameter information required by playback, the network communication message or processed log message of recording is single launch or double launch playback, the result of sending is analyzed.The application can efficiently record the relevant information of internal dependence of response request, real-time capture and record network communication message, and analyze and process static log message, and double launch flow test is carried out when needed, improve the accuracy and efficiency of system test.

Description

Technical Field

[0001] This invention belongs to the field of software testing technology, specifically relating to a method and apparatus for replaying dual-transmission traffic based on network layer recorded packets. Background Technology

[0002] With the rapid development of the internet industry and the continuous expansion of business scale, software system design is becoming increasingly complex. Rapid iteration on systems with high business complexity places higher demands on system robustness, compatibility, test coverage, and effectiveness. Traditional automated regression scripts and manually written scripts are insufficient to effectively cover a large number of real-world business scenarios, and the continuous evolution of systems makes it difficult to guarantee the correctness and effectiveness of automated regression scripts.

[0003] To mitigate the impact of complexity on system iteration, traffic replay technology is typically used to ensure system testing under complex business scenarios and service architectures. Traffic replay technology refers to recording business traffic in a production environment and then replaying the recorded traffic in a production or testing environment. In network communication and software development, accurate recording and replaying of network traffic is a crucial means of testing and verifying software functionality, performance, and security. However, while various traffic recording and replay tools exist, they have limitations in flexibility, efficiency, and packet coverage, particularly when dual-transmission testing of specific packets is required, lacking a simple and efficient solution. Summary of the Invention

[0004] To address the aforementioned problems, this invention discloses a method and apparatus for replaying dual-transmission traffic based on network layer recorded packets. It records data traffic at the network layer, extracts, filters, parses, and stores the recorded packets. The transformed packet data can be recombine into new packets according to specific rules, flexibly enabling single-system and dual-system transmission functions. Furthermore, it allows for effective difference analysis of the transmission results. This invention can efficiently record relevant information dependent on the internal dependencies of response requests and perform dual-transmission traffic tests when needed, improving the accuracy and efficiency of system testing.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] A method for playing back traffic from two sources based on network layer recorded packets includes the following steps:

[0007] Step 1: Debug the basic environment of the traffic recording engine, enable access permissions and configure access policies on the monitored end, and install the traffic recording engine after the system connectivity authentication is passed. Configure parameters for the recording task. The parameter configuration includes packet filtering rule configuration and related parameter configurations such as recording, playback, and storage.

[0008] Step 2: Execute the message recording task to obtain the required data, capture and record network communication messages in real time at the network layer, and store the recorded messages in local or remote storage media;

[0009] Step 3: Execute message processing tasks, parse and process static log messages, extract key information, and store it in a specified database or file;

[0010] Step 4: Set the relevant parameters required for playback, including playback type, playback speed, and network environment;

[0011] Step 5: Based on the message data processed in Step 3.6 and the specification of the request message received by the system under test, construct the request message of the system under test; according to the parameters set in Step 4 and the replay-related parameter configuration in Step 1, perform single-send or dual-send replay of the constructed request message of the system under test; when performing dual-send replay, read the data processed and stored in Step 3 and send it to the dual-send environment for replay testing, specifically including:

[0012] Step 5.1, Read processing: Based on the pre-set read operation configuration parameters, retrieve the message to be processed from the corresponding storage medium;

[0013] Step 5.2, Preprocessing: Based on the data read in Step 5.1, assemble and send the message according to the defined message format;

[0014] Step 5.3: Sending Processing. Based on the parameters set in Step 4, send the message twice using the message processed in Step 5.2.

[0015] Step 5.4: Post-sending processing. Based on the sending situation in Step 5.3, process the data returned by both requests.

[0016] Step 5.4: Send write, which stores the data processed in Step 5.4; design corresponding read plugins for different types of data and provide standard interfaces;

[0017] Step 6: Execute result analysis, monitor the playback progress and results in real time, view and analyze the test results, adjust the playback parameters and re-perform the playback test until the test requirements are met.

[0018] Furthermore, in step 1, the message recording engine is uploaded to the management platform, and the management platform performs operations on the message recording engine on the monitored server.

[0019] Furthermore, in step 1, the message filtering rules include: protocol filtering rules, IP address filtering rules, port filtering rules, MAC address filtering rules, HTTP mode filtering rules, and custom filtering rules.

[0020] Furthermore, step 2 specifically includes the following sub-steps:

[0021] Step 2.1: After the system connectivity authentication is passed, configure the relevant execution tasks of the message recording engine on the management platform based on the parameter configuration, and start the message recording engine;

[0022] Step 2.2: Obtain traffic data through the message recording engine and verify the data correctness.

[0023] Furthermore, step 2.2 specifically includes the following steps:

[0024] (1) Data request reception: The message recording engine receives and parses the data processing request, and extracts key metadata information from the request.

[0025] (2) The link association module in the message recording engine performs intelligent routing according to the pre-set mapping rules, forwards the request message to the specified address, dynamically allocates a dedicated storage partition and persists the request data.

[0026] (3) The monitoring module in the message recording engine monitors the data processing requests' calls to their respective intermediate components, records the call response information, and sends the information to the link association module;

[0027] (4) The link association analysis module in the recording engine performs path analysis on the data, and partitions and persists the path information and related request and response information;

[0028] (5) When the response to the data processing request is detected to be finished, the request parameters and call response information in the recording link storage area are obtained through the link association module, the recorded traffic is identified, and stored in the target storage area. If the data does not match the expectations, the parameter configuration is adjusted and subsequent steps are executed.

[0029] Furthermore, the message processing task in step 3 includes:

[0030] Configure message extraction rules, including message header and message body information configuration;

[0031] The read operation processing subtask is used to retrieve the message to be processed from the corresponding storage medium based on the pre-set read operation configuration parameters;

[0032] The data preprocessing subtask is used to parse and replace the data obtained by the read operation processing subtask, and to extract the formatted data into relational data using Key-Val format.

[0033] The data processing subtask is used to process the Key-Val data based on the data results preprocessed by the data preprocessing subtask.

[0034] The data post-processing subtask is used to perform post-processing on the data processed by the data processing subtask.

[0035] The data write processing subtask is used to change the data format according to the pre-set rules and store it in the corresponding storage medium after the data has been processed by the read operation processing subtask, data preprocessing subtask, data processing subtask, and data postprocessing subtask.

[0036] Furthermore, the data preprocessing subtask parses and replaces the data obtained by the read operation processing subtask, which includes: according to the extraction mapping relationship configuration of the message header and message body, based on the syntax parsing rules of different types of messages, the data is parsed in a recursive loop, and the parsed data is extracted and converted into relational data in Key-Val format.

[0037] Furthermore, in step 5, for dual-transmission operations with strong inter-service dependencies, serial transmission is used for execution; for dual-transmission operations without inter-service dependencies, parallel transmission is used for execution.

[0038] Furthermore, step 5.2 specifically includes the following process:

[0039] When the business code of both environments is exactly the same, the request message and the return message are identical, and the request message and the return message of both environments are processed according to the same rules;

[0040] When business changes occur in either of the two environments, the request message and the response message need to be processed separately according to the business rules of the two environments.

[0041] When both the transmitting and recording environments undergo significant changes compared to the recording system, it is necessary to process the message formats and attributes contained in the messages in both transmitting environments according to the changed logical rules.

[0042] The present invention also provides a playback device for dual-transmission of traffic based on communication message recording, which is used to implement the playback method for dual-transmission of traffic based on communication message recording, including: a network message recording module, a message processing module, a message sending configuration and execution module, and a user interaction module;

[0043] The network packet recording module mainly captures and records network communication packets in real time at the network layer or based on the WLAN network card layer, and stores the recorded packets in local or remote storage media through a message caching mechanism;

[0044] The message processing module is used to parse and process recorded messages and static log messages. It extracts key information from the messages through read processing, data preprocessing, data processing, and data postprocessing, and stores the extracted information in the medium through data write processing.

[0045] The message sending configuration and execution module is mainly used to replay recorded messages or processed log messages in single or dual transmission according to user needs. It simulates sending under different basic application environments by configuring sending and reading processing, sending preprocessing, sending processing, and sending postprocessing. It is also used for real-time monitoring and statistical analysis of the replay results.

[0046] The user interaction web module provides a user-friendly interface, allowing users to set recording and playback parameters, view recording and playback progress in real time, and view and analyze test results.

[0047] The beneficial effects of this invention are as follows:

[0048] 1. This invention can capture and record network communication packets in real time, and parse and process static log packets, realizing comprehensive recording and playback of traffic and improving testing efficiency.

[0049] 2. This invention allows users to set recording and playback parameters according to their needs, supports multiple network protocols and log formats, and enables flexible testing of traffic.

[0050] 3. This invention, by simulating different network environments and monitoring playback results in real time, can comprehensively test the adaptability and stability of the software, thereby improving testing accuracy. Since no complex testing environment needs to be built, only one or more computers are required to complete traffic recording and playback testing, thus reducing testing costs. Attached Figure Description

[0051] Figure 1 The flowchart of the playback method for dual-transmission of traffic based on communication message recording provided by the present invention is shown.

[0052] Figure 2 This is a schematic diagram of the recording and dual-transmission playback process based on the recording engine. Detailed Implementation

[0053] The technical solutions provided by the present invention will be described in detail below with reference to specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention.

[0054] The device for dual-transmission of traffic based on communication message recording provided by the present invention mainly includes four modules: network message recording module, message processing module, message sending configuration and execution module, and user interaction module.

[0055] The network packet recording module is primarily used to capture and record network communication packets in real time at the network layer or based on the WLAN network card layer. The recorded packets are stored in local or remote storage media through a message caching mechanism. This module supports packet recording under various network protocols, such as HTTP (Hypertext Transfer Protocol), HTTPS (Hypertext Transfer Protocol Secure), TCP (Transmission Control Protocol), and UDP (Transmission Control Protocol). Furthermore, it allows users to set recording filtering conditions on demand through a rules engine, supporting dynamic adjustments based on different business scenarios to reduce unnecessary packet recording.

[0056] The message processing module is mainly used to parse and process recorded messages and static log messages. It extracts key information through read processing, data preprocessing, data processing, and data post-processing. It can also filter, sort, and statistically analyze log messages according to user needs. Simultaneously, it transmits the extracted information to the corresponding storage medium or message channel through data write processing. Specifically, the message preprocessing module implements step 3 of the method for playing back traffic from network layer recorded messages.

[0057] The message sending configuration and execution module is primarily used to replay recorded messages or processed log messages in single or dual transmissions according to user requirements. This module simulates sending under different basic application environments by configuring sending and reading processing, sending preprocessing, sending processing, and sending postprocessing, such as hardware resources, software resources, different software versions, different data volumes, and different application architectures, to comprehensively test the software's adaptability and stability. Simultaneously, this module also supports real-time monitoring and statistical analysis of the playback results, providing testers with intuitive test feedback.

[0058] The user interaction web module provides a user-friendly interface, allowing users to set recording and playback parameters, view recording and playback progress in real time, and view and analyze test results. This module supports multiple operating systems and platforms, such as Windows, Linux, and macOS, and boasts high scalability and customizability.

[0059] This invention provides a method for playing back traffic from two sources based on network layer recorded packets, the process of which is as follows: Figure 1 As shown, it includes the following steps:

[0060] Step 1: Basic Environment Debugging of the Packet Recording Engine. This step involves pre-setting basic information and preparing the basic environment for network layer packet recording, ensuring the recording engine can run smoothly in the monitored environment, and verifying communication and other related operations with the platform. This step mainly includes two core functions: recording parameter configuration and verification of the recording engine's compatibility and connectivity. Specifically, it includes the following sub-steps:

[0061] Step 1.1: Upload the message recording engine to the management platform through the user interaction web module.

[0062] Step 1.2: Through the user-interactive web module, establish and configure the service communication authentication information of the monitored service resource on the management platform, ensuring correct access to the corresponding permissions of the monitored service resource. For example, using SSH-password-based authentication: In this method, the SSH (Secure Shell) client needs to know the username and password, the corresponding port, and enable the corresponding access policy to establish a connection with the SSH server. Alternatively, using SSH-key-based authentication: In this method, the SSH client needs a key pair: a private key and a public key. The private key is stored on the client, and the public key is sent to the SSH server. Regardless of the authentication method used, the monitored end must have the corresponding access permissions enabled and the corresponding access policies configured.

[0063] Step 1.3: Based on the configuration information in Step 1.2 and after verifying the correct system connectivity authentication, the management platform distributes the message recording engine installation package to the monitored service resources, executes the recording engine installation operation (installing the recording engine service in the monitored resource service at this time), and starts the message recording engine (starting the recording engine service installed in the monitored resource at this time). The management platform monitors whether the recording engine service is running normally and performs periodic inspections on the engine. To ensure communication security, network security, and ease of operation, the installation, uninstallation, start, stop, and restart of the recording engine on the monitored server do not require direct access to the monitored service resources. A unified entry point enables one-stack management of the engine service, thereby avoiding the complexity of switching back and forth between multiple terminals and improving the convenience and efficiency of user operations.

[0064] Step 1.4: Configure parameters for the recording task, such as:

[0065] Message rules can be configured as needed in the following ways: ① Protocol filtering: Filter messages by specifying the protocol name. For example, to filter HTTP messages, simply enter "http"; ② IP address filtering: Filter messages by source IP address, destination IP address, or any IP address that meets preset conditions. Expressions include: ip.src==xx, ip.dst==xx, ip.addr==xx; ③ Port filtering: Filter messages by source port, destination port, or any port that meets preset conditions. Expressions include: tcp.srcport==xx, tcp.dstport==xx, tcp.port==xx; ④ MAC address filtering: Filter messages by MAC address that meets preset conditions. Expression includes: eth.addr==xx; ⑤ HTTP mode filtering: Filter messages containing GET or POST requests. Expressions such as: http.request.method == "GET" or http.request.method == "POST"; ⑥ Combined condition filtering: Logical operators such as AND (&&) and OR (||) can be used to combine multiple conditions. For example, a message that simultaneously satisfies the source IP address 192.168.0.9 and the protocol HTTP can use the expression: ip.src == 192.168.0.9 && http; ⑦ Custom filtering: Filtering messages based on regular expressions, etc.

[0066] Anomaly handling mechanism configuration (handling methods such as task continuation, skipping anomalies, and task interruption): Constraints are configured for scenarios involving anomalies during recording, such as rules for handling garbled messages and abnormal message formats. Based on relevant configuration information, the anomaly handling mechanism is continuously optimized and updated through anomaly information collection, cluster analysis of anomaly categories, and a continuous unsupervised learning process. This enables the device to achieve intelligent decision-making for configuration items (providing configuration suggestions for known and unknown anomaly scenarios), dynamic adjustment (dynamically adjusting corresponding configurations at runtime based on anomaly category analysis), and application to the current execution scenario. Simultaneously, the revised strategy is synchronized to the message center.

[0067] Recording environment configuration: Different environments may have different network configurations and traffic characteristics.

[0068] Recording Interface Configuration: Select the interface or URL to be recorded.

[0069] Sampling Configuration: Configure the sampling rate and sampling read strategy for the recorded traffic to control the amount and proportion of recorded traffic and the read strategy. A higher sampling rate results in recorded traffic that is closer to the actual traffic; a lower sampling rate results in less recorded traffic, but can reduce storage and processing pressure.

[0070] Request rewrite rule configuration: Rewrite the recorded requests.

[0071] Traffic storage configuration: Specifies the storage method and location for recorded traffic. Common storage methods include local file storage, database storage, and cloud storage.

[0072] Recording duration configuration: Set the recording duration for the specified data.

[0073] Concurrent recording configuration: Supports multiple concurrent recording tasks and other related parameter configurations.

[0074] Step 2: Execute the message recording task to obtain the required data. Capture and record network communication messages in real time at the network layer, and store the recorded messages in local or remote storage media. This includes the following sub-steps:

[0075] Step 2.1: Start the recording engine. Based on the verification of the normal execution of the recording engine service in Step 1.3 and the relevant parameter configurations in Step 1.4, such as message filtering rules, exception handling mechanism, recording environment, interface, sampling rate and strategy, request rewrite rules, traffic storage, recording duration, and whether to perform concurrent operations, create execution tasks and configuration settings on the management end. The configuration settings include task execution mode (manual, automatic), task execution time (specific time range, time range segment, etc.), task execution frequency (once, multiple times, loop), etc. The management platform can realize operations such as starting, stopping, terminating, and retransmitting tasks, and can also support breakpoint resume and task rollback related operations.

[0076] Step 2.2: Acquire traffic data and verify its correctness. During task execution, the acquired data can be monitored in real time. The logic for acquiring data and verifying its correctness is as follows:

[0077] 1. Data request reception: The message recording engine receives and parses the data processing request, extracting key metadata information such as the request message header and payload.

[0078] 2. The link association module in the message recording engine performs intelligent routing based on preset mapping rules, forwarding request messages to the specified address, dynamically allocating dedicated storage partitions, and persisting the request data. The link association module mainly implements routing mapping and request forwarding functions.

[0079] 3. The monitoring module in the recording engine monitors the calls to the corresponding intermediate components by data processing requests, records the call response information, and sends the information to the link association module. The monitoring module monitors request data and response data, request link nodes, processing time, success and failure status, and adds information such as traceId to the request header for the same request path.

[0080] 4. The link correlation analysis module in the recording engine performs path analysis on the data, partitions and persists the path information and related request-response information. The link correlation analysis module is used to perform dynamic intelligent analysis of link data and draw the complete call chain. Through unsupervised learning algorithms, it realizes intelligent monitoring and anomaly alarms.

[0081] 5. When the response to the data processing request is detected to be finished, the request parameters and call response information in the recording link storage area are obtained through the link association module. If the data does not match the expectations, the configuration information in step 1.4 is adjusted. For example, if there are unauthorized IP requests and responses in the data, the IP filtering policy in the configuration of 1.4 needs to be adjusted to see if it is effective or correct; if there is sensitive data that we do not expect in the message, we modify it by adjusting the regular expression in the configuration of 1.4. If it fully meets the expectations, the subsequent steps are executed.

[0082] Step 3: Execute message processing tasks, parse and process static log messages, extract key information, and store it in a specified database or file. This step cleanses the data obtained in Step 2 to meet the requirements of subsequent steps. The core of this step is to provide a complete processing framework (read operation processing, data preprocessing, data processing, data post-processing, data write operation processing, etc.), and then perform plug-in processing for different business needs. It can be appropriately tailored according to different application scenarios, allowing only a portion of the framework's functions to be executed as needed. Each component throughout the process achieves plug-and-play hot-swappability, thus adapting to changing environments. This step specifically includes the following sub-steps:

[0083] Step 3.1: Configure the rules for message generation, including message header and message body information configuration. This part supports automatic parsing of regular message formats such as XML and JSON. Based on the uploaded message template, it uses XPath and JsonPath syntax parsing and semantic parsing to parse it into a hierarchical tree format. Configure the response data and mapping rules according to the parsed message.

[0084] Step 3.2: Read Operation Processing Subtask. For the storage medium set in Step 1.4, which can be local files, remote shared files, relational databases like MySQL, distributed databases like TiDB, data warehouses like Hadoop, message queues (Kafka, RabbitMQ, etc.), etc., a customized read plugin can be developed. This plugin provides a standard interface; developers only need to implement the corresponding methods by inheriting the interface. This step's configuration is adapted to the sampling configuration parameters set in Step 1.4, such as file read strategy processing (e.g., line-by-line reading, block-by-block reading), file content format processing (JSON, XML, 8583 format), data encryption / decryption processing (symmetric encryption, asymmetric encryption), data encoding conversion (GBK, UTF-8, etc.), read strategy (serial read, parallel read, etc.), and the number of threads started. Based on the above configuration, executing this task can retrieve the required messages from the corresponding storage medium.

[0085] Step 3.3: Data Preprocessing Subtask. Combining the data processed in Step 3.2, this subtask performs real-time message parsing and replacement on the data processed in Step 3.2, such as JSON messages, XML messages, etc. According to the extraction mapping relationship configuration of message header and message body in 3.1, such as the parsing path corresponding to the transaction code path being " / Header / TranCode", and the parsing path corresponding to the channel path being " / Header / Channel", the formatted data is extracted in Key-Val format and converted into relational data.

[0086] Step 3.4: Data Processing Subtask. Based on the processing results of Step 3.3, customized data processing plugins are developed for different types of data. These plugins provide standard interfaces; developers only need to implement the corresponding methods by inheriting from these interfaces. The data processing plugin performs operations such as replacing Key-Val data based on the data results processed in Step 3.3. Replacement rules support fixed value replacement, interface replacement, custom function replacement, and random number replacement, among others.

[0087] Step 3.5: Data post-processing subtask. Data post-processing performs various business-specific post-processing on the data results processed in Step 3.4, such as data type processing, data format processing, and handling of null values.

[0088] Based on steps 3.2-3.5 above, the processed log messages can also be filtered, sorted, and statistically analyzed according to user needs.

[0089] Step 3.6: Data Write Processing Subtask. The post-processing operation mainly configures how to store the results from steps 3.2, 3.3, 3.4, and 3.5, and the storage strategy. For example, it can perform hash sharding based on the key value, sharding based on request time, or sharding based on transaction code, etc., meaning the data can be processed according to pre-defined rules. Furthermore, to improve execution efficiency, the processed results can be flexibly defined and stored on different media, such as local files, remote shared files, relational databases like MySQL, distributed databases like TiDB, data warehouses like Hadoop, and message queues (Kafka, RabbitMQ, etc.).

[0090] Step 4: Preset the relevant parameters required for playback, including playback type (single or dual transmission), playback speed, network environment, etc. This step configures the relevant parameters for the playback scenario ("Whether to perform a one-sided comparison", "Whether to test in the new environment", "Whether to test in the old environment"). By configuring the three core parameters "Whether to perform a one-sided comparison", "Whether to test in the new environment", and "Whether to test in the old environment", the following scenarios are commonly controlled: single-environment correctness and stability verification scenario (this scenario is mainly applicable to the technical and business correctness verification of the system under test. The business level verification is based on the business data verification of the results returned by the system under test and the recorded data return results), dual-environment connectivity verification scenario (this scenario is mainly applicable to the technical correctness verification of the system under test), and dual-transmission scenario (when "Whether to perform a one-sided comparison", "Whether to test in the new environment", and "Whether to test in the old environment" are set to "No", the current configuration task will be performed according to the dual-transmission logic. This scenario is mainly applicable to the business level correctness verification of the dual-transmission system. The business level verification is based on the business data verification of the real-time return results of the dual-transmission system); playback speed configuration, playback compression ratio parameter configuration, latency parameter configuration, network environment, etc. This process, combined with step 1.4, configures the access addresses of both sending targets, request methods (HTTP request, HTTPS request, Socket request, etc.), message formats (XML format, JSON format, 8583 format, etc.), message encoding (GBK, UTF-8, etc.), and other related settings and response code settings.

[0091] Step 5: Execute the message replay module. The core function of message replay is to construct a request message for the system under test based on the message data processed in Step 3.6 and the specifications of the request message received by the system under test. Depending on the set replay parameters, single or dual replay is performed using the newly constructed request message.

[0092] When executing the message dual-send task, based on the dual-send scenario configuration in step 4 (when "Whether to compare one side" is set to "No", "Whether to test in the new environment" is set to "No", and "Whether to test in the old environment" is set to "No", the current configuration task will proceed according to the dual-send logic) and other parameters, configure the relevant execution tasks and start replaying the sending request. Monitor the dual-send execution results during the process.

[0093] When dual-transmission traffic is required, the recorded traffic data is read from the target storage area of ​​the processing results in step 3.5. The read traffic data is then sent to the dual-transmission environments (environment A and environment B) for dual-transmission playback testing (based on real-world system data collection, simulated transmission to environment A and environment B at the same time, ultimately verifying the consistency of the results returned by environment A and environment B, and finally judging the consistency and correctness of the system processing logic). This process is subdivided into five major processes: read processing, transmission preprocessing, transmission processing, transmission postprocessing, and transmission write process. The key point is that for dual services with strong dependencies, serial execution is required; for dual-transmission operations without dependencies between services, parallel transmission can be used.

[0094] Step 5.1: Read processing subtask, handles the source and reading strategy of dual-transmission message data. This step reads the data written in step 3.6. The logic and method of reading the data are the same as in step 3.2.

[0095] Step 5.2: Sending Preprocessing. Based on the data processed in Step 5.1, sending preprocessing mainly involves assembling the sending message, defining the sending message format, and replacing the message to be sent. There are three typical application scenarios: Scenario 1: In environments A and B with identical business code, the request and response messages are the same. In this case, only the request and response messages in both environments need to be processed according to a unified rule. Scenario 2: In cases where business changes occur in either environment A or B, the request and response messages need to be processed separately according to the business rules of environment A and environment B. Since only environment B has a business change, environment A only needs to simply replace specific attribute values ​​in the message, while environment B needs to reassemble the message format and the attributes contained in the message. Scenario 3: For scenarios where both Environment A and Environment B undergo significant changes compared to the recording system, it is necessary to process the message formats and attributes contained in the messages of Environment A and Environment B according to the modified logical rules. For example, if the request message in Environment A changes from an XML message to a JSON message, or if the request parameters in Environment B change, properties may be added or removed, or data objects may be added or removed.

[0096] Step 5.3: Sending Processing. Sending processing refers to steps 4, 5.1, and 5.2, preparing to send the task twice.

[0097] Step 5.4: Post-send processing. Based on the sending situation in Step 5.3, the returned data of the two requests are processed, including parsing and extracting the message format of the returned message.

[0098] Step 5.4: Send Write. For different types of data, customized write data plugins are developed. These plugins provide standard interfaces, and developers only need to implement the corresponding methods by inheriting from these interfaces. The send write operation mainly implements operations on the storage objects containing the results processed in steps 5.1, 5.2, 5.3, and 5.4. Storage media include local files, remote shared files, relational databases (MySQL), distributed databases (TiDB), data warehouses (Hadoop), and message queues (Kafka, RabbitMQ, etc.).

[0099] Step Six: Execution Result Analysis. Monitor the replay progress and results in real time. View and analyze the test results through the user interaction module, adjust the replay parameters, and re-perform the replay test until the test requirements are met. This step verifies the replay results of the two-way transmission in Step 5 according to the expected goals. It verifies the correctness of the two-sided transmission results by combining the verification rules and standards of the actual business scenario. For example, it judges the acceptability of the two-sided system by verifying the 100% matching, verifies the acceptance of the two-sided system matching degree by verifying the return code, and verifies whether the matching degree of the two-sided system meets the expected goals by verifying the key attributes in the return message.

[0100] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. A method for playback of dual-transmission traffic based on network layer recorded packets, characterized in that, Includes the following steps: Step 1: Debug the basic environment of the message recording engine, enable access permissions and configure access policies on the monitored end, and install the message recording engine after the system connectivity authentication is passed. Configure parameters for the recording task, including message filtering rule configuration and recording, playback and storage related parameter configuration. Step 2: Execute the message recording task to obtain the required data, capture and record network communication messages in real time at the network layer, and store the recorded messages in local or remote storage media; Step 3: Execute message processing tasks, parse and process static log messages, extract key information, and store it in a specified database or file; Step 4: Set the relevant parameters required for playback, including playback type, playback speed, and network environment; Step 5: Based on the message data processed in Step 3 and the request message specifications of the system under test, construct the request message of the system under test; according to the parameters set in Step 4 and the replay-related parameter configuration in Step 1, replay the constructed request message of the system under test in one or two transmissions. When performing dual-transmission playback, the data processed and stored in step 3 is read and sent to the dual-transmission environment for playback testing, specifically including: Step 5.1, Read processing: Based on the pre-set read operation configuration parameters, retrieve the message to be processed from the corresponding storage medium; Step 5.2, Preprocessing: Based on the data read in Step 5.1, assemble and send the message according to the defined message format; Step 5.3: Sending Processing. Based on the parameters set in Step 4, send the message twice using the message processed in Step 5.

2. Step 5.4: Post-sending processing. Based on the sending situation in Step 5.3, process the data returned by both requests. Step 5.4: Send write, which stores the data processed in Step 5.4; design corresponding read plugins for different types of data and provide standard interfaces; Step 6: Execute result analysis, monitor the playback progress and results in real time, view and analyze the test results, adjust the playback parameters and re-perform the playback test until the test requirements are met.

2. The method for playback of dual-transmission traffic based on network layer recorded packets according to claim 1, characterized in that, In step 1, the message recording engine is uploaded to the management platform, and the management platform performs operations on the message recording engine on the monitored server.

3. The method for playback of dual-transmission traffic based on network layer recorded packets according to claim 1, characterized in that, In step 1, the message filtering rules include: protocol filtering rules, IP address filtering rules, port filtering rules, MAC address filtering rules, HTTP mode filtering rules, and custom filtering rules.

4. The method for playback of dual-transmission traffic based on network layer recorded packets according to claim 1, characterized in that, Step 2 specifically includes the following sub-steps: Step 2.1: After the system connectivity authentication is passed, configure the relevant execution tasks of the message recording engine on the management platform based on the parameter configuration, and start the message recording engine; Step 2.2: Obtain traffic data through the message recording engine and verify the data correctness.

5. The method for playback of dual-transmission traffic based on network layer recorded packets according to claim 4, characterized in that, Step 2.2 specifically includes the following steps: (1) Data request reception: The message recording engine receives and parses the data processing request, and extracts the key metadata information in the request; (2) The link association module in the message recording engine performs intelligent routing according to the pre-set mapping rules, forwards the request message to the specified address, dynamically allocates a dedicated storage partition and persists the request data; (3) The monitoring module in the message recording engine monitors the data processing requests calling their respective intermediate components, records the call response information, and sends the information to the link association module; (4) The link association analysis module in the recording engine performs path analysis on the data, and partitions and persists the path information and related request and response information; (5) When the response to the data processing request is detected to be finished, the request parameters and call response information in the recording link storage area are obtained through the link association module, the recorded traffic is identified, and stored in the target storage area. If the data does not match the expectations, the parameter configuration is adjusted and subsequent steps are executed.

6. The method for replaying dual-transmission traffic based on network layer recorded packets according to claim 1, characterized in that, The message processing task in step 3 includes: Configure message extraction rules, including message header and message body information configuration; The read operation processing subtask is used to retrieve the message to be processed from the corresponding storage medium based on the pre-set read operation configuration parameters; The data preprocessing subtask is used to parse and replace the data obtained by the read operation processing subtask, and to extract the formatted data into relational data using Key-Val format. The data processing subtask is used to process the Key-Val data based on the data results preprocessed by the data preprocessing subtask. The data post-processing subtask is used to perform post-processing on the data processed by the data processing subtask. The data write processing subtask is used to change the data format according to the pre-set rules and store it in the corresponding storage medium after the data has been processed by the read operation processing subtask, data preprocessing subtask, data processing subtask, and data postprocessing subtask.

7. The method for playback of dual-transmission traffic based on network layer recorded packets according to claim 6, characterized in that, The data preprocessing subtask parses and replaces the data obtained by the read operation processing subtask. This process includes: parsing the data in a recursive loop based on the extraction mapping relationship configuration of the message header and message body, and the syntax parsing rules of different message types, and then extracting the parsed data in Key-Val format and converting it into relational data.

8. The method for playback of dual-transmission traffic based on network layer recorded packets according to claim 1, characterized in that, In step 5, for dual-transmission operations with strong inter-service dependencies, serial transmission is used; for dual-transmission operations without inter-service dependencies, parallel transmission is used.

9. The method for replaying dual-transmission traffic based on network layer recorded packets according to claim 1, characterized in that, Step 5.2 specifically includes the following process: When the business code of both environments is exactly the same, the request message and the return message are identical, and the request message and the return message of both environments are processed according to the same rules; When business changes occur in either of the two environments, the request message and the response message need to be processed separately according to the business rules of the two environments. When both the transmitting and recording environments undergo significant changes compared to the recording system, it is necessary to process the message formats and attributes contained in the messages in both transmitting environments according to the changed logical rules.

10. A playback device for dual-transmission of traffic based on communication message recording, characterized in that, The method for implementing the dual-transmission traffic method based on communication message recording as described in any one of claims 1-9 comprises: a network message recording module, a message processing module, a message sending configuration and execution module, and a user interaction module; The network packet recording module mainly captures and records network communication packets in real time at the network layer or based on the WLAN network card layer, and stores the recorded packets in local or remote storage media through a message caching mechanism; The message processing module is used to parse and process recorded messages and static log messages. It extracts key information from the messages through read processing, data preprocessing, data processing, and data postprocessing, and stores the extracted information in the medium through data write processing. The message sending configuration and execution module is mainly used to replay recorded messages or processed log messages in single or dual transmission according to user needs. It simulates sending under different basic application environments by configuring sending and reading processing, sending preprocessing, sending processing, and sending postprocessing. It is also used for real-time monitoring and statistical analysis of the replay results. The user interaction module provides a user-friendly interface, allowing users to set recording and playback parameters, view recording and playback progress in real time, and view and analyze test results.

Images