Implementation method for traffic recording and playback

Abstract

The invention discloses a method for realizing traffic recording and playback. The method comprises the following steps of: recording online traffic, and performing mock comparison in a full-link playback and playback process. Online traffic recording is carried out at the calling end and the called end, so that full-link playback can be realized; downloading to the local for playback, and acquiring data of each link in the recording period according to traceId in the playback process; according to the method, flexible expansion is carried out through scripts, the problem of refined management and control in the conventional traffic recording and playback process is solved, mock comparison can be carried out in the playback process, mock dependence is adjusted at a calling end in the playback process, data of all links in the recording period are obtained according to traceId in the playback process, more comprehensive mock or comparison is carried out, and the efficiency of recording and playback is improved. And compared with other recorded playback finished products, the playback scene is more real.

Description

Technical Field

[0001] The present invention relates to a method for flow recording and playback, in particular to a method for realizing flow recording and playback, belonging to the technical field of performance stress testing. Background Art

[0002] Generally speaking, during the iteration process of an application, a lot of testing work needs to be done on the new version that is launched. However, it is difficult to ensure that the manually constructed test data covers all online scenarios. On the one hand, in the stress test scenario, manually constructed data cannot stress test the performance indicators of the real online version; on the other hand, the upstream and downstream dependencies of the application increase the complexity of the test.

[0003] R2 (Record & Replay) is a network security tool that can provide traffic recording, functional testing, stress testing, Mock and other functions. It can be used to simulate and reproduce actual network traffic to facilitate network security testing, vulnerability analysis and production problem reproduction, and solve user pain points in backend business system testing scenarios.

[0004] In the prior art, a traffic playback method disclosed in publication number CN110474820A is applied to the traffic initiator to record the online business traffic and obtain the recorded traffic; determine the dependency between the business requests in the recorded traffic, and divide the business requests in the recorded traffic into several traffic groups based on the dependency; perform traffic playback for the business requests in several traffic groups respectively, divide the recorded traffic into several traffic groups, and perform traffic playback for the business requests in several traffic groups respectively; thereby achieving stateful traffic simulation and playback, improving test efficiency, and reducing labor costs and system overhead. However, the problems existing in the prior art are: first, it can only play back read operations, but not write operations, does not have full-link characteristics, and has poor scalability; second, it cannot fully support all business test scenarios, that is, it cannot allow users to inject custom scripts to expand all functions in the recording and playback process, which leads to the playback scene being not realistic enough and the inability to perform playback operations flexibly. Summary of the invention

[0005] The purpose of the present invention is to provide a method for implementing traffic recording and playback in order to solve at least one of the above technical problems.

[0006] The present invention achieves the above-mentioned purpose through the following technical solutions: a method for realizing traffic recording and playback, including a traffic recording and playback tool, the traffic recording and playback tool provides traffic recording, functional testing, stress testing and Mock functions, and the traffic recording and playback tool allows users to inject custom scripts to expand all functions in the traffic recording and playback process;

[0007] The implementation method includes the following steps:

[0008] Step 1: Record online traffic: The traffic recording and playback tool records online traffic on the calling end and the called end. Recording on the called end is used for playback testing, including functional testing and stress testing; recording on the calling end is used for mocking or comparison during playback.

[0009] Step 2: Full-link playback: After the online traffic is recorded, it is downloaded to the local computer for playback. During the playback process, the data of each link during the recording period is obtained according to the traceId.

[0010] Step 3: Playback process for mock comparison: During the playback process, mock the dependency on the calling end.

[0011] As a further solution of the present invention: in step one, when the traffic recording and playback tool records online traffic, there are two modes: zero-intrusive recording and intrusive recording; wherein, zero-intrusive recording refers to zero intrusion on user code and configuration, and intrusive recording refers to the intrusion of functional logic into process operation.

[0012] As a further solution of the present invention: in step 2, the download of the online traffic after recording can be downloaded from an external node or from the callee itself.

[0013] As a further solution of the present invention: in step 3, the calling end mock call dependency includes but is not limited to mock call RPC and mock call MySQL;

[0014] Among them, RPC is a remote procedure call protocol service, and MySQL is a relational database management system.

[0015] As a further solution of the present invention: the traffic recording and playback tool supports write traffic playback, and the traffic recording and playback tool mocks the write operation API in the playback process, and the mock write operation API includes the RPC interface at the service level and the API at the storage middleware level, that is: write traffic playback is supported through the mock at the middleware level; wherein the API is an application program interface.

[0016] As a further solution of the present invention: the traffic recording and playback tool is based on the full link, and the traffic recording and playback tool is combined with Pfinder. In the process of recording online traffic, traceId is carried to completely record the call link within an application or even the call link across applications, which specifically includes:

[0017] Display the complete call chain;

[0018] Configure recording, playback, mock, and comparison rules on the visualization link;

[0019] Store traffic data using traceId as clue;

[0020] Retrieve data based on traceId for playback and comparison.

[0021] As a further solution of the present invention: the traffic recording and playback tool supports noise processing, which allows users to specify field names, and these fields will not be compared during comparison, and the traffic recording and playback tool also supports the injection of comparison scripts.

[0022] The beneficial effects of the present invention are as follows: the present invention can record online traffic at the calling end and the called end, and the recording at the called end is used for playback testing, including functional testing and stress testing; the recording at the calling end is used for mocking or comparison during playback, and full-link playback can be achieved. After the traffic is recorded, it is downloaded to the local for playback, and data of each link in the recording period is obtained according to traceId during the playback process. The traffic recording and playback tool adopted by the present invention allows the user to inject custom scripts to expand all functions in the recording and playback process, and flexibly expands through scripts to solve the problem of refined management and control in the conventional traffic recording and playback process. Mock comparison can also be performed during the playback process. During the playback process, mock dependency is called at the calling end to obtain data of each link in the recording period according to traceId during the playback process, thereby performing a more comprehensive mock or comparison. Compared with other recording and playback products, the playback scene is more realistic. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is the Record flow chart of the traffic recording and playback tool of the present invention;

[0024] Figure 2 It is a Replay flow chart of the traffic recording and playback tool of the present invention;

[0025] Figure 3 This is a structural module architecture diagram of the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] Embodiment 1, as Figures 1 to 3As shown, this embodiment provides a method for implementing traffic recording and playback, including a traffic recording and playback tool with traffic recording, functional testing, stress testing and Mock functions, and the traffic recording and playback tool allows the user to inject a custom script to extend all functions in the traffic recording and playback process;

[0028] The implementation method includes the following steps:

[0029] First, record online traffic: The traffic recording and playback tool records online traffic on the calling end and the called end. Recording on the called end is used for playback testing, including functional testing and stress testing; recording on the calling end is used for mocking or comparison during playback.

[0030] When recording online traffic, the traffic recording and playback tool has two modes: zero-intrusive recording and intrusive recording. Among them, the zero-intrusive recording refers to zero intrusion on user code and configuration, and the intrusive recording refers to the intrusion of functional logic into process operation.

[0031] Second, full-link playback: After the online traffic is recorded, it is downloaded to the local computer for playback, and the data of each link during the recording period is obtained according to the traceId during the playback process;

[0032] The online traffic recording is downloaded from an external node or from the callee itself.

[0033] Third, perform mock comparison during playback: During playback, mock the dependency on the calling end;

[0034] The mock call dependencies on the calling end include but are not limited to mock calls to RPC and mock calls to MySQL; RPC is a remote procedure call protocol service, and MySQL is a relational database management system.

[0035] The traffic recording and playback tool supports write traffic playback, and the traffic recording and playback tool mocks the write operation API in the playback process. The mock write operation API includes the RPC interface at the service level and the API at the storage middleware level; wherein the API is an application program interface.

[0036] The traffic recording and playback tool is based on the entire link, and the traffic recording and playback tool is combined with Pfinder. During the recording of online traffic, the traceId is carried to completely record the call link within an application or even the call link across applications, including:

[0037] Display the complete call chain;

[0038] Configure recording, playback, mock, and comparison rules on the visualization link;

[0039] Translate traceId to store traffic data for the clue;

[0040] Retrieve data based on traceId for playback and comparison.

[0041] The traffic recording and playback tool supports noise processing, which allows the user to specify field names and not compare these fields during comparison. The traffic recording and playback tool also supports the injection of comparison scripts.

[0042] Example 2: Assume that there is a system A. We want to know the system capacity value TPM so that we know how many machines need to be deployed online. We also want to know whether the system can run stably under this pressure. We need to do online traffic pressure testing, record the traffic and then play it back offline to get a more accurate capacity.

[0043] Step 1: Traffic modeling: First, we still need to figure out what business scenarios system A provides, what is the call ratio of each interface, and what is the pressure trend of each interface. By modeling and analyzing the business volume of the online system, we can get the interface call ratio of the system over a period of time. We select the main 4 interfaces (accounting for about 95% of the total pressure flow)

[0044] Step 2: Business modeling: We need to analyze the corresponding business model. For example, if two business scenarios or pages call this interface, we need to separate the call volume of each scenario or page and write a stress test script for each scenario or page. Of course, if it is a backend system, you can also directly record the traffic of these interfaces.

[0045] Step 3: Set two values ​​for the full-link coloring mark. Both are used for traffic recording, one for full-link read traffic recording and the other for log playback.

[0046] (1) t = 1, bypassing the full-link read traffic logic

[0047] (2) T = 2, when replaying logs, some business tracking points are bypassed (to avoid affecting online business data statistics)

[0048] if((“1”).equals(Trans.getUserData(“t”))){

[0049] / / TODO reading traffic recording process

[0050] }else{

[0051] / / TODO follow the normal process

[0052] }

[0053] if((“2”).equals(Trans.getUserData(“t”))){

[0054] / / When replaying TODO logs, bypass some business point statistics to reduce the impact of statistical data on the online

[0055] }else{

[0056] / / TODO follow the normal process

[0057] }

[0058] Step 4: Perform mock comparison during playback. During playback, mock the dependency on the calling end. Download the program to the local computer for playback, and obtain data from each link of the recording period according to traceId during playback.

[0059] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0060] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A method for realizing traffic recording and playback, characterized in that: It includes a traffic recording and playback tool with traffic recording, functional testing, stress testing and mocking functions, and the traffic recording and playback tool allows users to inject custom scripts to expand all functions in the traffic recording and playback process; The implementation method comprises the following steps: Step 1: Record online traffic: The traffic recording and playback tool records online traffic at the calling end and the called end. The recording at the called end is used for playback testing, including functional testing and stress testing; the recording at the calling end is used for mocking or comparison during playback. Step 2: Full-link playback: After the online traffic is recorded, it is downloaded to the local computer for playback. During the playback process, the data of each link during the recording period is obtained according to the traceId. Step 3: Playback process for mock comparison: During the playback process, mock the dependency on the calling end.

2. The implementation method according to claim 1, characterized in that: In the step 1, the traffic recording and playback tool has two modes when recording online traffic: zero-intrusive recording and intrusive recording; wherein, the zero-intrusive recording refers to zero intrusion on user code and configuration, and the intrusive recording refers to the intrusion of functional logic into process operation.

3. The implementation method according to claim 1, characterized in that: In the step 2, the online traffic is recorded and downloaded from an external node or from the callee itself.

4. The implementation method according to claim 1, characterized in that: In step 3, the calling end mock call dependencies include but are not limited to mock call RPC and mock call MySQL; Among them, RPC is a remote procedure call protocol service, and MySQL is a relational database management system.

5. The implementation method according to claim 1, characterized in that: The traffic recording and playback tool supports write traffic playback, and the traffic recording and playback tool mocks the write operation API in the playback process, and the mock write operation API includes the RPC interface at the service level and the API at the storage middleware level; Among them, API is an application programming interface.

6. The implementation method according to claim 1, characterized in that: The traffic recording and playback tool is based on the entire link, and is combined with Pfinder. During the recording of online traffic, the traceId is carried to completely record the call link within an application or even the call link across applications, including: Display the complete call chain; Configure recording, playback, mock, and comparison rules on the visualization link; Translate traceId to store traffic data for the clue; Retrieve data based on traceId for playback and comparison.

7. The implementation method according to claim 1, characterized in that: The traffic recording and playback tool supports noise processing, which allows the user to specify field names and not compare these fields during comparison. The traffic recording and playback tool also supports the injection of comparison scripts.

Images