Performance test method, device, medium and system based on flow playback

Abstract

The invention provides a performance test method, device and system based on flow playback, and a medium. According to the method, a traffic compensation strategy based on time fragmentation is adopted to ensure that the number of data streams meets test requirements, a token updating mode is adopted to ensure that authentication information in the data streams is continuous and effective, the availability of traffic is enhanced, the traffic value is ensured, the multiplexing value of the traffic is improved through mechanisms of parameterization, authentication and traffic compensation, and the service life of the traffic is prolonged. According to the method, the effectiveness and the applicability of the grabbed flow are ensured, script modification work caused by using different pressure measurement tools is avoided, the test efficiency is improved, and compared with an existing scheme, the test result in the performance test process and the performance index during actual operation do not have deviation exceeding the specified standard, and the test efficiency is improved. Therefore, the problem of deviation between the test result and the performance index during actual operation in the performance test process of the existing scheme is solved.

Description

Technical Field

[0001] This application relates to the technical field of system performance testing. Specifically, it relates to a performance testing method, device, medium, and system based on traffic playback. Background Art

[0002] Traditional system performance testing methods include: performance testing methods based on recording and playback, performance testing methods based on scripts, and performance testing methods based on traffic playback.

[0003] The performance testing method based on recording and playback mainly uses automated testing tools such as LoadRunner, etc., to record the system user operations simulated by software on the client side, and then records the operation process into a script, and performs performance testing on the system by executing the script. This method is simple to operate and can quickly implement performance testing, but it can only test the performance of the system front end and cannot comprehensively understand the processing capabilities of the entire system architecture. And during the testing process, the system load level must be controlled to ensure the normal operation of the system, so this testing method has high requirements for the testing environment, and there are deviations between the test results and the performance indicators during actual operation.

[0004] The performance testing method based on scripts mainly uses automated testing tools such as JMeter and LoadRunner, and simulates a large number of system user operations by writing scripts to perform performance testing on the system. The test results can be achieved by sending the test script to the target system or by running the test script on the local system. This method can test the performance of both the system front end and the back end, but in the process of writing the script, it is necessary to have an in-depth understanding of the test system architecture, and the process of writing, debugging, modifying, and optimizing the test script is relatively cumbersome, the test efficiency is low, and at the same time, the reusability of the test script is poor.

[0005] The performance testing method based on traffic playback mainly conducts system function testing through a recording tool, and then uses the recorded data stream of the system function testing to simulate system user operations to perform performance testing on the system. This method can comprehensively understand the processing capabilities of the entire system architecture by directly simulating system user operations, and at the same time, the test efficiency is relatively high. However, this method has high requirements for the format of the data stream of the system function testing, and there are deviations between the test results and the performance indicators during actual operation.

[0006] That is, there are deviations between the test results and the performance indicators during actual operation in the existing solutions for performance testing. Summary of the Invention

[0007] The main purpose of this application is to provide a performance testing method, device, medium, and system based on traffic playback, so as to at least solve the problem that there are deviations between the test results and the performance indicators during actual operation in the existing solutions for performance testing.

[0008] To achieve the above object, according to one aspect of the present application, a performance testing method based on traffic replay is provided. The method includes: capturing data streams from a real-time trading system or historical data, serializing and encapsulating the data streams using a tool class in a common component, and storing the serialized and encapsulated data streams in a storage component, where the storage component includes a distributed file system, cloud storage, and a local database; adopting a traffic compensation strategy based on time slicing to ensure that the number of data streams meets the test requirements, and adopting the method of updating tokens to ensure the continuous validity of the authentication information in the data streams. The traffic compensation strategy is selected from data replay, data generation, or historical data extraction. The token is encapsulated in the form of userid:value, and the userid is the unique identifier of the user; converting the test script of the data stream into a script format recognizable by an automated testing tool based on a JMeter or Postman script template; using a multi-node distributed deployment method, sending the test script after format conversion to a target system for performance testing through the NIO communication framework of a replay engine component, collecting the response data returned by the replay engine nodes by a management component through a real-time messaging mechanism, and performing analysis of performance metrics using a time slicing algorithm, where the performance metrics include response time, throughput, and TPS.

[0009] According to another aspect of the present application, a performance testing device based on traffic replay is provided. The device includes: a first processing unit for capturing data streams from a real-time trading system or historical data, serializing and encapsulating the data streams using a tool class in a common component, and storing the serialized and encapsulated data streams in a storage component, where the storage component includes a distributed file system, cloud storage, and a local database; a second processing unit for adopting a traffic compensation strategy based on time slicing to ensure that the number of data streams meets the test requirements, and adopting the method of updating tokens to ensure the continuous validity of the authentication information in the data streams. The traffic compensation strategy is selected from data replay, data generation, or historical data extraction. The token is encapsulated in the form of userid:value, and the userid is the unique identifier of the user; a third processing unit for converting the test script of the data stream into a script format recognizable by an automated testing tool based on a JMeter or Postman script template; a fourth processing unit for using a multi-node distributed deployment method, sending the test script after format conversion to a target system for performance testing through the NIO communication framework of a replay engine component, collecting the response data returned by the replay engine nodes by a management component through a real-time messaging mechanism, and performing analysis of performance metrics using a time slicing algorithm, where the performance metrics include response time, throughput, and TPS.

[0010] According to another aspect of the present application, there is provided a computer-readable storage medium, which includes a stored program. When the program runs, it controls the device where the computer-readable storage medium is located to execute any one of the above methods.

[0011] According to another aspect of the present application, there is provided a performance testing system based on traffic replay, including: one or more processors, a memory, and one or more programs. The one or more programs are stored in the memory and are configured to be executed by the one or more processors. The one or more programs include those for executing any one of the above methods.

[0012] Applying the technical solution of the present application, adopting a traffic compensation strategy based on time slicing to ensure that the number of data streams meets the test requirements, and adopting the method of updating tokens to ensure the continuous validity of the authentication information in the above data streams, enhancing the availability of traffic, guaranteeing the value of traffic. Through the mechanisms of parameterization, authentication, and traffic compensation, the reuse value of traffic is improved, the effectiveness and applicability of the captured traffic are ensured, the script modification work caused by using different stress testing tools is avoided, and the test efficiency is improved. Compared with the existing solutions, there will be no deviation exceeding the specified standard in the test results and the performance indicators during actual operation in the process of performance testing, thus solving the problem that there is a deviation between the test results and the performance indicators during actual operation in the process of performance testing in the existing solutions. Description of the Drawings

[0013] The schematic diagrams in the specification forming a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0014] Figure 1 It shows a schematic flow diagram of a performance testing method based on traffic replay provided according to an embodiment of the present application;

[0015] Figure 2 It shows a schematic framework diagram of a performance testing device based on traffic replay provided according to an embodiment of the present application;

[0016] Figure 3 It shows a schematic principle diagram of the NIO communication framework involved provided according to an embodiment of the present application;

[0017] Figure 4 It shows a schematic flow diagram of the script conversion function provided according to an embodiment of the present application;

[0018] Figure 5 It shows a schematic main function implementation flow diagram of the traffic control function provided according to an embodiment of the present application;

[0019] Figure 6 Shows a schematic diagram of the implementation process of the traffic compensation function of the traffic control function provided according to an embodiment of the present application;

[0020] Figure 7 Shows a schematic diagram of the implementation process of the updated traffic authentication function of the traffic control function provided according to an embodiment of the present application;

[0021] Figure 8 Shows a schematic diagram of the implementation process of the data collection and calculation function provided according to an embodiment of the present application;

[0022] Figure 9 Shows a schematic diagram of the implementation process of the playback executor function provided according to an embodiment of the present application;

[0023] Figure 10 Shows a schematic diagram of the implementation process of the result processor function provided according to an embodiment of the present application;

[0024] Figure 11 Shows a schematic diagram of the core implementation process of the plug-in template code provided according to an embodiment of the present application;

[0025] Figure 12 Shows a structural block diagram of a performance test device based on traffic playback provided according to an embodiment of the present application. Detailed implementation manners

[0026] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0027] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without making creative efforts shall fall within the protection scope of the present application.

[0028] It should be noted that the terms "first", "second", etc. in the specification, claims and the above-mentioned drawings of this application are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so as to implement the embodiments of this application described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0029] For the convenience of description, some nouns or terms related to the embodiments of this application are described below:

[0030] The vuser threshold refers to the upper limit of the number of virtual users simulated in a load test. When this threshold is reached or exceeded, the system may experience performance problems or crashes. Setting an appropriate vuser threshold can help testers evaluate the performance and stability of the system, so as to timely discover and solve potential problems.

[0031] Traffic replay: By technical means, the requests sent to a certain system in a set of environments are resent to the system in the current environment in the way of direct forwarding or resending after saving, or are sent to the system in other environments, so that no matter what environment the system is in, it can receive the resent request information, and the resent request is consistent with the original request content.

[0032] Jmeter: It is the abbreviation of Apache JMeter, which is a stress test tool developed by the Apache organization based on Java and is used to perform stress tests on software.

[0033] Postman: It is a web debugging and interface testing tool developed by Google, which can send any type of http request and supports methods such as GET / PUT / POST / DELETE.

[0034] TPS: The abbreviation of Transaction Per Second, which represents the number of transactions processed per second and is one of the performance indicators expressing the processing ability of the system.

[0035] Loadrunner: It is a load test tool that predicts system behavior and performance.

[0036] Component level: Refers to the same level of functional modules. If it is a microservice architecture, one module is one component.

[0037] Interface level: For requests using the Dubbo protocol, it refers to the interface defined by the keyword "interface", which is the interface name; for requests using the HTTP / HTTPS protocol, it refers to the interface request type provided externally, and for other protocols, it is customized according to the protocol template.

[0038] Method level: For requests using the Dubbo protocol, it refers to the specific member methods contained in the interface defined by the keyword "interface"; for requests using the HTTP / HTTPS protocol, it refers to the specific URL path provided externally.

[0039] NIO communication framework: It is a network communication framework developed based on NIO (Nonblocking / 0, non-blocking 10), and common frameworks such as Netty, etc.

[0040] As introduced in the background technology, the performance testing method based on traffic replay mainly conducts system function testing through a recording tool, and then uses the recorded data stream of the system function testing to simulate the operations of system users to perform performance testing on the system. This method can comprehensively understand the processing capabilities of the entire system architecture by directly simulating the operations of system users, and at the same time, the testing efficiency is relatively high. However, this method has high requirements for the format of the data stream of the system function testing, and there are deviations between the test results and the performance indicators during actual operation. To solve the problem that there are deviations between the test results and the performance indicators during actual operation in the performance testing process of the existing solutions, the embodiments of the present application provide a performance testing method, device, medium, and system based on traffic replay.

[0041] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention.

[0042] In this embodiment, a performance testing method based on traffic replay is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0043] Figure 1 It is a schematic flowchart of a performance testing method based on traffic replay provided according to the embodiments of the present application. As Figure 1 shown, the method includes the following steps:

[0044] Step S101, capture the data stream from the real-time trading system or historical data, serialize and encapsulate the above data stream using the tool class in the common component, and then store it in the storage component. The above storage component includes a distributed file system, cloud storage, and a local database;

[0045] Step S102, adopt a traffic compensation strategy based on time slicing to ensure that the number of data streams meets the test requirements, and adopt the method of updating tokens to ensure the continuous validity of the authentication information in the above data streams. The above traffic compensation strategy is selected from data replay, data generation or historical data extraction. The above token is encapsulated in the structure of userid:value in the form of value, and the above userid is the unique identifier of the user.

[0046] Among them, the method of updating tokens is adopted to ensure the continuous validity of the authentication information in the above data streams, including: receiving and responding to the authentication method selection operation, and adopting the corresponding authentication method to update the tokens.

[0047] Specifically, when the selected authentication method depends on providing an interface for directly obtaining new tokens based on old tokens, traverse the traffic data and scan the request information to obtain the current token; when there is a new token in the cache, read the above cache and replace the current token with the above new token; when there is no such new token in the cache, call the interface to obtain the above new token, write the above new token into the above cache, and then replace the current token with the above new token.

[0048] By updating tokens in real time, it is ensured that the authentication information in the playback traffic is up-to-date, avoiding security risks caused by using expired or leaked tokens. This mechanism requires verifying the validity of the token before each playback, thus enhancing the security of the system. Depending on providing an interface for directly obtaining new tokens based on old tokens, when the token expires or becomes invalid, no manual intervention is required, and the system can automatically update the token, reducing the workload of manual maintenance and improving the degree of automation. This update mechanism does not depend on a specific userid, but directly obtains new tokens through tokens, making the authentication process of traffic playback more flexible and capable of adapting to different business scenarios and different token generation rules, such as tokens based on timestamps.

[0049] In addition, a traffic compensation strategy based on time slicing is adopted, including: storing all the data in the information in the first list in full; splitting the above information into component information, interface information, and method information, and storing the above component information, the above interface information, and the above method information in the second list, the third list, and the fourth list respectively; reading the attribute value of the timeout time in the playback task, and asynchronously judging and processing the response time in the message for timeout; encapsulating all the data of the timed-out above message in full, and asynchronously writing the encapsulation result into the timeout table of the database, and at the same time outputting the above encapsulation result to the log file; when the sum of the start time and the time slicing threshold is greater than or equal to the current real-time time, determining that the current time slice is collecting data and not performing the calculation process; when the sum of the start time and the time slicing threshold is less than the above current real-time time, determining that the data of the above current time slice has been collected, entering the calculation process of the above current time slice, and at the same time starting to collect the data of the next time slice; when the total number of received messages counted is the same as the total playback amount in the playback task, after storing the data in the above first list, the above second list, the above third list, and the above fourth list in the corresponding temporary list, clearing the data in the above first list, the above second list, the above third list, and the above fourth list.

[0050] Storing all the data in the information in the first list in full is convenient for the preservation and retrieval of the overall data. At the same time, storing the component information, interface information, and method information in the second, third, and fourth lists respectively makes the data processing more efficient. This classified storage method is convenient for subsequent statistical analysis according to different dimensions, improving the flexibility and accuracy of data processing. Reading the attribute value of the timeout time in the playback task and asynchronously judging the response time in the message for timeout can detect and process timeout situations in real time, avoiding the impact of long waiting on the overall test efficiency. Encapsulating all the data of the timed-out message in full and asynchronously writing it into the timeout table of the database can ensure that all timeout information is accurately recorded, facilitating the analysis of problems and the identification of performance bottlenecks in the later stage. When the sum of the start time and the time slicing threshold is greater than or equal to the current real-time time, determining that the current time slice is collecting data and not performing the calculation process avoids the delay of data processing and ensures real-time performance. When the sum of the start time and the time slicing threshold is less than the current real-time time, it indicates that the data collection of this slice has been completed. At this time, the calculation process can be entered, and at the same time, the data of the next time slice can be collected, realizing the parallel processing of data collection and calculation and improving the data processing efficiency. When the total number of received messages counted is the same as the total playback amount in the playback task, storing the data in the list in the temporary list and clearing the original list ensures the timely processing of the data, avoids excessive memory occupation, and at the same time ensures the consistency and integrity of the data, providing an accurate data basis for subsequent calculation and analysis.

[0051] In one embodiment of the present application, during the process of adopting the traffic compensation strategy based on time slicing, the above method further includes: selecting a corresponding reference threshold according to the category to which the transaction belongs, and calculating the total amount of traffic expected to be required according to the magnitude of the concurrent pressure set when creating the task; when both the playback total parameter and the execution duration parameter exist, comparing the priorities of the playback total parameter and the execution duration parameter to obtain a comparison result; determining that the execution duration parameter is invalid or determining that the execution duration parameter is valid according to the above comparison result; and discarding the overflow traffic data when it is determined that the execution duration parameter is invalid and the transmitted traffic data is greater than or equal to the above expected total amount of traffic required.

[0052] By selecting a corresponding reference threshold according to the category to which the transaction belongs and calculating the total amount of traffic expected to be required in combination with the magnitude of the concurrent pressure, the accurate execution of the traffic playback task can be ensured, and the deviation of the test result caused by improper traffic control can be avoided. This strategy enables the system to automatically calculate the required traffic according to different types of transactions and the preset concurrent pressure, improving the predictability and accuracy of the test. When both the playback total parameter and the execution duration parameter exist, determining which parameters are valid by comparing the parameter priorities helps to avoid conflicts in the case of multi-parameter settings and ensures the stable operation of the system. This is particularly important in high-concurrency and large-data-volume test scenarios, helping to optimize the test process and reduce unnecessary data processing and storage costs. It can help the test system quickly and accurately reach the preset test target, and both the total amount of traffic and the concurrent pressure can be effectively controlled. This not only improves the test efficiency but also ensures the reliability and effectiveness of the test results, providing a solid foundation for subsequent performance analysis and system optimization.

[0053] Step S103, converting the test script of the above data stream into a script format recognizable by an automated test tool based on the JMeter or Postman script template;

[0054] First, preprocess the traffic data extracted from the storage component, including operations such as data cleaning, format verification, and duplicate removal, to ensure the integrity and consistency of the data. Extract key attributes from the preprocessed traffic data, such as request method, URL, parameters, protocol type, etc., and classify the traffic according to these attributes (by component, interface, or method). According to the protocol type of the traffic (such as HTTP, HTTPS, Dubbo, etc.), select the corresponding JMeter or Postman script template. For each classified traffic, set the parameter values in a parameterized manner, such as replacing the dynamic parameters or authentication information (such as token) in the request body with the parameters in the script, so that data can be dynamically passed in during testing. Based on the selected template, write information such as the request protocol, request address, and request path in the traffic data into the script. For the parameterized parameters, ensure that their values correspond to the parameter values in the parameterized file (such as CSV file) in the script, so that parameter data can be read when the script is executed. After completing the script assembly, optimize the script, such as adding assertions, setting thread groups, adjusting sampler parameters, etc., to meet the execution requirements of the automated testing tool and optimize the testing efficiency.

[0055] In step S104, in a multi-node distributed deployment manner, use the NIO communication framework of the playback engine component to send the above-mentioned test script after format conversion to the target system for performance testing. The management component collects the response data returned by the playback engine nodes through a real-time message mechanism, and uses a time sharding algorithm to analyze the performance metrics. The above performance metrics include response time, throughput, and TPS.

[0056] In the above steps, a traffic compensation strategy based on time sharding is adopted to ensure that the number of data streams meets the test requirements. The method of updating the token is used to ensure the continuous validity of the authentication information in the above data streams, enhancing the availability of the traffic and ensuring the value of the traffic. Through the mechanisms of parameterization, authentication, and traffic compensation, the reuse value of the traffic is improved, and the effectiveness and applicability of the captured traffic are ensured. By converting the test script of the above data stream into a script format recognizable by an automated testing tool based on the JMeter or Postman script template, the script modification work caused by using different stress testing tools is avoided, and the testing efficiency is improved. Compared with the existing solutions, there will be no deviation exceeding the specified standard between the test results and the performance metrics during actual operation in the process of performance testing, thus solving the problem that there is a deviation between the test results and the performance metrics during actual operation in the process of performance testing in the existing solutions.

[0057] Among them, using the time slicing algorithm for performance metric analysis provides a specific usage scenario: To conduct a performance stress test on a large financial trading system, the test objective is to evaluate the processing capacity and response speed of the system over a long period (such as 12 hours) under simulated real high-concurrency trading scenarios. Since the test time is long, the generated traffic data is extremely large, and directly analyzing all the data may lead to a slow analysis process and unable to provide real-time feedback on performance metrics.

[0058] Application of the time slicing algorithm: In this scenario, the time slicing algorithm is used to divide the 12-hour test cycle into multiple smaller time periods (such as each slice being 5 minutes), and the performance metrics (response time, throughput, and TPS) within each slice are calculated immediately at the end of each slice. In this way, not only can the performance of the system within each 5-minute slice be monitored in real time, but also the performance metric changes every 5 minutes can be continuously obtained during the 12-hour test cycle. Finally, by integrating the data of all slices, a detailed performance analysis report for the entire test cycle can be obtained.

[0059] Benefits of the above scenario: Through the performance metric data of time slicing, it is easier to identify the specific time periods when the system performance degrades, and then quickly locate possible performance bottlenecks or abnormal points, improving the efficiency of problem-solving. Data visualization and analysis: The time slicing algorithm can generate more detailed time series performance data, which can be visualized, facilitating testers and operation and maintenance teams to intuitively understand the change trend of system performance over time, providing data support for subsequent performance optimization and system upgrade. The size of the time slice can be flexibly adjusted according to test requirements. For example, when more refined data analysis is required in a short time, the time length of the slice can be reduced. This flexibility enables the system to adapt to different test scenarios and requirements. By calculating and monitoring performance metrics in real time within each time slice, the system has the ability of continuous testing, and can obtain the dynamic changes of system performance without interrupting the test, which is crucial for evaluating the stability and reliability of the system under different pressures.

[0060] In addition, using the NIO communication framework of the playback engine component in step S104 to send the above-mentioned test script after format conversion to the target system for performance testing includes: using the above-mentioned playback engine component as the client of the above-mentioned NIO communication framework, and using the above-mentioned management component as the server of the above-mentioned NIO communication framework; performing an initial inquiry interaction between the above-mentioned client and the above-mentioned server with keepAlive messages; based on the playback engine status flag and the traffic task status flag, determining whether the above-mentioned server is ready to establish a playback task for performance testing.

[0061] Specifically, the use of the keepAlive message ensures a stable connection between the client and the server. Meanwhile, through this message for initialization inquiry, the state synchronization between the client and the server is achieved, ensuring that both parties have the same understanding of the task state at the same time point, and improving the reliability and stability of the system. By detecting the status flags of the playback engine and the traffic task status flags, the server can intelligently determine whether to prepare to establish a playback task, and dynamically control the sending and receiving of traffic according to the current resource status and task status, avoiding resource waste and ensuring the efficiency of performance testing and the efficient utilization of resources.

[0062] In an embodiment of the present application, the above method further includes: downloading the extended protocol plug-in template code on the management console; writing a protocol plug-in according to the above extended protocol plug-in template code, and uploading the above protocol plug-in to the management console; restarting the engine service to deploy the relevant above protocol plug-ins together when deploying the engine node.

[0063] Specifically, by providing downloadable plug-in template code, the system allows users to develop and integrate new protocol plug-ins according to specific requirements, which greatly enhances the protocol support scope of the system, enabling the system to adapt to more types of traffic playback requirements, especially for non-standard or emerging network protocols. Using unified template code to write plug-ins can ensure that all plug-ins follow the same coding specifications and interface standards, which is conducive to code reuse and maintenance, reduces the repetitive labor in developing new protocol plug-ins, and improves the development efficiency. Users can directly upload the protocol plug-ins they write to the management console, and the system automatically processes the deployment of the plug-ins without complex configuration or manual intervention, simplifying the plug-in integration process and making protocol extension simpler and faster.

[0064] The framework of the performance testing device based on traffic playback is as Figure 2 shown. By extracting, analyzing, and diverting and playing back the actual traffic, it realizes the performance testing of conventional trading interfaces, the performance testing of trading scenarios, comparison testing, and capacity assessment, and provides a basis for determining the passing of interface performance. The device consists of six major components: a recording component, a storage component, a common component, a management component, a playback engine component, and a result display component, which realize functions such as traffic capture, traffic storage, traffic analysis, traffic control, traffic playback, result statistics and reporting, result monitoring and display.

[0065] Recording component: An independent component used to record traffic. Storage component: Stores the traffic information recorded or pulled. Common component: Extracts common utility classes and objects to form a common component. Management component: Includes functions such as task management, result collection, data calculation, node management, etc., and provides a WEB control management page.

[0066] Function Introduction of Management Component: Task Management Function: Implement pressure testing task management. The attributes of the created tasks include filtering the traffic to be replayed, setting the traffic size, pressure size, assertion conditions, execution duration, execution method, etc. Script Conversion Function: Format the traffic data in the storage medium and convert it into jmeter scripts and postman scripts using the pre-set jmeter script template and postman script template. Traffic Control Function: Filter the traffic that meets the conditions from the storage medium according to the set task information, and add replay marks to the traffic sent to the replay engine node. For the traffic with insufficient data, adopt a traffic compensation strategy for quantity compensation and trigger the update of traffic authentication information. Data Collection and Calculation Function: Design time slices, statistically calculate the key performance indicator data within the slices for all the data returned by the replay engines according to the time slice algorithm, and trigger the update of the overall performance indicator data (such as average TPS, average response time, 90% response time, etc.) after the end of the slice. Node Management Function: Manage the replay engine node devices, support device addition, editing, and deletion, mainly used for directly deploying the replay engine on the device nodes.

[0067] Replay Engine Component: Can be deployed on multiple nodes through the management component, including function modules such as traffic verification, pressure control, replay executor, and result processor.

[0068] Functions of the Replay Engine Component: Traffic Verification Function: Mainly verify the traffic protocol, integrity of traffic attributes, json format, etc. for the received traffic according to the subscribed traffic classification. The traffic that fails the verification will be discarded and the response message information to be returned will be assembled at the same time. Pressure Control Function: Set conditions such as the replay pressure size and replay time of the engine according to the received traffic control conditions, and initialize the pressure control object of the replay task. Replay Executor Function: Select the replay executor corresponding to the protocol according to the transaction request protocol of the replay task, apply pressure to replay the traffic, and recycle the result return data of the node applying pressure. Result Processor Function: Process and re-assemble the response message data of the transaction request and then return it to the management component.

[0069] Result Display Component: Rely on the prometheus component, mainly display the result data of the replay test in the form of charts, including key performance indicator data such as TPS and response time, and server resource consumption data of the engine nodes. Multi-Protocol Plug-in Extension Function: The extension protocol plug-in template code can be downloaded from the management console, the protocol plug-in can be written according to the template code, and uploaded to the management console. When deploying the engine nodes, the relevant protocol plug-ins will be deployed together. NIO Communication Framework: Used for traffic data transmission and replay result data transmission. In the technical solution, the common netty framework is used as an example for the solution description:

[0070] S1-1: The data recorded by the recording component is stored on the medium of the storage component. S1-2: Set the traffic conditions through the web page of the management component, and extract the filtered traffic data from the storage component. S1-3: The management component uses the utility classes such as message encapsulation and serialization in the common component to perform corresponding serialization and encapsulation operations on the traffic data to be sent. S1-4: In an asynchronous manner, store information such as playback task data, engine node data, and playback result data into the corresponding data tables in the database. S1-5: The management component interacts with the cache component for data caching and data retrieval. S1-6, S1-7: The management component sends the traffic data to be played back to the playback engine component through the NIO communication framework (such as netty). S1-8: The playback engine component uses the utility classes such as deserialization in the common component to parse the received traffic data. S1-9, S10: The playback engine component performs traffic playback testing on the system under test and collects the response results of transaction requests. S1-11: The playback engine component interacts with the cache component for data caching and data retrieval. S1-12, S13: The playback engine component sends the response result data of the transaction request to the calculation module of the management component through the NIO communication framework (such as netty). S1-14: The calculation module of the management component performs data calculation and then pushes the results to the result display component for data display.

[0071] The following elaborates on the core functions and implementation logics involved in the management component and the engine playback component (as Figure 3 shown, the NIO communication framework involved is described using the commonly used netty as an example):

[0072] S2-1: The playback engine, as a client of the NIO communication framework, will periodically send keepAlive messages to the management component when the message queue is idle; S2-2: The management component, as the server, will receive the keepAlive message sent by the client and return a keepAlive message to the client; S2-3: After receiving the keepAlive message sent by the client, the management component, as the server, will mark the status of the client as idle and write the status to the cache; S2-4: After receiving the keepAlive message sent by the server, the client will set the initFlag flag to false and write it to the cache; S2-5: After receiving the keepAlive message sent by the server, the client will send a checkInit message to the server to inquire whether initialization is required; S2-6: After receiving the checkInit message, the server will obtain the values of the playback engine status engineFlag and the traffic task status FlowFlag from the cache and determine whether the values of the flags respectively meet the conditions of engineFlag = engineInitStart and FlowFlag = flagRunning to determine whether the server is ready to establish a playback task; S2-7: When the server detects that the values of the engineFlag and FlowFlag flags do not meet the conditions, it will send a keepAlive message to the client to maintain the connection with the client; S2-8: Establish a traffic playback task through the console and update the values of the flags in the cache to engineFlag = engineInitStart and FlowFlag = flagRunning; S2-9: The client normally sends a checkInit message to the server; S2-10: After receiving the checkInit message, the server will obtain the values of the playback engine status engineFlag and the traffic task status FlowFlag from the cache and detect that they meet the initialization conditions, and update the flag engineFlag = engineInitRunning in the cache; S2-11: The server sends a msgProtocol message to the client, which contains the protocol information involved in the traffic in this playback task; S2-12: After receiving the msgProtocol message, the client will judge to perform initialization and send an initOk message to the server after the initialization is completed; S2-13: The client sets the initialization flag initFlag = true and stores it in the cache; S2-14: After receiving the initOk message, the server updates the playback engine flag in the cache to engineFlag = engineInitSuccess; S2-15: The server sends a getDBData message to the client, indicating the start of obtaining traffic data;S2-16: After the client receives the getDBData message, it sets the data flag bit dataFlag = true and stores it in the cache; S2-17: The server updates the playback engine flag bit in the cache to engineFlag = Transferring, indicating that data transfer is in progress; S2-18: The server starts to transfer traffic data to the client; S2-19: After the server completes data transfer, it updates the playback engine flag bit in the cache to engineFlag = TransferSuccess, indicating that data transfer is complete; S2-20: After receiving the traffic data, the client calls the executor for stress testing and sends the message result encapsulated by the result processor to the server; S2-21: The server receives the return message and performs calculations. When the return message of the traffic is received, it updates the flag bit value to engineFlag = flowOver, FlowFlag = FlowOver; S2-22: The server sends the complete message body to the client, and the task is completed; S2-23: The server clears the playback engine status engineFlag and the traffic task status FlowFlag flag bit values; S2-24: The client resets the flag bits dataFlag and initFlag to false.

[0073] Among them, the keepalive for the interaction and communication between the client and the server is mainly to keep the communication link unblocked and both parties are in an idle state. Each time the client receives the keepalive, it sets the initFlag flag bit to false in the buffer area, indicating that the client currently receives the keepalive signal and does not need to be initialized. Therefore, the initialization state is set to false.

[0074] Three core functions of the management component:

[0075] 1) Script conversion function process: As Figure 4As shown in the figure, S3-1: Screen traffic: Screen traffic information from the data source according to conditions. The screening dimensions include dimensions such as project dimension, component dimension, interface dimension, and protocol type. S3-2: Script types with transformation: Determine the script types to be transformed. Currently, two common script types, jmeter and postman, are supported. Select the corresponding initial script template according to the selected script type. The principles of other types of scripts are the same. S3-3: Traffic analysis: Mainly perform integrity verification of request attributes and verification of the format of the request body based on the traffic data screened in S3-1, classify the verified results according to different interfaces, and filter and deduplicate the request bodies of the traffic in the same interface. S3-4: Select the script export dimension: When exporting scripts by project, all interfaces in the project are in one script; when exporting scripts by component, the exported scripts are split according to the component dimension, and all interfaces within the same component are in the same script, and different components correspond to different scripts; when exporting scripts by interface, one script corresponds to one interface. S3-5: Parameterized script assembly: According to the script export dimension, start assembling the script, including information such as request protocol, request address, and request path. Set a parameter value in the request body of each interface in a parameterized manner and keep it corresponding to the parameter value of the corresponding csv parameterized file; S3-6: According to the interface classification result in S3-3, first traverse the different request bodies classified by a single interface, store them as the request body parameters of the interface in the csv parameterized file corresponding to the interface, and then traverse all the interface classifications until the parameterized files of all interfaces are generated. S3-7: Package the files generated in S3-5 and S3-6 to generate the final expected script.

[0076] 2) The traffic control function includes the main function, traffic compensation, and updated traffic authentication.

[0077] The implementation process of the main function is as Figure 5 As shown in the figure, S4-1: Create a playback task and set task parameters, including total traffic, traffic in a time period, a certain type of traffic, playback time, concurrency, etc. S4-2: Pull traffic from the storage medium according to the set conditions. If the set traffic volume is too large, adopt a strategy of pulling in batches. S4-3: Check whether the pulled traffic meets the requirements of quantity or execution duration and make corresponding handling. S4-4: Compensate for the traffic whose quantity does not meet the quantity index requirements or the actual playback duration of the traffic does not meet the expected execution duration requirements to ensure that the traffic quantity and playback duration can meet the expected requirements. S4-5: Update the authentication information in the traffic data to ensure the effectiveness of traffic authentication. S4-6: Send the prepared traffic to the playback engine.

[0078] The implementation process of the traffic compensation function is as Figure 6As shown, S5-1: The task parameters for creating a traffic playback task include but are not limited to the total playback traffic volume, pressure magnitude, traffic - affiliated project, traffic type, assertion conditions, execution duration, execution method, etc. When both the total playback traffic volume and the execution duration are empty, traffic can be further filtered by other conditions. S5-2: In the configuration file, set the corresponding vuser threshold and TPS threshold according to the transaction category (for example, for a simple query - type read interface with 50 vusers, the TPS threshold is 100, etc.). The transaction categories are initially divided into several major categories such as simple query - type read interfaces, simple insert - type write interfaces, complex query - type read interfaces, complex logic - type write interfaces, read - focused integrated transactions, write - focused integrated transactions, file upload and download - focused interfaces, etc. S5-3: Select the corresponding baseline threshold according to the transaction category, and calculate the expected total traffic volume based on the concurrent pressure magnitude set when creating the task (for example, for a simple query - type read interface with a threshold of 50 vusers and a TPS threshold of 100, the task is set to execute for 10 minutes with a pressure of 100 vusers, then the expected total traffic volume is 10 * 60 * 100 * 100 / 50 = 120,000). S5-4: The method for obtaining traffic data is to obtain it in batches. S5-5: Set the priority of task parameters in the configuration file. When the task parameters do not meet the condition of taking effect simultaneously, the parameter item with a higher priority in the configuration file is used as the valid parameter. S5-6: When both the playback total parameter and the execution duration parameter exist, judge the priorities of the playback total parameter and the execution duration parameter. If the former is higher, the process proceeds with the playback total parameter being valid and the execution duration parameter being invalid; if the latter is higher, the process proceeds with the playback total parameter being invalid and the execution duration parameter being valid. S5-7: In the scenario where the playback total is clearly valid, pull traffic data in batches and record the number of sent traffic. When the sent traffic data exceeds the expected playback total, the extra traffic data is directly discarded.

[0079] The implementation process of the updated traffic authentication function is as Figure 7As shown, taking the common token authentication information as an example, the processes of other authentication information are similar. There are usually four ways to update authentication: The first way to update authentication mainly relies on providing a separate interface to query the current valid token according to the user ID; the second way depends on the traffic analysis results, marks the interfaces that generate authentication information, traces back the token information in the specific traffic in the marked interfaces, determines the starting traffic generated by the token, replays it to generate a new token, and uniformly replaces the old token; the third way depends on providing an interface to directly obtain a new token based on the old token, which is applicable to ciphertext encryption methods with time information such as dates and timestamps; the fourth way is system cooperation transformation, and the requests marked for traffic replay are not verified for token validity. The basic processes of the other conventional ways to update authentication are variants of the above ways. Except for the fourth way that does not verify traffic, the specific processes of the other three ways to update authentication taking the token as an example are as follows: S6-1: Pull traffic according to the created task, traverse the traffic data pulled in each batch, and obtain the specific traffic request data information. S6-2: Scan the specific traffic request data, obtain the userid and token information in the request, and encapsulate it into a key:value structure with userid as the key and token as the value. S6-3: Obtain a new token from the cache according to the userid. If it exists, the value can be directly obtained for replacement; if it does not exist, go to step S6-4. S6-4: According to the provided interface to query the valid token according to the userid, obtain a new token again and write it into the cache asynchronously. All the valid tokens stored in the cache are set with an expiration time, and the expiration time is less than the token validity period. S6-5: The difference between this step and S6-2 is that only the token value in the traffic request data needs to be scanned, and there is no need to obtain the userid information and encapsulate the key:value structure. S6-6: Obtain a new token from the cache according to the old token. If it exists, the value can be directly obtained for replacement; if it does not exist, go to step S6-7. S6-7: The difference between this step and S6-4 is that the interface to obtain the new token is different. This step needs to call the interface to directly obtain a new token based on the old token, and store it in the cache in the form of the old token as the key and the new token as the value. All the valid tokens stored in the cache are set with an expiration time, and the expiration time is less than the token validity period. S6-8: Read the authentication interfaces edited during the traffic analysis process, de-duplicate them, and traverse the interface return values of all traffic under this interface to locate the specific traffic that generated the old token. S6-9: Send the located traffic to the replay engine for return visit, obtain the return result and parse it into a new token.

[0080] 3) The implementation process of data acquisition and calculation functions is as follows Figure 8As shown in the figure, S7-1: The manage management component, as the server of the NIO communication framework, receives the response message information sent back by the playback engine. The message is the message body encapsulated from the response message information. After receiving the message body, it is unpacked. S7-2: Obtain the task id value in the message body of the message and verify whether it is a new task. For a new task, parameters such as the data reception queue, the message quantity count value C, and the start time T1 are initialized; for the data of an old task, no initialization operation is performed. S7-3: Split the message information into three parts: component information, interface information, and method information, and store them in three lists, ListB, ListC, and ListD respectively, and store all the data in the message in the ListA list. S7-4: Read the property value of the timeout time in the playback task, and use an asynchronous method to judge and process the response time in the message for timeout. S7-5: Package all the message data judged to be timed out, and write the result to the timeout table in the database asynchronously, and at the same time output information to the log file. S7-6: Read the time slice threshold t0 in the configuration file for time slice calculation and message reception quantity calculation, and obtain the current real-time time as T2. When T1 + t0 ≥ T2, it means that the current time slice P1 is collecting data and the calculation process will not be performed; when T1 + t0 < T2, it means that the data of the time slice P1 has been collected, and the calculation process of P1 will be entered, and at the same time, the data of the next time slice P2 will start to be collected; when the total number of received messages C counted is equal to the total playback quantity C0 in the playback task, the data calculation within the slice will also be directly entered, regardless of the size of the time slice threshold. S7-7: If the total number of received messages C is equal to the total playback quantity C0 in the playback task, update engineFlag = flowOver and FlowFlag = FlowOver, indicating that the playback task is completed and the playback engine is completed, and update the message sending status to completed, and reset related task data such as the message quantity C. S7-8: Read the data in the four lists ListA, ListB, ListC, and ListD, and store them in the corresponding temporary lists TmpListA, TmpListB, TmpListC, and TmpListD, and then clear the data in the four lists ListA, ListB, ListC, and ListD. S7-9: Among the messages received within the time slice, the earliest execution time is recorded as replay_start, and the latest end time is recorded as replay_end, and combined with the data in TmpListA, TmpListB, TmpListC, and TmpList and information such as the playback task id, it is packaged into RelpaySingleStaTistics object data.S7-10: The putDetailResults() method is an asynchronous method used to deduplicate the three lists of TmpListB, TmpListC, and TmpListD in the RelpaySingleStaTistics object and then call the processing handlers for summary data, component-level data, interface-level data, and method-level data respectively. S7-11: A common computing component that performs data calculations and outputs data values in the format of ConcurrentHashMap, containing performance metric data such as average response time within a shard, 90% response time, total number of requests, number of successful requests, number of failed requests, total throughput, successful throughput, and failed throughput. This data serves as the metric data within the time shard. S7-12: Based on the output results of the common computing component and combined with the previous shard data, recalculate the performance metric data such as average response time, 90% response time, total number of requests, number of successful requests, number of failed requests, total throughput, successful throughput, and failed throughput for all shards up to the current time. This data serves as the overall metric data from the start of the task to the current time. Package the two sets of data into a MeterRegistry data object and send it to the result display component for data display. S7-13: Detect the execution status of the playback task, the execution status of the playback engine, and the message sending status. If all are completed, it indicates that the task playback is complete, and the performance metric data is asynchronously written to the database.

[0081] Two core functions of the playback engine component (playback executor function and result processor function):

[0082] The implementation process of the playback executor function is as Figure 9As shown in the figure, S8-1: Parse the received message data, obtain its interface protocol, and distribute it to the corresponding executor according to the interface protocol. S8-2: For the request data of the http protocol and the https protocol, use the http playback executor. S8-3: Parse the request data to obtain necessary information such as the request type, path, url, request body, and request headers. S8-4: Initialize the okHttpClient object depending on the okhttp component, and use the initialized okHttpClient to initiate an http or https request call. S8-5: For the request data of the dubbo protocol, use the dubbo playback executor. S8-6: Parse the request data to obtain necessary information such as the request type, path, url, request body, and request headers. S8-7: Write the dubboClient and complete the initialization, and use the initialized dubboclient to initiate a dubbo call. S8-8: For other protocols, refer to the http protocol processing process of S8-2, S8-3, and S8-4 or the dubbo processing process of S8-5, S8-6, and S8-7; S8-9: Obtain the response result of the request call and forward it to the result processor for processing. S8-10: During traffic playback, check the initialization status of the playback engine in the cache for each request playback, and keep the status as uninitialized during playback.

[0083] The implementation process of the result processor function is as Figure 10 As shown in the figure, S9-1: Complete other necessary information on the basis of the received response message, such as the task ID it belongs to, the component it belongs to, the interface it belongs to, the assertion result, the trace number, the execution time, the response time, etc. S9-2: Format the completed response information and convert it into a JSONObject object. S9-3: Package the JSONObject into a nettyMessage message body. S9-4: Call the packaged netty component to send the nettyMessage message body to the management end. S9-5: During traffic playback, check the initialization status of the playback engine in the cache for each response result processed, and keep the status as uninitialized during playback.

[0084] The plug-in extension function for multiple protocols: The operation process of this function is to download the extension protocol plug-in template code on the management console, perform secondary development according to the template code, and upload the compiled package to the management console as required. When deploying the engine node, the relevant protocol plug-ins will be deployed together or the plug-in will be deployed separately (when deploying the plug-in separately, the playback engine service needs to be restarted). As Figure 11As shown below, the core implementation process of the plugin template code is as follows: S10-1: Set the initialized config information and obtain the service to be initialized. Create the file GetXxxService.java (Xxx represents the specific protocol value, which is consistent with the protocol name newly added in the management console). Override the two methods serverInit and getServiceMap of GetService. The writing method can refer to the example method. S10-2: Create the file CreateXxxInitService.java (Xxx represents the specific protocol value, which is consistent with the protocol name newly added in the management console), inherit CreateInitService, and add the member method CreateXxxServer() according to the example. S10-3: Create the interface implementation file CreatexxxInitServiceImpl.java corresponding to S10-2, which is the implementation of CreateXxxInitService, and call the two initialization methods of GetXxxService. S10-4: Override the getClient() method in the interface implementation file CreateXxxInitServiceImpl.java, add case judgments to determine whether the protocol value meets the conditions. When there is specific protocol information in the message body, return the initialization implementation method corresponding to the protocol. S10-5: Create XxxReplayService.java to inherit ReplayService and add an execution member method. This interface is used to call the newly added protocol replay executor during traffic replay. S10-6: Create the interface implementation file XxxReplayServiceImpl.java corresponding to S10-5 and write the specific implementation process of the execution member method. Note: The steps from S10-1 to S10-4 are to perform necessary initialization on the engine according to the protocol, which is applicable to the protocol framework that needs to be initialized before submitting the traffic request to the system under test. Protocols that do not require initialization can directly start writing the implementation from S10-5.

[0085] Based on the heartbeat detection of the NIO communication framework itself, this application designs a set of message interaction mechanisms. By dynamically changing the message content, it triggers the change of flag bits of task status, message sending status, and playback engine status, thereby triggering corresponding business operations, maintaining effective communication and status detection between the management component and the playback engine component, and ensuring the stability and accuracy of the interaction process; a perfect traffic compensation mechanism is established from multiple dimensions. For the split of the designed task attributes, the attribute values involved in compensation are verified layer by layer, and traffic compensation is achieved through the configuration of key control conditions; a general traffic authentication mechanism is established to meet diversified usage scenarios. The traffic authentication mechanism considers multiple authentication modes and adapts to the project to ensure the effectiveness of traffic, realizing that available traffic can be measured, and measurable traffic can be used; the multi-protocol extension of the playback engine component is realized by uploading multi-protocol plugin packages, providing a protocol plugin template and supporting secondary development. After the plugin package is uploaded, it will be deployed together with the engine-side deployment operation, greatly improving the versatility and convenience of the engine playback ability and realizing the diversification of test scenarios; based on the idea of time slicing, a set of index slicing algorithms for traffic return values is designed to calculate metrics such as TPS, response time, and throughput of playback traffic, and the unified display of performance metric data and server resource consumption data is realized through the metric component.

[0086] The advantages of this application are as follows: First, it expands the application scenario. The traffic analysis function not only reduces the dependence on personal experience and skill level, improves efficiency, but also greatly improves the accuracy of traffic analysis, avoiding interface omissions and deviations caused by manual analysis, which is beneficial to both performance scenario analysis and interface coverage statistics and full-scale function regression. Second, it enhances the availability of traffic and ensures the value of traffic. Through mechanisms such as parameterization, authentication, and traffic compensation, the reuse value of traffic is improved, and the effectiveness and applicability of the captured traffic are ensured. Third, it realizes the batch conversion of scripts. Through template matching, traffic data is automatically batch-converted into test scripts required by common stress testing tools, avoiding the script modification work caused by using different stress testing tools and improving the test efficiency. Fourth, based on traditional resource monitoring, traffic response data is recycled in real time, and traffic performance metrics are calculated and displayed, improving the metric coverage of performance monitoring results and providing comprehensive data support for subsequent problem analysis and decision-making.

[0087] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0088] The embodiments of the present application also provide a performance testing device based on traffic replay. It should be noted that the performance testing device based on traffic replay in the embodiments of the present application can be used to execute the performance testing method based on traffic replay provided by the embodiments of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation manners, and those that have been described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0089] The following introduces the performance testing device based on traffic replay provided by the embodiments of the present application.

[0090] Figure 12 It is a structural block diagram of a performance testing device based on traffic replay provided according to an embodiment of the present application. As Figure 12 shown, the device includes:

[0091] A first processing unit 121, configured to capture a data stream from a real-time trading system or historical data, serialize and encapsulate the data stream using a tool class in a common component, and then store it in a storage component, where the storage component includes a distributed file system, cloud storage, and a local database;

[0092] A second processing unit 122, configured to adopt a traffic compensation strategy based on time slicing to ensure that the number of data streams meets the test requirements, and adopt the method of updating tokens to ensure the continuous validity of the authentication information in the data stream. The traffic compensation strategy is selected from data replay, data generation, or historical data extraction. The token is encapsulated in the form of userid:value, and the userid is the unique identifier of the user;

[0093] A third processing unit 123, configured to convert the test script of the data stream into a script format recognizable by an automated testing tool based on a JMeter or Postman script template;

[0094] A fourth processing unit 124, configured to send the test script after format conversion to a target system for performance testing in a multi-node distributed deployment manner by using the NIO communication framework of the replay engine component, collect the response data returned by the replay engine nodes by the management component through a real-time message mechanism, and perform analysis of performance metrics by using a time slicing algorithm. The performance metrics include response time, throughput, and TPS.

[0095] In the above device, a traffic compensation strategy based on time slicing is adopted to ensure that the number of data streams meets the test requirements. The token is updated to ensure the continuous validity of the authentication information in the above data streams, enhancing the availability of the traffic and ensuring the value of the traffic. Through the mechanisms of parameterization, authentication, and traffic compensation, the reuse value of the traffic is improved, and the effectiveness and applicability of the captured traffic are ensured. By converting the test script of the above data stream into a script format recognizable by an automated test tool based on the JMeter or Postman script template, the script modification work caused by using different stress testing tools is avoided, and the test efficiency is improved. Compared with the existing solution, there will be no deviation exceeding the specified standard in the test results and the performance indicators during actual operation during performance testing, thus solving the problem that there is a deviation between the test results and the performance indicators during actual operation in the existing solution for performance testing.

[0096] In an embodiment of the present application, the fourth processing unit includes a first processing module, a second processing module, and a third processing module. The first processing module is used to use the above playback engine component as the client of the above NIO communication framework and the above management component as the server of the above NIO communication framework; the second processing module is used to perform an initial inquiry interaction between the above client and the above server with keepAlive messages; the third processing module is used to determine whether the above server is ready to establish a playback task based on the playback engine status flag and the traffic task status flag for performance testing.

[0097] In an embodiment of the present application, the second processing unit includes a fourth processing module, which is used to receive and respond to the authentication method selection operation and update the token using the corresponding authentication method.

[0098] In an embodiment of the present application, the fourth processing module includes a first processing sub-module, a second processing sub-module, and a third processing sub-module. The first processing sub-module is used to traverse the traffic data and scan the request information to obtain the current token when the selected above authentication method depends on providing an interface for directly obtaining a new token based on the old token; the second processing sub-module is used to read the above cache and replace the above current token with the above new token when there is a new token in the cache; the third processing sub-module is used to call the interface to obtain the above new token when there is no such new token in the cache, write the above new token into the above cache, and then replace the above current token with the above new token.

[0099] In an embodiment of the present application, the second processing unit includes a fifth processing module, a sixth processing module, a seventh processing module, an eighth processing module, a ninth processing module, and a tenth processing module. The fifth processing module is configured to store all the data in the information into a first list, split the above information into component information, interface information, and method information, and store the above component information, the above interface information, and the above method information into a second list, a third list, and a fourth list respectively; the sixth processing module is configured to read the attribute value of the timeout time in the playback task, and perform timeout judgment and processing on the response time in the message asynchronously; the seventh processing module is configured to perform full encapsulation on the data of the timed-out above message, write the encapsulation result into the timeout table of the database asynchronously, and output the above encapsulation result to a log file at the same time; the eighth processing module is configured to determine that data collection is in progress for the current time slice and not perform the calculation process when the sum of the start time and the time slice threshold is greater than or equal to the current real-time time; the ninth processing module is configured to determine that the data of the above current time slice has been collected and enter the calculation process of the above current time slice, and start collecting data for the next time slice at the same time when the sum of the start time and the time slice threshold is less than the above current real-time time; the tenth processing module is configured to, when the total number of received messages counted is the same as the total playback amount in the playback task, after storing the data in the above first list, the above second list, the above third list, and the above fourth list into corresponding temporary lists, clear the data in the above first list, the above second list, the above third list, and the above fourth list.

[0100] In an embodiment of the present application, the above device further includes a fifth processing unit, a sixth processing unit, and a seventh processing unit. The fifth processing unit is configured to download the extended protocol plug-in template code on the management console; the sixth processing unit is configured to write a protocol plug-in according to the above extended protocol plug-in template code and upload the above protocol plug-in to the management console; the seventh processing unit is configured to restart the engine service to deploy the relevant above protocol plug-ins together when deploying the engine node.

[0101] In an embodiment of the present application, the second processing unit includes an eleventh processing module, a twelfth processing module, a thirteenth processing module, and a fourteenth processing module. The eleventh processing module is configured to select a corresponding reference threshold according to the category to which the transaction belongs during the process of adopting the traffic compensation strategy based on time slicing, and calculate the total amount of traffic expected to be required according to the magnitude of the concurrent pressure set when creating the task. The twelfth processing module is configured to compare the priorities of the playback total amount parameter and the execution duration parameter when both the playback total amount parameter and the execution duration parameter exist, and obtain a comparison result. The thirteenth processing module is configured to determine that the execution duration parameter is invalid or determine that the execution duration parameter is valid according to the above comparison result. The fourteenth processing module is configured to discard the overflow traffic data when it is determined that the execution duration parameter is invalid and the sent traffic data is greater than or equal to the above expected total amount of traffic.

[0102] The above performance testing device based on traffic playback includes a processor and a memory. The above first processing unit, second processing unit, third processing unit, fourth processing unit, etc. are all stored in the memory as program units, and the processor executes the above program units stored in the memory to implement corresponding functions. The above modules are all located in the same processor; or, the above each module is located in different processors in any combination form.

[0103] The processor contains a kernel, and the kernel retrieves the corresponding program unit from the memory. One or more kernels can be set, and by adjusting the kernel parameters, the problem that there is a deviation between the test result and the performance index during actual operation in the existing solution for performance testing can be solved.

[0104] The memory may include non-permanent memory in a computer-readable medium, forms such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM), and the memory includes at least one storage chip.

[0105] The embodiment of the present invention provides a computer-readable storage medium. The above computer-readable storage medium includes a stored program, wherein when the above program runs, it controls the device where the above computer-readable storage medium is located to execute the above performance testing method based on traffic playback.

[0106] The embodiment of the present invention provides a processor. The above processor is used to run a program, wherein when the above program runs, it executes the above performance testing method based on traffic playback.

[0107] An embodiment of the present invention provides a device, which includes a processor, a memory, and a program stored on the memory and executable on the processor. When the processor executes the program, it implements at least the following steps: capturing a data stream from a real-time trading system or historical data, serializing and encapsulating the data stream using a tool class in a common component, and storing the serialized and encapsulated data stream in a storage component, where the storage component includes a distributed file system, cloud storage, and a local database; adopting a traffic compensation strategy based on time slicing to ensure that the number of data streams meets the test requirements, and adopting a method of updating the token to ensure the continuous validity of the authentication information in the data stream. The traffic compensation strategy is selected from data replay, data generation, or historical data extraction. The token is encapsulated in the form of value into a structure of userid:value, and the userid is the unique identifier of the user; converting the test script of the data stream into a script format recognizable by an automated test tool based on a JMeter or Postman script template; sending the test script after format conversion to a target system for performance testing in a multi-node distributed deployment manner by using the NIO communication framework of a playback engine component, collecting the response data returned by the playback engine nodes by a management component through a real-time messaging mechanism, and performing performance metric analysis using a time slicing algorithm, where the performance metrics include response time, throughput, and TPS. The device in this article can be a server, a PC, a PAD, a mobile phone, etc.

[0108] The present application also provides a computer program product, which, when executed on a data processing device, is adapted to execute a program initialized with at least the following method steps: capturing a data stream from a real-time trading system or historical data, serializing and encapsulating the data stream using a tool class in a common component, and storing the serialized and encapsulated data stream in a storage component, where the storage component includes a distributed file system, cloud storage, and a local database; adopting a traffic compensation strategy based on time slicing to ensure that the number of data streams meets the test requirements, and adopting a method of updating the token to ensure the continuous validity of the authentication information in the data stream. The traffic compensation strategy is selected from data replay, data generation, or historical data extraction. The token is encapsulated in the form of value into a structure of userid:value, and the userid is the unique identifier of the user; converting the test script of the data stream into a script format recognizable by an automated test tool based on a JMeter or Postman script template; sending the test script after format conversion to a target system for performance testing in a multi-node distributed deployment manner by using the NIO communication framework of a playback engine component, collecting the response data returned by the playback engine nodes by a management component through a real-time messaging mechanism, and performing performance metric analysis using a time slicing algorithm, where the performance metrics include response time, throughput, and TPS.

[0109] The present application also provides a performance testing system based on traffic replay, including: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include those for executing any of the above methods.

[0110] Obviously, those skilled in the art should understand that the various modules or steps of the present invention described above can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed over a network composed of multiple computing devices. They can be implemented with program codes executable by the computing device, so that they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a different order from here, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module to implement. Thus, the present invention is not limited to any specific combination of hardware and software.

[0111] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program codes.

[0112] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processors of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0113] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implements the functions in Figure 1 one flow or multiple flows and / or blocks Figure 1The functions specified in one or more boxes.

[0114] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide for implementing the steps of the functions specified in one Figure 1 one process or more processes and / or boxes Figure 1 step of the functions specified in one box or more boxes.

[0115] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0116] The memory may include non-permanent memory in the computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

[0117] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can store information by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0118] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, commodity or device comprising the element.

[0119] As can be seen from the above description, the above embodiments of the present application achieve the following technical effects:

[0120] 1), The performance testing method based on traffic replay of the present application adopts a traffic compensation strategy based on time slicing to ensure that the number of data streams meets the test requirements, and adopts the method of updating tokens to ensure the continuous validity of the authentication information in the above data streams, enhancing the availability of traffic, ensuring the value of traffic. Through the mechanisms of parameterization, authentication, and traffic compensation, the reuse value of traffic is improved, and the effectiveness and applicability of the captured traffic are ensured. By converting the test scripts of the above data streams into a script format recognizable by automated testing tools based on JMeter or Postman script templates, the script modification work caused by using different stress testing tools is avoided, and the test efficiency is improved. Compared with the existing solutions, there will be no deviation exceeding the specified standard between the test results and the performance indicators during actual operation in the process of performance testing, thus solving the problem that there is a deviation between the test results and the performance indicators during actual operation in the process of performance testing in the existing solutions.

[0121] 2), The performance testing device based on traffic replay of the present application adopts a traffic compensation strategy based on time slicing to ensure that the number of data streams meets the test requirements, and adopts the method of updating tokens to ensure the continuous validity of the authentication information in the above data streams, enhancing the availability of traffic, ensuring the value of traffic. Through the mechanisms of parameterization, authentication, and traffic compensation, the reuse value of traffic is improved, and the effectiveness and applicability of the captured traffic are ensured. By converting the test scripts of the above data streams into a script format recognizable by automated testing tools based on JMeter or Postman script templates, the script modification work caused by using different stress testing tools is avoided, and the test efficiency is improved. Compared with the existing solutions, there will be no deviation exceeding the specified standard between the test results and the performance indicators during actual operation in the process of performance testing, thus solving the problem that there is a deviation between the test results and the performance indicators during actual operation in the process of performance testing in the existing solutions.

[0122] The above are only the preferred embodiments of the present application and are not used to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A performance testing method based on traffic playback, characterized in that, Including: Capturing a data stream from a real-time trading system or historical data, serializing and encapsulating the data stream using a tool class in a common component, and then storing it in a storage component, where the storage component includes a distributed file system, cloud storage, and a local database; Adopting a traffic compensation strategy based on time slicing to ensure that the number of data streams meets the test requirements, and adopting the method of updating tokens to ensure the continuous validity of the authentication information in the data stream. The traffic compensation strategy is selected from data replay, data generation, or historical data extraction. The token is encapsulated in the structure of userid:value in the form of value, and the userid is the unique identifier of the user; Converting the test script of the data stream into a script format recognizable by an automated test tool based on a JMeter or Postman script template; In a multi-node distributed deployment manner, using the NIO communication framework of the playback engine component to send the format-converted test script to the target system for performance testing. The management component collects the response data returned by the playback engine nodes through a real-time messaging mechanism, and uses a time slicing algorithm for performance metric analysis. The performance metrics include response time, throughput, and TPS.

2. The method according to claim 1, characterized in that, Using the NIO communication framework of the playback engine component to send the format-converted test script to the target system for performance testing, including: Regarding the playback engine component as the client of the NIO communication framework, and regarding the management component as the server of the NIO communication framework; Performing an initial inquiry interaction between the client and the server with keepAlive messages; Based on the playback engine status flag and the traffic task status flag, determining whether the server is ready to establish a playback task for performance testing.

3. The method according to claim 1, wherein Adopting the method of updating tokens to ensure the continuous validity of the authentication information in the data stream, including: Receiving and responding to an authentication method selection operation, and using the corresponding authentication method to update the token.

4. The method according to claim 3, wherein Using the corresponding authentication method to update the token, including: When the selected authentication method depends on an interface that provides directly obtaining a new token based on an old token, traversing the traffic data and scanning the request information to obtain the current token; When there is a new token in the cache, reading the cache and replacing the current token with the new token; When there is no such new token in the cache, calling the interface to obtain the new token, writing the new token into the cache, and then replacing the current token with the new token.

5. The method according to claim 1, characterized in that Adopting a traffic compensation strategy based on time slicing, including: Storing all the data in the information in a first list, splitting the information into component information, interface information, and method information, and storing the component information, the interface information, and the method information in a second list, a third list, and a fourth list respectively; Read the property value of the timeout in the playback task, and asynchronously judge and process the response time in the message for timeout; Fully encapsulate the data of the timed-out message, and write the encapsulation result into the timeout table of the database asynchronously, and at the same time output the encapsulation result to the log file; When the sum of the start time and the time slice threshold is greater than or equal to the current real-time time, it is determined that data collection is in progress for the current time slice, and the calculation process is not performed; When the sum of the start time and the time slice threshold is less than the current real-time time, it is determined that the data of the current time slice has been collected, enter the calculation process of the current time slice, and at the same time start collecting data for the next time slice; When the total number of received messages counted is the same as the total playback volume in the playback task, after storing the data in the first list, the second list, the third list, and the fourth list into the corresponding temporary list, clear the data in the first list, the second list, the third list, and the fourth list.

6. The method according to claim 1, wherein The method further includes: Download the extended protocol plug-in template code on the management console; Write a protocol plug-in according to the extended protocol plug-in template code, and upload the protocol plug-in to the management console; Restart the engine service to deploy the relevant protocol plug-ins when deploying the engine node.

7. The method according to any one of claims 1 to 6, characterized in that In the process of adopting the traffic compensation strategy based on time slices, the method further includes: Select the corresponding benchmark threshold according to the category to which the transaction belongs, and calculate the total amount of traffic expected to be required according to the concurrent pressure set when creating the task; When the playback total parameter and the execution duration parameter exist at the same time, compare the priorities of the playback total parameter and the execution duration parameter to obtain a comparison result; According to the comparison result, determine that the execution duration parameter is invalid or determine that the execution duration parameter is valid; When it is determined that the execution duration parameter is invalid and the sent traffic data is greater than or equal to the total amount of traffic expected to be required, discard the overflow traffic data.

8. A performance testing device based on traffic replay, characterized in that, It includes: The first processing unit is used to capture the data stream from the real-time trading system or historical data, serialize and encapsulate the data stream using the utility class in the common component and then store it in the storage component, and the storage component includes a distributed file system, cloud storage, and a local database; The second processing unit is used to adopt a traffic compensation strategy based on time slices to ensure that the number of data streams meets the test requirements, and adopt the method of updating the token to ensure the continuous validity of the authentication information in the data stream. The traffic compensation strategy is selected from data replay, data generation, or historical data extraction. The token is encapsulated in the form of userid:value, and the userid is the unique identifier of the user; The third processing unit is used to convert the test script of the data stream into a script format recognizable by the automated test tool based on the JMeter or Postman script template; A fourth processing unit, which is used to send the test script after format conversion to a target system for performance testing in a multi-node distributed deployment manner by using the NIO communication framework of the playback engine component, collect response data returned by the playback engine nodes by the management component through a real-time messaging mechanism, and perform analysis of performance metrics by using a time slicing algorithm, where the performance metrics include response time, throughput, and TPS.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program runs, it controls the device where the computer-readable storage medium is located to execute the method according to any one of claims 1 to 7.

10. A performance testing system based on traffic replay, characterized in that, Comprising: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include those for executing the method according to any one of claims 1 to 7.

Images