Distributed stress testing platform system and method based on rust asynchronous architecture

Abstract

This invention relates to a distributed load testing platform system and method based on a Rust asynchronous architecture. The system includes a web console module for creating and managing load testing tasks; a Manager node configured to receive load testing tasks, perform task orchestration, and broadcast tasks to a Worker cluster; and a Worker cluster comprising multiple Worker nodes, each configured to: initialize an asynchronous runtime thread pool based on Tokio, using an event-driven non-blocking I / O model to handle concurrent connections; obtain a CSV test data file from a data storage module and calculate the data sharding range of its node based on the node index and the total number of nodes; locate and replace placeholders in the request template using a SIMD instruction set-based placeholder parser; and send HTTP requests to the target service through the asynchronous runtime thread pool. This invention can reduce resource consumption in high-concurrency scenarios and improve processing efficiency in large-scale request scenarios.

Description

Technical Field

[0001] This invention relates to the field of computer software testing technology, and in particular to a distributed load testing platform system and method based on the RUST asynchronous architecture. Background Technology

[0002] Stress testing is a technique used in software development and system operation and maintenance to evaluate system performance and stability. With the rapid development of internet applications and the widespread adoption of distributed systems and microservice architectures, the performance requirements for stress testing tools are increasing. Existing stress testing tools typically employ a multi-threaded synchronous blocking model to simulate concurrent user requests, where each thread handles one concurrent connection. In distributed stress testing scenarios, the control node is responsible for task scheduling and distribution, while worker nodes are responsible for executing specific stress testing tasks and reporting the results. To support dynamic test data, stress testing tools often use parameterization techniques, replacing variables in the test template with actual data through placeholder mechanisms. Furthermore, for interfaces requiring security verification, stress testing tools also need to support signature calculation and data encryption functions to simulate real production environment requests.

[0003] Existing stress testing technologies face several challenges in practical applications. First, thread architectures based on synchronous blocking models suffer from significant resource consumption in high-concurrency scenarios, and the overhead of thread switching affects stress testing efficiency. Second, in distributed stress testing scenarios, the allocation and management of test data requires consideration of coordination among multiple nodes to avoid data reuse or contention. Third, placeholder parsing and replacement operations during parameterization can become performance bottlenecks in large-scale request scenarios. Furthermore, the large amount of log data generated during stress testing puts pressure on storage systems, necessitating a balance between log integrity and storage costs. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, the purpose of this invention is to provide a distributed load testing platform system and method based on the Rust asynchronous architecture. By adopting an asynchronous non-blocking model, resource consumption in high-concurrency scenarios can be reduced. By using a distributed data allocation mechanism, data competition between multiple nodes can be reduced. By using optimized parameterized processing, processing efficiency in large-scale request scenarios can be improved. And by using a log management strategy, a balance can be achieved between log integrity and storage cost.

[0005] To achieve the above objectives, the present invention adopts the following technical solution.

[0006] Firstly, the present invention provides a distributed load testing platform system based on Rust asynchronous architecture, which adopts the following technical solution: The Web console module is configured to provide a task management interface for creating and managing load testing tasks; The Manager node is configured to receive load testing tasks from the Web console module, perform task orchestration, and broadcast the tasks to the Worker cluster via a distributed communication protocol. A worker cluster comprises multiple worker nodes. Each worker node is configured to: initialize an asynchronous runtime thread pool based on Tokio, handling concurrent connections using an event-driven non-blocking I / O model; retrieve a CSV test data file from the data storage module and calculate the data sharding range for its node based on the node index and the total number of nodes; locate and replace placeholders in the request template using a SIMD instruction set-based placeholder parser; send HTTP requests to the target service through the asynchronous runtime thread pool; and... The data storage module is configured to store task configuration data, CSV test data files, and load test log data. The Manager node is also configured to aggregate performance metrics data reported by all Worker nodes in real time and generate performance reports.

[0007] Furthermore, in the above system, the placeholder parser based on the SIMD instruction set is configured as follows: Use the SIMD Finder from the memchr library during compilation to find the start and end positions of all placeholders in the request template; Mark the index, start position, and end position of each placeholder; and The placeholder replacement operation is performed at runtime with O(1) complexity.

[0008] Furthermore, in the above system, the placeholders include CSV variable placeholders and user-defined function placeholders. The CSV variable placeholders adopt the {{variable}} syntax format, and the user-defined function placeholders adopt the {{_function()}} syntax format.

[0009] Furthermore, in the above system, each Worker node is also configured as follows: Use an atomic counter to repeatedly retrieve CSV data rows within the data fragment range; and Lock-free data access is achieved based on the atomic counter, avoiding multi-threaded data contention.

[0010] Furthermore, the above system also includes a log sampling module, which is configured as follows: Perform 100% log collection on error responses; Successful responses are sampled using a pre-defined sampling probability; and The sampled logs are sent asynchronously to the message queue via a lock-free message channel.

[0011] Furthermore, in the above system, each Worker node is also configured to perform security processing operations, which include: Use the HMAC-SHA256 algorithm to verify the signature of the request; Sensitive data is encrypted using the AES encryption algorithm; and The Nonce mechanism is used to prevent replay attacks.

[0012] Secondly, the present invention provides a distributed load testing method based on Rust asynchronous architecture, which adopts the following technical solution: The Manager node receives load testing tasks and orchestrates them, and broadcasts the tasks to multiple Worker nodes through a distributed communication protocol. An asynchronous runtime thread pool based on Tokio is initialized on each Worker node, using an event-driven non-blocking I / O model; Obtain the CSV test data file on each Worker node, and calculate the data sharding range for this node based on the node index and the total number of nodes; The placeholders in the request template are located by a placeholder parser based on the SIMD instruction set, and test data lines are obtained from the data fragment range using an atomic counter to replace the placeholders. Send an HTTP request to the target service through the asynchronous runtime thread pool; and The Manager node aggregates performance metrics data reported by all Worker nodes in real time and generates performance reports.

[0013] Furthermore, in the above method, the step of locating the placeholders in the request template using a placeholder parser based on the SIMD instruction set includes: Use the SIMD Finder from the memchr library during compilation to find the start and end positions of all placeholders in the request template; Mark the index, start position, and end position of each placeholder; and The placeholder replacement operation is performed at runtime with O(1) complexity.

[0014] Furthermore, in the above method, the placeholders include CSV variable placeholders and user-defined function placeholders. The CSV variable placeholders adopt the {{variable}} syntax format, and the user-defined function placeholders adopt the {{_function()}} syntax format.

[0015] Furthermore, the above method also includes a log sampling step, which includes: Perform 100% log collection on error responses; Successful responses are sampled using a pre-defined sampling probability; and The sampled logs are sent asynchronously to the message queue via a lock-free message channel.

[0016] Furthermore, the above method also includes performing a security processing step before sending the HTTP request, the security processing step including: The request is signed using the HMAC-SHA256 algorithm; Sensitive data is encrypted using the AES encryption algorithm; and The Nonce mechanism is used to prevent replay attacks.

[0017] Thirdly, the present invention provides a readable storage medium, which adopts the following technical solution: A readable storage medium storing computer instructions that, when executed by a processor, implement the method as described in any one of the second aspects above.

[0018] In summary, compared with the prior art, the present invention has at least one of the following beneficial technical effects: The distributed load testing platform system based on Rust asynchronous architecture provided by this invention, by adopting an asynchronous runtime thread pool based on Tokio and an event-driven non-blocking I / O model, can support a large number of concurrent connections with fewer system resources in high-concurrency scenarios. By calculating the data sharding range based on the node index and the total number of nodes, each worker node can obtain non-overlapping test data, thereby reducing data contention issues in the distributed environment. Through a placeholder parser based on the SIMD instruction set, the vectorized computing capabilities of the processor can be used to accelerate the placeholder location process, thereby improving the efficiency of parameterization processing. Through the Manager node to summarize performance index data in real time, it can provide users with real-time monitoring and performance analysis capabilities for the load testing process. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 The diagram shows a system architecture diagram of an embodiment of the distributed load testing platform system based on Rust asynchronous architecture according to the present invention.

[0021] Figure 2A block diagram illustrating an embodiment of the internal architecture of the Worker node of the present invention is shown.

[0022] Figure 3 A flowchart of an embodiment of the distributed load testing method based on the Rust asynchronous architecture of the present invention is shown.

[0023] Figure 4 A flowchart of an embodiment of the placeholder parsing method based on the SIMD instruction set of the present invention is shown.

[0024] Figure 5 A flowchart of one embodiment of the security processing and log sampling process of the present invention is shown. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Furthermore, it should be understood that the specific embodiments described herein are only for illustration and explanation of this application and are not intended to limit this application.

[0026] It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments of this application. Furthermore, the descriptions of each embodiment in the following embodiments have their own emphasis; for parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0027] The method steps described in this embodiment of the invention can be executed in the order described in the specific implementation, or the execution order of each step can be adjusted according to actual needs, provided that the technical problem can be solved. These are not listed one by one here.

[0028] The present invention will be further described in detail below with reference to the accompanying drawings.

[0029] Reference Figure 1 The distributed load testing platform system 100 provided in this embodiment of the invention is built on a Rust asynchronous architecture and includes a Web console module 102, a Manager node 104, a Worker cluster 112, and a data storage module 122. The distributed load testing platform system 100 uses jemalloc instead of the system's default memory allocator to reduce memory fragmentation, improve allocation speed, and reduce memory consumption. The distributed load testing platform system 100 uses Arc smart pointers to share data among multiple Worker nodes 114, avoiding duplicate data copying.

[0030] Web console module 102 is configured to provide a task management interface for creating and managing load testing tasks. Web console module 102 is connected to Manager node 104 and passes user-created load testing tasks to Manager node 104.

[0031] Manager node 104 includes a task orchestration unit 106, a metric aggregation unit 108, and a report generation unit 110. Manager node 104 is configured to receive load testing tasks from the Web console module 102, perform task orchestration, and broadcast the tasks to the Worker cluster 112 via a distributed communication protocol. Manager node 104 uses the Snowflake algorithm to generate a unique task_id for task identification and tracking. Manager node 104 achieves automatic node discovery and task allocation through the Goose Gaggle protocol, supporting unlimited horizontal scaling. Manager node 104 loads load testing configuration information from a MySQL database and constructs Scenario and Transaction objects for task orchestration and execution.

[0032] like Figure 1 As further shown, the task orchestration unit 106 is configured to receive load testing tasks from the Web console module 102 and execute task orchestration. The metric aggregation unit 108 is configured to aggregate performance metric data reported by all Worker nodes 114 in real time. The report generation unit 110 is configured to generate a performance report based on the aggregated performance metric data. The Manager node 104 communicates with the Worker cluster 112 through a bidirectional connection, broadcasting tasks to the Worker cluster 112 via a distributed communication protocol and receiving performance metric data reported by the Worker cluster 112.

[0033] Worker cluster 112 includes multiple worker nodes 114. Each worker node 114 includes a Tokio asynchronous runtime thread pool 116, a data sharding processor 118, and a SIMD placeholder resolver 120. Each worker node 114 is configured to initialize the Tokio-based asynchronous runtime thread pool 116, which uses an event-driven non-blocking I / O model to handle concurrent connections.

[0034] The data sharding processor 118 is configured to obtain a CSV test data file from the data storage module 122 and calculate the data sharding range of its node based on the node index and the total number of nodes. The SIMD placeholder parser 120 is configured to locate and replace placeholders in the request template using a SIMD instruction set-based approach. The worker node 114 sends an HTTP request to the target service 130 through the Tokio asynchronous runtime thread pool 116.

[0035] The data storage module 122 includes a task configuration storage 124, a CSV test data storage 126, and a load testing log storage 128. The task configuration storage 124 is configured to store configuration data for load testing tasks. The CSV test data storage 126 is configured to store test data files in CSV format. The load testing log storage 128 is configured to store log data generated during the load testing process. The Manager node 104 is connected to the data storage module 122 and is used to obtain task configuration data. The Worker node 114 is connected to the data storage module 122 and is used to obtain CSV test data files and store load testing log data.

[0036] Target service 130 is located outside the distributed load testing platform system 100 and receives HTTP requests from worker node 114 as the target of load testing. The distributed load testing platform system 100 supports a fault tolerance mechanism that does not affect the overall task in the event of a single node failure. In some implementations, the distributed load testing platform system 100 can support 5000+ concurrent connections with a 2-core 1GB configuration and a memory footprint of only 512MB. In some implementations, the distributed load testing platform system 100 can generate 22,225 TPS with an 8-core 8GB configuration.

[0037] Reference Figure 2 The Worker node internal architecture 200 illustrates the internal module composition and connection relationships of the Worker node 114. The Worker node internal architecture 200 includes the Tokio asynchronous runtime 202, the data processing module 208, the SIMD placeholder parser 214, the log sampling module 220, and the security processing module 228.

[0038] The Tokio asynchronous runtime 202 includes an event-driven engine 204 and a non-blocking I / O processor 206. The event-driven engine 204 is configured to handle concurrent connections using an event-driven model, allowing a single thread to handle thousands of concurrent connections simultaneously. The non-blocking I / O processor 206 is configured to perform non-blocking I / O operations, preventing threads from being blocked while waiting for I / O operations to complete, thus supporting efficient handling of a large number of concurrent connections.

[0039] The data processing module 208 includes a CSV data sharder 210 and an atomic counter 212. Worker node 114 downloads CSV test data files from the OSS object storage service. The CSV data sharder 210 is configured to calculate the data sharding range for its node based on the node index and the total number of nodes, supporting automatic sharding of 100,000+ rows of CSV data. The distributed load testing platform system 100 uses the rayon library for parallel CSV parsing, fully utilizing multi-core CPU performance to accelerate data processing.

[0040] Atomic counter 212 is implemented based on the AtomicUsize type and configured to iteratively retrieve CSV data rows within a data shard range. Each Worker node 114 uses atomic counter 212 to achieve lock-free data access, avoiding multi-threaded data contention. In some implementations, the first request from Worker node 114 uses row 0 of the data shard range, the second request uses row 1, and when the number of requests exceeds the data shard range, atomic counter 212 iterates back to row 0 to continue using it.

[0041] The SIMD placeholder parser 214 includes a CSV variable parser 216 and a custom function parser 218. The CSV variable parser 216 is configured to parse CSV variable placeholders using the {{variable}} syntax format. The custom function parser 218 is configured to parse custom function placeholders using the {{_function()}} syntax format. The SIMD placeholder parser 214 optimizes the placeholder lookup operation using the SIMD instruction set of the memchr library, marking placeholder positions at compile time and performing replacement operations at runtime with O(1) complexity.

[0042] The log sampling module 220 includes an error log collector 222, a success log sampler 224, and a lock-free message channel 226. The error log collector 222 is configured to perform 100% log collection on error responses, ensuring that all error information is completely recorded. The success log sampler 224 is configured to perform log sampling on success responses according to a preset sampling probability. In some embodiments, the success log sampler 224 performs log sampling with a sampling probability of 1 / 500, that is, generating a random number between 0 and 499, and sampling is performed when the random number equals 0.

[0043] The log sampling module 220 uses a lock-free Flume channel to achieve high-performance multi-producer, multi-consumer message passing. The lock-free message channel 226 is configured to asynchronously send sampled logs to message queue 236. The log sampling module 220 asynchronously sends the sampled logs to a specified topic in the Kafka message queue 236. The log object contains fields such as request URL, request body, response status code, response body, and response time. The log throughput of the log sampling module 220 can reach 50,000 log entries per second.

[0044] The security processing module 228 includes an HMAC-SHA256 signer 230, an AES encryptor 232, and a Nonce anti-replay mechanism 234. The HMAC-SHA256 signer 230 is configured to use the HMAC-SHA256 algorithm to sign and verify requests, preventing tampering. The AES encryptor 232 is configured to encrypt sensitive data using the AES encryption algorithm. In some implementations, the AES encryptor 232 uses the AES-128-ECB encryption mode to encrypt and protect sensitive data. The security processing module 228 performs timestamp validity verification to ensure the timeliness of the request. The Nonce anti-replay mechanism 234 is configured to prevent replay attacks through a Nonce mechanism.

[0045] In operation, the Tokio asynchronous runtime 202 is connected to the data processing module 208, which in turn is connected to the SIMD placeholder parser 214. The SIMD placeholder parser 214 is connected to the security processing module 228, and the security processing module 228 is connected to the Tokio asynchronous runtime 202 to form a processing loop. The Tokio asynchronous runtime 202 is also connected to the log sampling module 220, and the lock-free message channel 226 is connected to the message queue 236 to achieve asynchronous transmission of log data.

[0046] In summary, the distributed load testing platform system 100 based on the Rust asynchronous architecture achieves distributed load testing functionality through the collaborative work of multiple modules. In terms of implementation principles, the distributed load testing platform system 100 adopts a distributed architecture of Manager node 104 and Worker cluster 112. Manager node 104 is responsible for task orchestration and metric aggregation. Worker node 114 uses an event-driven non-blocking I / O model to handle concurrent connections through an asynchronous runtime thread pool 116 based on Tokio. Data sharding processor 118 automatically shards test data according to node indexes to avoid data contention between multiple nodes. SIMD placeholder parser 120 uses the SIMD instruction set to mark placeholder positions during compilation and performs replacement operations at runtime with O(1) complexity. Log sampling module 220 balances log integrity and storage costs through a differentiated sampling strategy. Security processing module 228 provides security guarantees such as signature verification, data encryption, and anti-replay attack measures. In terms of beneficial effects, the distributed load testing platform system 100 enables a single thread to handle thousands of concurrent connections simultaneously through an asynchronous non-blocking model, thereby supporting a large number of concurrent requests with limited system resources; through a data sharding mechanism based on node indexes, each Worker node 114 obtains non-overlapping test data, reducing data contention issues in the distributed environment; through a placeholder parser optimized with the SIMD instruction set, it improves the efficiency of parameterization processing; through the jemalloc memory allocator and Arc smart pointers, it reduces memory fragmentation and data copying; through the Goose Gaggle protocol, it supports automatic node discovery and unlimited horizontal scaling; and through a fault-tolerance mechanism that does not affect the overall task in the event of a single node failure, it improves system reliability.

[0047] Based on the aforementioned distributed load testing platform system, referring to Figure 3 This invention also provides a distributed load testing method 300 based on Rust asynchronous architecture. Method 300 includes steps 302, 304, 306, 308, 310, and 312.

[0048] In step 302, the Manager node 104 receives and orchestrates the load testing tasks, and broadcasts them to multiple Worker nodes 114 via a distributed communication protocol. The Manager node 104 creates a ControllerRequestchannel and caches it in GOOSE_ATTACKS_SENDER for real-time metric aggregation. The Manager node 104 broadcasts the tasks to all Worker nodes 114 in the Worker cluster 112 via the GooseGaggle protocol. Each Worker node 114, upon receiving the task, initializes its Goose Worker mode and connects to the Manager node 104, awaiting the start command.

[0049] In step 304, an asynchronous runtime thread pool 116 based on Tokio is initialized on each Worker node 114, employing an event-driven non-blocking I / O model. The Tokio asynchronous runtime thread pool 116 enables a single thread to handle thousands of concurrent connections simultaneously, avoiding the overhead of thread switching in traditional synchronous blocking models.

[0050] In step 306, a CSV test data file is obtained on each Worker node 114, and the data sharding range of this node is calculated based on the node index and the total number of nodes. Each Worker node 114 downloads the CSV test data file from the OSS object storage service and calculates the range of data rows that this node is responsible for processing based on the node index and the total number of nodes, ensuring that there is no data duplication or contention among multiple Worker nodes 114.

[0051] In step 308, placeholders in the request template are located using a SIMD instruction set-based placeholder parser, and test data rows are retrieved from the data shard range using atomic counter 212 for placeholder replacement. The SIMD placeholder parser 120 marks all placeholder positions at compile time and performs the replacement operation at runtime with O(1) complexity. Atomic counter 212 implements lock-free data access based on the AtomicUsize type, iteratively retrieving CSV data rows from the data shard range.

[0052] In step 310, an HTTP request is sent to the target service 130 via the asynchronous runtime thread pool 116. The Tokio asynchronous runtime thread pool 116 adopts an event-driven non-blocking I / O model, enabling the Worker node 114 to handle a large number of concurrent HTTP requests with limited thread resources.

[0053] In step 312, the Manager node 104 aggregates the performance metrics data reported by all Worker nodes 114 in real time and generates a performance report. Worker nodes 114 report current TPS, P99 latency, and error rate to the Manager node 104 every second. Upon receiving the GetMetrics command, the Manager node 104 requests the latest metric data from all Worker nodes 114 via the ControllerRequest channel and aggregates the data from all Worker nodes 114 to calculate the total TPS. The HTML performance report generated by the Manager node 104 includes the total number of requests, the TPS curve, latency distribution, error rate statistics, and response time percentiles.

[0054] Reference Figure 4 This invention also provides a placeholder parsing method 400 based on the SIMD instruction set, including steps 402, 404, 406, 408, and 410. Method 400 improves the efficiency of parameterization processing by pre-marking placeholder position information during compilation, enabling runtime replacement operations to be completed in constant time complexity.

[0055] In step 402, the SIMD Finder from the memchr library is used at compile time to locate the start and end positions of all placeholders in the request template. SIMD Finder utilizes Single Instruction Multiple Data (SIMD) technology, processing multiple data items simultaneously with a single instruction, significantly improving the performance of string search operations. In step 402, SIMD Finder locates the positions of all "{{" and "}}" character sequences in the request template to pinpoint the boundaries of the placeholders.

[0056] In step 404, the index, start position, and end position of each placeholder are marked. Step 404 records the position information of each placeholder in the form of (index, start_pos, end_pos), where index represents the placeholder's sequence number, start_pos represents the placeholder's start position, and end_pos represents the placeholder's end position. After step 404 is completed, the position information of all placeholders is pre-marked during compilation, providing a basis for fast replacement operations at runtime.

[0057] In step 406, the system determines whether the current placeholder is a CSV variable or a user-defined function. Placeholders include CSV variable placeholders and user-defined function placeholders. CSV variable placeholders use the syntax \{{variable}\} and are used to retrieve the value of the corresponding column from the CSV test data for replacement. User-defined function placeholders use the syntax \{{_function()}\}, where the underscore prefix indicates that the placeholder is a user-defined function call.

[0058] like Figure 4 Further, if the placeholder is determined to be a CSV variable type in step 406, the process proceeds to step 408A to parse the CSV variable placeholder. In step 408A, the CSV variable parser 216 parses the variable name in the placeholder and determines the column index in the corresponding CSV test data. If the placeholder is determined to be a user-defined function type in step 406, the process proceeds to step 408B to parse the user-defined function placeholder. In step 408B, the user-defined function parser 218 parses the function name and parameter information in the placeholder.

[0059] Continue to refer to Figure 4 In step 410, the placeholder replacement operation is performed at runtime with O(1) complexity. Since steps 402 and 404 have already completed the lookup and marking of placeholder positions at compile time, the runtime replacement operation does not need to traverse the request template again to find the placeholder positions, but directly performs the replacement based on the pre-marked position information. For CSV variable placeholders, step 410 reads the value of the corresponding column from the CSV data row obtained from atomic counter 212 and replaces it. For custom function placeholders, step 410 executes the corresponding function and replaces the return value in the request template.

[0060] In some implementations, the request template is ` / api / user / \{\{userId\}\} / order?time=\{\{\_time()\}\}`, where `\{\{userId\}\}` is a CSV variable placeholder, and `\{\{\_time()\}\}` is a custom function placeholder. During compilation, steps 402 and 404 mark `\{\{userId\}\}` as corresponding to the third column of the CSV test data, and `\{\{\_time()\}\}` as a custom function that returns the current timestamp. At runtime, step 410 retrieves the value of the third column (e.g., "100") from the CSV data row and replaces `\{\{userId\}\}`, executes the `\_time()` function to obtain the current timestamp (e.g., 1699999999) and replaces `\{\{\_time()\}\}`. The final generated request path is ` / api / user / 100 / order?time=1699999999`.

[0061] Method 400 supports parallel processing paths for two placeholder syntax formats, parsing and replacing CSV variables and user-defined functions respectively. By combining compile-time preprocessing and runtime O(1) complexity replacement, Method 400 achieves improved parameterization performance.

[0062] Furthermore, refer to Figure 5 This invention also provides a security processing and log sampling method 500, including steps 502, 504, 506, 508, 510, 512, 514, and 516. Method 500 performs the security processing steps before sending an HTTP request and performs the log sampling steps after receiving a response.

[0063] In step 502, the system determines whether security processing is required. This security processing step ensures the integrity, confidentiality, and prevention of replay attacks of the request. If security processing is required in step 502, the process proceeds to step 504. If security processing is not required in step 502, the process directly proceeds to step 510B to send an HTTP request and receive a response.

[0064] In step 504, the request is signed using the HMAC-SHA256 algorithm. The HMAC-SHA256 signer 230 generates a message authentication code based on the SHA-256 hash algorithm to verify the request signature and prevent the request from being tampered with during transmission. After step 504 is completed, the process proceeds to step 506.

[0065] In step 506, the sensitive data is encrypted using the AES encryption algorithm. The AES encryptor 232 encrypts the sensitive data fields in the request to ensure the confidentiality of sensitive information during transmission. In some implementations, the AES encryptor 232 performs the encryption operation using the AES-128-ECB encryption mode. After step 506 is completed, the process proceeds to step 508.

[0066] In step 508, a nonce mechanism is used to prevent replay attacks. The nonce anti-replay mechanism 234 generates a unique random number for each request, ensuring that the same request cannot be intercepted and resent by an attacker. After step 508 is completed, the process proceeds to step 510A to send an HTTP request and receive a response.

[0067] In steps 510A and 510B, Worker node 114 sends an HTTP request to target service 130 and receives a response through Tokio asynchronous runtime thread pool 116. Step 510A corresponds to a request sending path that has undergone secure processing, while step 510B corresponds to a request sending path that has not undergone secure processing.

[0068] In step 512, the system determines whether the response is an error response. The log sampling step employs different collection strategies based on the response type to reduce storage costs while ensuring the integrity of error information. If the response is determined to be an error response in step 512A or step 512B, the process proceeds to step 514. If the response is determined not to be an error response in step 512A or step 512B, the process proceeds to step 516.

[0069] In step 514, 100% logging is performed on error responses. Error log collector 222 ensures that all error response log information is completely recorded, including fields such as request URL, request body, response status code, response body, and response time. Step 514A corresponds to error log collection for paths that have undergone secure processing, and step 514B corresponds to error log collection for paths that have not undergone secure processing.

[0070] In step 516, log sampling is performed on successful responses according to a preset sampling probability. Success log sampler 224 samples successful responses according to the preset sampling probability. In some embodiments, success log sampler 224 performs log sampling with a sampling probability of 1 / 500, that is, it generates a random number between 0 and 499, and samples when the random number equals 0. Step 516A corresponds to successful log sampling along a secure processing path, and step 516B corresponds to successful log sampling along a path without secure processing.

[0071] After the log sampling step is completed, the lock-free message channel 226 asynchronously sends the sampled logs to the message queue 236. The lock-free message channel 226 is implemented based on Flume's lock-free channel, supporting high-performance message delivery in a multi-producer, multi-consumer mode. Method 500 achieves request security and optimizes log storage costs by combining the security processing step and the log sampling step.

[0072] In summary, the distributed load testing method based on Rust asynchronous architecture achieves distributed load testing functionality through the coordinated execution of multiple steps. Regarding the implementation principle, method 300 receives load testing tasks through the Manager node 104 and broadcasts the tasks to the Worker cluster 112 via the Goose Gaggle protocol. Each Worker node 114 initializes an asynchronous runtime thread pool 116 based on Tokio, employing an event-driven non-blocking I / O model to handle concurrent connections. A data sharding processor 118 calculates the data sharding range of its node based on the node index to avoid data contention between multiple nodes. Method 400 uses a SIMD instruction set-based placeholder parser to mark placeholder positions during compilation and performs replacement operations at runtime with O(1) complexity. Method 500 employs security processing steps to execute HMAC-SHA256 signature, AES encryption, and a Nonce anti-replay mechanism to ensure request security. Furthermore, a log sampling step performs 100% log collection on error responses and log sampling on successful responses according to a preset sampling probability to balance log integrity and storage costs. In terms of beneficial effects, this method enables a single thread to handle thousands of concurrent connections simultaneously through an asynchronous non-blocking model, thereby supporting a large number of concurrent requests with limited system resources; it enables each Worker node to obtain non-overlapping test data through a node index-based data sharding mechanism, reducing data contention issues in a distributed environment; it improves parameterization processing efficiency through a placeholder parser optimized with a SIMD instruction set; it reduces storage costs while ensuring complete recording of error information through a differentiated log sampling strategy; and it supports interface stress testing scenarios requiring signature verification and data encryption through secure processing steps.

[0073] This invention also discloses a readable storage medium.

[0074] A computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any of the above embodiments. The computer-readable storage medium may include any entity or device capable of carrying a computer program, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc. The computer program includes computer program code. The computer program code may be in the form of source code, object code, an executable file, or some intermediate form, etc. The computer-readable storage medium may include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc.

[0075] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0076] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a system including a processing module or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0077] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A distributed load testing platform system based on Rust asynchronous architecture, characterized in that, include: The Web console module is configured to provide a task management interface for creating and managing load testing tasks; The Manager node is configured to receive load testing tasks from the Web console module, perform task orchestration, and broadcast the tasks to the Worker cluster via a distributed communication protocol. A worker cluster comprises multiple worker nodes. Each worker node is configured to: initialize an asynchronous runtime thread pool based on Tokio, handling concurrent connections using an event-driven non-blocking I / O model; retrieve a CSV test data file from the data storage module and calculate the data sharding range for its node based on the node index and the total number of nodes; locate and replace placeholders in the request template using a SIMD instruction set-based placeholder parser; send HTTP requests to the target service through the asynchronous runtime thread pool; and... The data storage module is configured to store task configuration data, CSV test data files, and load test log data. The Manager node is also configured to aggregate performance metrics data reported by all Worker nodes in real time and generate performance reports.

2. The system according to claim 1, characterized in that, The placeholder parser based on the SIMD instruction set is configured as follows: Use the SIMD Finder from the memchr library during compilation to find the start and end positions of all placeholders in the request template; Mark the index, start position, and end position of each placeholder; and The placeholder replacement operation is performed at runtime with O(1) complexity.

3. The system according to claim 2, characterized in that, The placeholders include CSV variable placeholders and user-defined function placeholders. The CSV variable placeholders use the {{variable}} syntax format, and the user-defined function placeholders use the {{_function()}} syntax format.

4. The system according to claim 1, characterized in that, Each Worker node is also configured as follows: Use an atomic counter to repeatedly retrieve CSV data rows within the data fragment range; and Lock-free data access is achieved based on the atomic counter, avoiding multi-threaded data contention.

5. The system according to claim 1, characterized in that, It also includes a log sampling module, which is configured as follows: Perform 100% log collection on error responses; Log sampling is performed on successful responses according to a preset sampling probability; as well as The sampled logs are sent asynchronously to the message queue via a lock-free message channel.

6. The system according to claim 5, characterized in that, Each Worker node is also configured to perform security processing operations, including: Use the HMAC-SHA256 algorithm to verify the signature of the request; Sensitive data is encrypted using the AES encryption algorithm; and The Nonce mechanism is used to prevent replay attacks.

7. A distributed load testing method based on Rust asynchronous architecture, characterized in that, Includes the following steps: The Manager node receives load testing tasks and orchestrates them, and broadcasts the tasks to multiple Worker nodes through a distributed communication protocol. An asynchronous runtime thread pool based on Tokio is initialized on each Worker node, using an event-driven non-blocking I / O model; Obtain the CSV test data file on each Worker node, and calculate the data sharding range for this node based on the node index and the total number of nodes; The placeholders in the request template are located by a placeholder parser based on the SIMD instruction set, and test data lines are obtained from the data fragment range using an atomic counter to replace the placeholders. The asynchronous runtime thread pool is used to send HTTP requests to the target service. as well as The Manager node aggregates performance metrics data reported by all Worker nodes in real time and generates performance reports.

8. The method according to claim 7, characterized in that, The step of locating placeholders in the request template using a SIMD instruction set-based placeholder parser includes: Use the SIMD Finder from the memchr library during compilation to find the start and end positions of all placeholders in the request template; Mark the index, start position, and end position of each placeholder; and The placeholder replacement operation is performed at runtime with O(1) complexity.

9. The method according to claim 8, characterized in that, The placeholders include CSV variable placeholders and user-defined function placeholders. The CSV variable placeholders use the {{variable}} syntax format, and the user-defined function placeholders use the {{_function()}} syntax format.

10. The method according to claim 7, characterized in that, It also includes a log sampling step, which includes: Perform 100% log collection on error responses; Successful responses are sampled using a pre-defined sampling probability; and The sampled logs are sent asynchronously to the message queue via a lock-free message channel.

11. The method according to claim 7, characterized in that, It also includes performing security processing steps before sending the HTTP request, the security processing steps including: The request is signed using the HMAC-SHA256 algorithm; Sensitive data is encrypted using the AES encryption algorithm; and The Nonce mechanism is used to prevent replay attacks.

12. A readable storage medium, characterized in that, The readable storage medium stores computer instructions that, when executed by a processor, implement the method as described in any one of claims 7-11.

Images