A current limiting method and system combining token bucket and leaky bucket for payment system

Abstract

The present invention relates to the technical field of payment system flow control, and specifically discloses a flow limiting method and system for a payment system that combines a token bucket and a leaky bucket. The method comprises: S1: obtaining payment system indicators and calculating a health score; S2: determining the load status of the payment system based on the health score, and adjusting the token generation rate and token bucket capacity; the adjustment step size of the token generation rate and token bucket capacity is adjusted based on the response of the payment system after multiple adjustments; S3: receiving a service request and determining whether to approve or reject the service request based on the number of available tokens; and S4: reading and processing approved service requests based on a leaky bucket algorithm. The present invention improves system stability and user experience by dynamically adjusting the token generation rate, designing a hierarchical token bucket, and implementing a leaky bucket smoothing output mechanism.

Description

Technical Field

[0001] The present invention relates to the technical field of payment system flow control, and in particular to a flow limiting method and system combining a token bucket and a leaky bucket for a payment system. Background Art

[0002] Currently, payment systems lack effective rate limiting mechanisms, making them significantly vulnerable to high concurrent requests. Unlike typical web applications, payment systems have extremely high security and real-time requirements. Every payment operation undergoes a rigorous verification process, and data consistency and integrity must be ensured. Therefore, any rate limiting solution must account for these special requirements. An ideal rate limiting mechanism must not only guarantee transaction success rates—that is, completing as many valid transactions as possible during peak hours—but also be able to identify anomalous behavior, such as detecting and blocking suspicious repeated payment attempts. Furthermore, the rate limiting mechanism needs to be highly scalable and flexible to support different levels of service segmentation.

[0003] In the current technological environment, although there are a variety of flow-limiting products and solutions, there are still many limitations and shortcomings when dealing with complex application scenarios, especially payment systems used in multiple countries.

[0004] (1) Fixed rate limiting is a simple and direct method of limiting traffic by setting a fixed upper limit on the number of requests. During peak hours, even if the set maximum number of requests is reached, it may still not be enough to meet actual demand. This rigid method of limiting traffic not only wastes resources, but may also cause users to encounter frequent request rejections or timeouts in high-concurrency scenarios, seriously affecting the user experience.

[0005] (2) Token bucket and leaky bucket algorithms are two common flow control mechanisms, used to control the speed of inflow and outflow, respectively. These two algorithms can alleviate system pressure to a certain extent, but when used alone, they are difficult to fully cover the various complex scenarios in the payment system.

[0006] Chinese invention patent application publication number CN119520187A discloses a unified computing power access management method based on API technology. This method utilizes adaptive current limiting to dynamically adjust the token generation rate and leaky bucket leakage rate based on real-time interface traffic changes, providing refined control and smoothing of traffic. While this invention combines the token bucket and leaky bucket algorithms, it fails to account for the variability of requests at different levels within the payment system.

[0007] Unlike typical throttling scenarios, in payment systems, certain core transaction APIs, such as payment confirmation and refund processing, have extremely high response time requirements. However, common query APIs, such as order and account balance inquiries, have relatively loose requirements. If all requests were treated equally, critical services might not be processed promptly because their resources were occupied by lower-priority requests, impacting overall system stability and user experience. Summary of the Invention

[0008] The present invention aims to address the aforementioned issues. To this end, it provides a method and system for limiting current in a payment system by combining a token bucket with a leaky bucket. This method improves system stability and user experience by dynamically adjusting the token generation rate, designing a hierarchical token bucket, and implementing a leaky bucket smoothing mechanism.

[0009] The present invention provides a current limiting method combining a token bucket and a leaky bucket for a payment system, and the technical solution adopted is as follows:

[0010] S1: Obtain payment system indicators and calculate health scores;

[0011] S2: Determine the load status of the payment system based on the health score and adjust the token generation rate and token bucket capacity. The adjustment step size of the token generation rate and token bucket capacity is adjusted based on the response of the payment system after multiple adjustments.

[0012] When the health score is less than the low load threshold, the token generation rate and token bucket capacity are reduced;

[0013] When the health score is greater than the high load threshold, increase the token generation rate and token bucket capacity;

[0014] When the health score is less than or equal to the high load threshold and greater than or equal to the low load threshold, the token generation rate and token bucket capacity remain unchanged;

[0015] S3: Receive the service request and decide whether to approve or reject the service request based on the number of available tokens;

[0016] S4: Read and process the passed service requests according to the leaky bucket algorithm.

[0017] Furthermore, the payment system indicators include: CPU utilization, memory usage, network bandwidth, real-time traffic, response time and error rate; each indicator is calculated by weighted summation to obtain a health score.

[0018] Furthermore, the function of health score is:

[0019] Health score = w1 (1-normalized CPU utilization) + w2 (1-normalized memory usage) + w3 (normalized network bandwidth) + w4 (1-normalized real-time traffic) + w5 (1-normalized response time) + w6 (1-normalized error rate)

[0020] Among them, w1 is the first weight coefficient, w2 is the second weight coefficient, w3 is the third weight coefficient, w4 is the fourth weight coefficient, w5 is the fifth weight coefficient, and w6 is the sixth weight coefficient.

[0021] Furthermore, different token buckets are set for service requests of different priorities.

[0022] When the health score is less than the low load threshold, the token generation rate and token bucket capacity of high-priority service requests are increased, and the token generation rate and token bucket capacity of low-priority service requests are reduced;

[0023] When the health score is greater than the high load threshold, the token generation rate and token bucket capacity of high-priority service requests are reduced, and the token generation rate and token bucket capacity of low-priority service requests are increased.

[0024] Furthermore, the specific process of S3 is as follows:

[0025] S31: Receive a service request and search the corresponding token bucket from the hash table based on the user ID or IP address;

[0026] S32: Update the status of the token bucket;

[0027] S33: Check whether the service request complies with the flow limiting rules. If not, reject the service request. If it complies with the flow limiting rules, execute S34.

[0028] S34: Determine whether there are available tokens in the token bucket. If there are enough tokens, allow the request to pass and deduct the corresponding tokens; if there are not enough tokens, reject the service request.

[0029] Furthermore, the flow limiting rules include exceeding the limit of the number of service requests per unit time. The timestamps of the latest N service requests stored in the queue are used to determine whether the service request of the user ID or IP address meets the flow limiting rules;

[0030] In S34, if the request is allowed to pass, the service request timestamp is added to the queue.

[0031] Furthermore, in S3, the passed service request is added to the MQ message queue; in S4, according to the leaky bucket algorithm, the passed service request is read and processed from the MQ message queue at a fixed rate.

[0032] Furthermore, if the response of the payment system after multiple adjustments satisfies conditions (1) to (4) at the same time, the token generation rate adjustment step and the token bucket capacity adjustment step are increased;

[0033] Case (1): Health score improved,

[0034] Case (2): Request failure rate decreases,

[0035] Case (3): Response time is shortened,

[0036] Case (4): Throughput increases;

[0037] If the response of the payment system after multiple adjustments falls into any of the situations (5) to (8), the token generation rate adjustment step and the token bucket capacity adjustment step are reduced;

[0038] Case (5): Unstable health score,

[0039] Case (6): The request failure rate has no significant change or worsens.

[0040] Case (7): Response time is extended,

[0041] Case (8): Throughput decreases.

[0042] The present invention also provides a current limiting system combining a token bucket and a leaky bucket for a payment system, which adopts the following technical solution: comprising: a scoring module, a token adjustment module, a token allocation module, a leaky bucket module and a step adjustment factor module,

[0043] The scoring module is used to obtain indicators of the payment system and calculate the health score;

[0044] The token adjustment module is used to determine the load status of the payment system according to the health score and adjust the token generation rate and token bucket capacity;

[0045] The token allocation module is used to receive a service request and determine whether to approve or reject the service request based on the number of available tokens;

[0046] The leaky bucket module is used to read and process the passed service requests according to the leaky bucket algorithm;

[0047] The step adjustment factor module is used to adjust the adjustment step of the token generation rate and the token bucket capacity according to the response of the payment system after multiple adjustments.

[0048] The above one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects:

[0049] 1. The present invention improves the stability of the payment system.

[0050] This invention significantly improves the stability of the payment system by introducing dynamic adjustment of token generation speed, a hierarchical token bucket design, and a leaky bucket smoothing output mechanism. First, the monitoring tool Grafana is used to collect real-time system status indicators (such as CPU utilization, memory usage, and network bandwidth), and a health score function is used to comprehensively evaluate the system's current load. This intelligent dynamic adjustment mechanism enables the system to flexibly adjust the rate limiting strategy based on real-time monitoring data, avoiding resource waste or system overload caused by static rate limiting rules. For example, during a promotional event, when a large number of users simultaneously initiate service requests, the system can promptly reduce the token generation rate based on the health score to prevent an influx of requests, effectively preventing system crashes.

[0051] Furthermore, a hierarchical token bucket design further enhances system stability. By setting up independent token bucket instances for service requests of different priorities, and allocating different initial token counts and generation rates, this ensures that critical services receive sufficient resources even under high load. This hierarchical design not only improves the availability of critical services but also achieves optimal resource allocation.

[0052] 2. The present invention helps to improve user experience.

[0053] This invention significantly improves the user experience through refined current limiting control and traffic smoothing. First, a hash table and queue data structure are used to implement user-level personalized current limiting strategies. The hash table is used to quickly find the token bucket instance corresponding to the user ID or IP address, while the queue stores the timestamps of each user's most recent N requests to facilitate determination of compliance with current limiting rules. This design not only improves search efficiency but also enables personalized current limiting policy configuration for specific users or IP addresses, ensuring that legitimate users' normal requests are processed promptly and reducing the probability of users encountering request rejections or timeouts.

[0054] After using a token bucket for initial flow control at the access layer, a leaky bucket algorithm was introduced to smooth traffic flow and prevent downstream services from being overloaded by sudden traffic bursts. The optimized leaky bucket mechanism, combined with MQ message queues, enables asynchronous request processing, reducing synchronization latency. This not only improves system responsiveness but also enhances fault tolerance. Even if a consumer fails, other consumers can continue processing requests, ensuring system continuity and reliability, significantly improving the user experience.

[0055] 3. The present invention improves the scalability and flexibility of the payment system.

[0056] The design of the present invention fully considers the scalability and flexibility of the system, so that the payment system can easily cope with changing business needs and technical challenges. First, the hierarchical token bucket design allows for dynamic adjustment of the current limiting parameters according to different service interface types and user groups, ensuring that the system can maintain efficient operation in high-concurrency scenarios. Secondly, the introduction of the feedback closed-loop mechanism enables the current limiting strategy to have adaptive adjustment capabilities. The system regularly evaluates the adjusted current limiting effect and further optimizes the current limiting parameters based on actual conditions. This continuous optimization capability not only improves the flexibility of the system, but also ensures that the current limiting strategy is always in the best state. In addition, the leaky bucket mechanism combined with the MQ message queue realizes asynchronous processing of requests, further enhancing the scalability and fault tolerance of the system. Even in the face of large-scale concurrent requests, the system can maintain efficient and stable operation, providing strong technical support for enterprises.

[0057] 4. The present invention reduces the operation and maintenance costs and complexity of the payment system.

[0058] The present invention significantly reduces the operation and maintenance costs and complexity of the payment system through automated and intelligent design. First, the introduction of dynamic adjustment mechanisms and feedback closed-loop mechanisms reduces the need for manual intervention. The system can automatically adjust the current limiting strategy based on real-time monitoring data, without the need for operation and maintenance personnel to manually modify configuration files or restart services, greatly reducing the operation and maintenance burden. Secondly, the hierarchical token bucket design and the data structure based on hash tables and queues simplify the implementation process of current limiting control. Operation and maintenance personnel only need to focus on how to reasonably set the priority and initial parameters of each service interface, and the system will automatically complete subsequent current limiting operations. In addition, the leaky bucket mechanism combined with the MQ message queue not only improves the system's processing capacity, but also enhances the system's fault tolerance and reduces system downtime caused by single point failures. These improvement measures work together to significantly reduce operation and maintenance costs and complexity, and bring considerable economic benefits.

[0059] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0061] Figure 1 It is a flow chart of the method provided by the present invention.

[0062] Figure 2This is a system structure diagram provided by the present invention.

[0063] Reference numerals:

[0064] 1. Scoring module; 2. Token adjustment module; 3. Token allocation module; 4. Leaky bucket module; 5. Step adjustment factor module. DETAILED DESCRIPTION

[0065] To make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the present invention. Obviously, the embodiments described are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. The following embodiments are used to illustrate the present invention, but are not used to limit the scope of the present invention.

[0066] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0067] The following combination Figure 1 and Figure 2 The present invention is further described in detail, and a current limiting method and system combining a token bucket and a leaky bucket for a payment system is described.

[0068] In this embodiment, Figure 1 As shown, a flow limiting method combining a token bucket and a leaky bucket for a payment system is provided, comprising the following steps:

[0069] S1: Obtain payment system indicators and calculate health scores.

[0070] Payment system metrics include CPU utilization, memory usage, network bandwidth, real-time traffic, response time, and error rate. A health score is calculated using a weighted summation of these metrics.

[0071] The metrics selected when defining the health scoring function (CPU utilization, memory usage, network bandwidth, real-time traffic, response time, and error rate) are all key factors that directly impact system performance and stability. Together, they determine the system's current load and service quality. CPU utilization and memory usage directly impact the server's ability to process requests. Excessive usage can cause the system to respond more slowly or even crash. Network bandwidth reflects data transmission bottlenecks, which is particularly important in high-concurrency scenarios. Real-time traffic provides information on the number of requests currently being processed by the system, helping to assess system workload. Response time is a direct indicator of user experience; shorter response times mean greater user satisfaction. The error rate refers to the proportion of errors that occur while the system is processing requests. A higher error rate generally indicates system issues.

[0072] This example uses the monitoring tool Grafana to collect indicators such as CPU utilization, memory usage, and network bandwidth of the payment system.

[0073] The goal of the health score is to comprehensively reflect the current state of the system (such as CPU utilization, memory usage, and network latency), and to determine whether the system is healthy based on these metrics. Different metrics may have varying degrees of impact on system health, so each metric needs to be weighted. For example, if the system is CPU-sensitive, CPU utilization can be given a higher weight, while network latency, which has a significant impact on user experience, can be given a higher weight. Weighting can be determined based on experience or historical data analysis.

[0074] Because different metrics have different value ranges, all metrics need to be normalized to a common range, such as 0 to 1. This embodiment uses maximum and minimum value normalization. For example, CPU utilization is normalized from 0% to 100% to 0 to 1, and network latency is normalized from 0ms to 500ms to 0 to 1.

[0075] Combine the normalized index values ​​with the weights to design the final health score function:

[0076] Health score = w1 (1-normalized CPU utilization) + w2 (1-normalized memory usage) + w3 (normalized network bandwidth) + w4 (1-normalized real-time traffic) + w5 (1-normalized response time) + w6 (1-normalized error rate)

[0077] Among them, w1 is the first weight coefficient, w2 is the second weight coefficient, w3 is the third weight coefficient, w4 is the fourth weight coefficient, w5 is the fifth weight coefficient, w6 is the sixth weight coefficient, and it satisfies .

[0078] S2: Determine the load status of the payment system based on the health score and adjust the token generation rate and token bucket capacity; the adjustment step of the token generation rate and token bucket capacity is adjusted according to the response of the payment system after multiple adjustments.

[0079] In order to facilitate subsequent flow control decisions, two thresholds need to be set: low load threshold and high load threshold. By comparing the health score with the two thresholds, the load status of the payment system can be determined, and then how to adjust the token generation rate and token bucket capacity can be determined. Specifically:

[0080] When the health score is less than the low load threshold, it means that the system is overloaded and the token generation rate and token bucket capacity need to be reduced. hour, , .

[0081] When the health score is greater than the high load threshold, it means that the system is idle and the token generation rate and token bucket capacity can be appropriately increased. hour, , .

[0082] When the health score is less than or equal to the high load threshold and greater than or equal to the low load threshold, the token generation rate and token bucket capacity remain unchanged.

[0083] in, Score your health, is the low load threshold, is the high load threshold, is the token generation rate at the current moment, is the token generation rate at the next moment, Adjust the step size for the token generation rate, is the token bucket capacity at the current moment, is the token bucket capacity at the next moment, The token bucket capacity adjustment step size. and The choice of is crucial and should not be too large to avoid system oscillation. The appropriate step size should be selected according to the actual situation.

[0084] Based on experience and historical data analysis, in this embodiment, the low load threshold is set to 0.3 and the high load threshold is set to 0.7.

[0085] In addition, you need to set upper and lower limits for the token generation rate and token bucket capacity. The adjusted token generation rate and token bucket capacity should not exceed the corresponding upper and lower limits.

[0086] Based on the characteristics of the payment system, this implementation also sets up independent token buckets for service interfaces or user groups of different priorities, ensuring that critical services receive sufficient resources even under high load. By creating independent token bucket instances for each service level or user group and dynamically adjusting its parameters (token generation rate and token bucket capacity) based on the payment system's load, system stability and user experience are effectively guaranteed.

[0087] Specifically, requests corresponding to service interfaces are assigned different priorities based on their type. For example, payment confirmations and refunds are prioritized high, while order inquiries and account balance inquiries are prioritized low. User groups are prioritized based on their importance. For example, VIP customers are prioritized high, while general customers are prioritized low. Of course, the number of priority levels is not limited to two, depending on the actual situation.

[0088] Token bucket status:

[0089]

[0090] in, is the number of tokens at the current moment, is the capacity of the token bucket, is the token generation rate, i.e. the number of tokens generated per second, , is the time interval between two moments, is the number of tokens at the previous moment, It is to take the minimum value.

[0091] The calculation formula for the token bucket capacity is:

[0092]

[0093] in, is the maximum number of requests allowed per second, The interval for replenishing tokens is usually 1 second. Tokens are generated according to the token generation rate, and the total number of tokens does not exceed the token bucket capacity.

[0094] When adjusting the token generation rate and token bucket capacity, it is necessary to dynamically adjust the token bucket capacity and token generation rate of each level based on priority to ensure that key services get resources first. Adjustment rules based on different request priorities:

[0095] When the health score is below the low load threshold and the system is overloaded, the token generation rate and token bucket capacity for high-priority service requests are increased, while those for low-priority service requests are reduced. This means that the token generation rate and token bucket capacity for low-priority service requests are significantly reduced, while the token generation rate and token bucket capacity for high-priority service requests are slightly increased to ensure that high-priority service requests are processed smoothly.

[0096] When the health score exceeds the high load threshold and the system is idle, the token generation rate and token bucket capacity for high-priority service requests are reduced, while the token generation rate and token bucket capacity for low-priority service requests are increased. It should be noted that by properly setting the initial values ​​for the token generation rate and token bucket capacity for high-priority service requests, you can prevent the system from being idle and insufficiently processing priority service requests. These initial values ​​are set and adjusted based on actual usage.

[0097] When the health score is less than or equal to the high load threshold and greater than or equal to the low load threshold, the token generation rate and token bucket capacity are kept unchanged, that is, the token generation rate and token bucket capacity of high-priority service requests and low-priority service requests are not changed.

[0098] Taking 2 priorities as an example, the token generation rate is calculated as follows:

[0099] When the system is overloaded:

[0100]

[0101]

[0102] in, The token generation rate at the next moment for high priority service requests, The current token generation rate for high-priority service requests, Adjust the token generation rate step size for high priority service requests, The token generation rate at the next moment for low-priority service requests, The current token generation rate for low-priority service requests, For low priority service requests, .

[0103] When the system is idle:

[0104]

[0105] .

[0106] The token bucket capacity is adjusted in a similar way to the token generation rate.

[0107] This embodiment introduces a feedback mechanism to evaluate the current limiting effect after adjustment, and adjusts the adjustment step size of the token generation rate and the token bucket capacity according to the response of the payment system after multiple adjustments.

[0108] This embodiment ensures that the system's throttling policy can dynamically adjust based on actual operating conditions, thereby better responding to changes in system load. If the system remains overloaded after adjustments, the throttling rules are further tightened; otherwise, they are appropriately relaxed. Specifically, a feedback loop mechanism is a key component that ensures the system's throttling policy can dynamically adjust based on actual operating conditions. First, the system regularly collects the latest health score and other key indicator data from the monitoring layer (such as Grafana) and uses this data to evaluate the effectiveness of the current throttling policy. If the evaluation results indicate that the system is still overloaded (the health score is below the low load threshold), the throttling rules are further tightened. The choice of adjustment step size is crucial and should not be too large to avoid system oscillation. An appropriate step size should be selected based on actual conditions. Subsequently, the adjusted throttling policy is applied to the system in real time, and a new round of monitoring and evaluation is initiated to verify the effectiveness of the adjustments. This process balances response speed and system overhead by setting a reasonable evaluation interval, ensuring continuous optimization of the throttling policy and forming a complete feedback loop. This allows the payment system to effectively protect itself from overload in the face of sudden traffic bursts while fully utilizing resources when they are available, providing a better user experience. This adaptive adjustment capability not only improves the stability of the system, but also enhances its flexibility and fault tolerance.

[0109] In high-concurrency scenarios or abnormal conditions, the system needs to dynamically adjust the token bucket capacity and token generation rate at each level to ensure that critical services receive priority resources. To improve the system's adaptability, this embodiment also introduces a Step Adjustment Factor (SAF), allowing the step size of each adjustment (token generation rate adjustment step size and token bucket capacity adjustment step size) to dynamically change based on the system's response.

[0110] Design and implementation of SAF:

[0111] Initial setting: First, set an initial value for the adjustment step size. This value can be determined based on historical data or empirical rules.

[0112] Step Adjustment Factor: A step adjustment factor (SAF) is introduced to adjust the magnitude of each adjustment. The SAF is dynamically adjusted based on the payment system's historical response. In this embodiment, if the current adjustment is found to significantly improve system load, the SAF can be appropriately increased, for example, by increasing the token generation rate adjustment step size and token bucket capacity adjustment step size to 1.1 times the current value. Conversely, if system performance fluctuates significantly or no significant improvement is seen after the adjustment, the SAF can be reduced, for example, to 0.9 times the current value. Otherwise, the current token generation rate adjustment step size and token bucket capacity adjustment step size remain unchanged.

[0113] The situations where the system load is significantly improved are:

[0114] Improved health score: If the adjusted payment system's health score improves significantly, it indicates that the system's current resource usage is more balanced and the load has been effectively alleviated. For example, key indicators such as CPU utilization and memory usage have dropped to a more reasonable level.

[0115] Reduced request failure rate: As the system load is reduced, the ability to process requests is enhanced, resulting in a significant decrease in the request failure rate. For example, before the adjustment, 30% of requests might fail due to overload, but after the adjustment, this ratio can be reduced to less than 5%.

[0116] Shortened response time: As a direct indicator of user experience, shorter system response time means improved user-perceived service quality. For example, the average response time has been reduced from over 1 second to under 250 milliseconds.

[0117] Increased throughput: The system can handle more transactions or queries (TPS / QPS) while maintaining or improving quality of service. For example, the core transaction API can reach and maintain a target of 1500 TPS in a steady state.

[0118] If the response of the payment system after multiple adjustments meets the above conditions at the same time, the token generation rate adjustment step and the token bucket capacity adjustment step are increased.

[0119] If system performance fluctuates significantly or does not show significant improvement after adjustment, it may indicate the following situations:

[0120] Unstable health score: Even after adjusting the current limiting parameters, if the system health score still fluctuates within a large range and does not converge to an ideal value, the adjustment measures have not achieved the expected results. For example, if the health score drops sharply from 0.8 to 0.3 and then rebounds to 0.7 in a short period of time, it indicates that the system status is extremely unstable.

[0121] The request failure rate remains unchanged or worsens: If the request failure rate remains high after adjustment, or even increases compared to before the adjustment, this indicates that the current rate limiting strategy is not suitable for the current traffic pattern or system status. For example, the original 10% request failure rate may increase to 15% after adjustment.

[0122] Increased response time: If the average user response time increases rather than decreases after the system adjustment, this is also a sign that performance has not improved. For example, the original average response time was 300 milliseconds, but after the adjustment it increased to over 500 milliseconds.

[0123] Throughput Decline: Despite implementing throttling measures, the system's overall throughput declined, failing to meet business needs. For example, the core transaction API, which previously could handle a stable 1500 TPS, could only maintain around 1200 TPS after adjustments.

[0124] If the response of the payment system after multiple adjustments falls into any of the above situations, the token generation rate adjustment step and the token bucket capacity adjustment step will be reduced.

[0125] S3: Receive the service request and determine whether to approve or reject the service request based on the number of available tokens.

[0126] This embodiment uses a hash table and queue data structure. The hash table is used to quickly find the token bucket instance corresponding to the user ID or IP address, while the queue stores the timestamps of the last N service requests for each user to facilitate determination of compliance with the current limiting rules. Specifically, the hash table is used to quickly find the token bucket instance corresponding to the user ID or IP address. Key: User ID or IP address. Value: Corresponding token bucket object. Queue: Stores the timestamps of the last N service requests for the user ID or IP address.

[0127] The specific process is:

[0128] S31: Receive service request. When a service request arrives, search the corresponding token bucket from the hash table according to the user ID or IP address.

[0129] S32: Update the status of the token bucket and calculate the number of newly added tokens based on the last update time.

[0130] S33: Check whether the service request complies with the flow limiting rules. If it does not comply with the flow limiting rules, reject the service request; if it complies with the flow limiting rules, execute S34.

[0131] In this embodiment, the current limiting rule includes exceeding the limit on the number of service requests per unit time. The timestamps of the last N service requests stored in the queue are used to determine whether the request frequency of the user ID or IP address complies with the current limiting rule. For example, a maximum of five service requests are allowed within one second. For example, if the request timestamps in the queue determine that the current service request is the sixth service request within one second, then the current service request is considered to not comply with the current limiting rule. Other current limiting rules may also be set based on actual circumstances.

[0132] S34: Determine whether there are available tokens in the token bucket. If there are sufficient tokens, allow the service request to pass, deduct the corresponding tokens, and add the service request timestamp to the queue; if there are insufficient tokens, reject the service request.

[0133] It should be noted that in this step, service requests of different priorities correspond to different token buckets.

[0134] S4: Read and process the passed service requests according to the leaky bucket algorithm.

[0135] The basic principle of the leaky bucket algorithm is that the bucket "leaks water" (i.e., processes requests) at a fixed rate, so that the output flow remains constant regardless of fluctuations in input flow. If the bucket is full (i.e., the request queue overflows), the excess requests are discarded.

[0136] This embodiment optimizes leaky bucket performance through MQ message queues. An MQ message queue is introduced between the token bucket and the leaky bucket as a buffer layer, enhancing asynchronous processing capabilities and system fault tolerance. Requests first enter the MQ message queue, where consumers read and process requests at a fixed rate. Even if a consumer fails, other consumers can continue to process requests. Specifically, in S3, approved service requests are added to the MQ message queue. In S4, based on the leaky bucket algorithm, approved service requests are read and processed from the MQ message queue at a fixed rate.

[0137] Furthermore, the MQ message queue's backlog status is fed back to the token bucket, enabling it to better adjust the token issuance rate, forming a closed-loop mechanism. Service requests of different priorities are consumed by different consumer groups, ensuring the prioritization of core transaction requests. The MQ message queue not only enhances the system's fault tolerance but also provides the data foundation for the status of the token bucket and leaky bucket.

[0138] The token bucket algorithm combines the leaky bucket algorithm with the token bucket algorithm. The token bucket algorithm primarily limits the rate of incoming requests, allowing for a certain level of burst traffic. It's suitable for filtering out excessive requests at the network edge (such as at an API gateway) to prevent system resources from being exhausted. The leaky bucket algorithm primarily smooths outbound traffic, ensuring that downstream services aren't overloaded by upstream bursts. It "leaks" traffic outward at a fixed rate, ensuring that outbound traffic remains constant regardless of fluctuations in incoming traffic.

[0139] The leaky bucket's service request processing rate can also be adjusted synchronously with the token bucket's parameters based on the health score. For example, when the system is idle, the token bucket's parameters can be increased while also increasing the service request processing rate. This mechanism ensures that both can respond in a coordinated manner to changes in the payment system's load.

[0140] This method is particularly suitable for payment systems operating in high-concurrency scenarios. It incorporates dynamic adjustments to the token generation rate, a hierarchical token bucket design, and a leaky bucket smoothing mechanism to enhance the stability and user experience of the payment system. This method utilizes the monitoring tool Grafana to collect real-time status metrics for the payment system and calculates a scoring function based on these metrics to reflect the system's health. This health scoring function comprehensively considers the weighting of multiple factors and, after normalization, produces a unified health score that is used to determine whether the system is currently overloaded or idle.

[0141] This method dynamically adjusts the token generation rate and capacity of the token bucket based on the health score. When the health score falls below a preset low-load threshold, the system gradually reduces the token generation rate to reduce traffic entering the payment system. Conversely, when the health score rises above a high-load threshold, the token generation rate is increased to allow more service requests to enter the payment system. Furthermore, to meet the needs of service requests of varying priorities, this method proposes a hierarchical token bucket design. High-priority service requests are allocated a higher initial token quantity and generation rate to ensure sufficient resources even under high load. Low-priority service requests are assigned a lower priority and corresponding parameters.

[0142] To achieve refined rate limiting, this method also employs a hash table and queue data structure. The hash table is used to quickly find the token bucket instance corresponding to a user ID or IP address, while the queue stores the timestamps of each user's most recent N requests to facilitate determination of rate limiting compliance. This design not only improves query efficiency but also enables customized rate limiting policy configuration for specific users or IP addresses.

[0143] After using a token bucket for initial flow control, this method introduces a leaky bucket algorithm to smooth traffic flow and prevent downstream services from being overloaded by sudden traffic. The optimized leaky bucket mechanism, combined with MQ message queues, enables asynchronous request processing, enhancing the payment system's fault tolerance and processing efficiency.

[0144] This method incorporates a feedback loop mechanism, enabling adaptive adjustment of the throttling strategy. The payment system regularly evaluates the effectiveness of the adjusted throttling and further optimizes the throttling parameters (token generation rate adjustment step size and token bucket capacity adjustment step size) based on actual conditions. This continuous optimization capability not only improves system flexibility but also ensures that the throttling strategy remains optimal.

[0145] To verify the effectiveness of this method, a series of experiments were designed and implemented, including basic scenarios, high-concurrency scenarios, and abnormal traffic (such as simulated DDoS attacks) scenarios.

[0146] 1. Test environment configuration

[0147] Server configuration: High-configuration servers or server clusters to support high concurrent requests, including key hardware indicators such as CPU, memory, and network bandwidth.

[0148] Simulation tools: Use the JMeter performance testing tool to generate load and ensure that it can simulate a request volume of up to 1500 TPS.

[0149] 2. Test request volume

[0150] Core Transaction API: Target load of 1500 transactions per second (TPS).

[0151] Normal query API: Adjust the QPS based on actual conditions. Here, we assume 2000 QPS as a reference.

[0152] 3. Analysis of experimental results

[0153] 3.1 Basic Scenario (Targeting 1500 TPS)

[0154] In the basic scenario, the system mainly processes requests from the core transaction API and general query API.

[0155] Performance when multi-level current limiting method is not adopted:

[0156] TPS: When attempting to reach 1500 TPS, the system experienced significant delays and an increased failure rate. The actual stable TPS that could be handled was approximately 1000 times per second. Beyond this value, system response time increased dramatically, and the request failure rate reached 30%.

[0157] Resource utilization: The CPU is nearly fully loaded, and the memory usage exceeds 80%, affecting other services.

[0158] Performance after using this method

[0159] TPS: We successfully achieved and maintained our target of 1500 TPS, while maintaining reasonable system response times, with an average response time of less than 250 milliseconds. The request failure rate was less than 5%.

[0160] Resource utilization optimization: By dynamically adjusting token bucket and leaky bucket parameters, CPU utilization is maintained at around 75% and memory usage is controlled at 60%, effectively avoiding resource bottlenecks.

[0161] Standard Query API: While processing 1500 TPS of core transaction requests, the Standard Query API can still maintain approximately 1900 QPS, with a request failure rate of less than 5%.

[0162] 3.2 High Concurrency and Abnormal Traffic Scenarios

[0163] The system receives a sudden load of requests far exceeding the normal load in a short period of time, such as 3000 TPS, to test whether the system can cope with the impact through the burst processing capability of the token bucket.

[0164] Experimental results:

[0165] TPS: The system successfully processed some burst requests, but as the load continued to increase, it gradually entered a throttling state and eventually stabilized at around 1800 TPS.

[0166] Response time: The average response time is within 400 milliseconds, and the P99 response time does not exceed 800 milliseconds.

[0167] Request failure rate: About 8%, mainly because some requests are blocked by the rate limiting mechanism.

[0168] 3.3 Long-tail request impact scenarios

[0169] Simulate some requests so that the response time is significantly prolonged due to network delays or database locks, and observe the changes in the overall system throughput and whether the current limiting strategy effectively isolates abnormal requests.

[0170] Experimental results:

[0171] Long-tail request ratio: 10% of the total requests.

[0172] System performance: The overall TPS decreased slightly, but remained above 1,400 times / second. The P99 response time did not exceed 1 second, and abnormal requests were effectively isolated.

[0173] In summary, this method also performs well when facing higher concurrent requests or abnormal traffic (such as simulated DDoS attacks). It can protect the system from malicious traffic while ensuring that the core transaction API reaches 1500 TPS, significantly reducing request failure rate and optimizing resource allocation.

[0174] Through the detailed experimental design and results analysis above, we can see that applying this method not only enables the payment system to successfully handle load requirements of up to 1500 TPS, but also significantly improves system stability, resource utilization efficiency, and user experience. This demonstrates the effectiveness of this method in meeting the requirements of a high-performance payment system.

[0175] This embodiment also provides a current limiting system combining a token bucket and a leaky bucket for a payment system, such as Figure 2 As shown, the adopted technical solution is as follows: including: scoring module 1, token adjustment module 2, token allocation module 3, leaky bucket module 4 and step adjustment factor module 5.

[0176] The scoring module is used to obtain indicators of the payment system and calculate the health score;

[0177] The token adjustment module is used to determine the load status of the payment system according to the health score and adjust the token generation rate and token bucket capacity;

[0178] The token allocation module is used to receive a service request and determine whether to approve or reject the service request based on the number of available tokens;

[0179] The leaky bucket module is used to read and process the passed service requests according to the leaky bucket algorithm;

[0180] The step adjustment factor module is used to adjust the adjustment step of the token generation rate and the token bucket capacity according to the response of the payment system after multiple adjustments.

[0181] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A current limiting method combining a token bucket and a leaky bucket for a payment system, characterized in that: include: S1: Obtain payment system indicators and calculate health scores; S2: Determine the load status of the payment system based on the health score and adjust the token generation rate and token bucket capacity. The adjustment step size of the token generation rate and token bucket capacity is adjusted based on the response of the payment system after multiple adjustments. When the health score is less than the low load threshold, the token generation rate and token bucket capacity are reduced; When the health score is greater than the high load threshold, increase the token generation rate and token bucket capacity; When the health score is less than or equal to the high load threshold and greater than or equal to the low load threshold, the token generation rate and token bucket capacity remain unchanged; Different token buckets are set for service requests of different priorities. When the health score is less than the low load threshold, the token generation rate and token bucket capacity of high-priority service requests are increased, and the token generation rate and token bucket capacity of low-priority service requests are reduced; When the health score is greater than the high load threshold, the token generation rate and token bucket capacity of high-priority service requests are reduced, and the token generation rate and token bucket capacity of low-priority service requests are increased; If the response of the payment system after multiple adjustments satisfies conditions (1) to (4) at the same time, the token generation rate adjustment step and the token bucket capacity adjustment step are increased; Case (1): Health score improved, Case (2): Request failure rate decreases, Case (3): Response time is shortened, Case (4): Throughput increases; If the response of the payment system after multiple adjustments falls into any of the situations (5) to (8), the token generation rate adjustment step and the token bucket capacity adjustment step are reduced; Case (5): Unstable health score, Case (6): The request failure rate has no significant change or worsens. Case (7): Response time is extended, Case (8): throughput decreases; S3: Receive the service request and decide whether to approve or reject the service request based on the number of available tokens; S4: Read and process the passed service requests according to the leaky bucket algorithm.

2. A current limiting method combining a token bucket and a leaky bucket for a payment system as claimed in claim 1, characterized in that: The payment system's indicators include: CPU utilization, memory usage, network bandwidth, real-time traffic, response time, and error rate; each indicator is calculated through weighted summation to obtain a health score.

3. The current limiting method combining a token bucket and a leaky bucket for a payment system according to claim 1, characterized in that: The function of health score is: Health score = w1 (1-normalized CPU utilization) + w2 (1-normalized memory usage) + w3 (normalized network bandwidth) + w4 (1-normalized real-time traffic) + w5 (1-normalized response time) + w6 (1-normalized error rate) Among them, w1 is the first weight coefficient, w2 is the second weight coefficient, w3 is the third weight coefficient, w4 is the fourth weight coefficient, w5 is the fifth weight coefficient, and w6 is the sixth weight coefficient.

4. A current limiting method combining a token bucket and a leaky bucket for a payment system as claimed in claim 1, characterized in that: The specific process of S3 is: S31: Receive a service request and search the corresponding token bucket from the hash table based on the user ID or IP address; S32: Update the status of the token bucket; S33: Check whether the service request complies with the flow limiting rules. If not, reject the service request. If it complies with the flow limiting rules, execute S34. S34: Determine whether there are available tokens in the token bucket. If there are enough tokens, allow the request to pass and deduct the corresponding tokens. If there are not enough tokens, the service request will be rejected.

5. A current limiting method combining a token bucket and a leaky bucket for a payment system as claimed in claim 4, characterized in that: The flow limiting rules include exceeding the limit of service requests per unit time. The timestamps of the latest N service requests stored in the queue are used to determine whether the service requests of the user ID or IP address meet the flow limiting rules. In S34, if the request is allowed to pass, the service request timestamp is added to the queue.

6. A current limiting method combining a token bucket and a leaky bucket for a payment system as claimed in claim 1, characterized in that: In S3, the approved service requests are added to the MQ message queue; in S4, according to the leaky bucket algorithm, the approved service requests are read and processed from the MQ message queue at a fixed rate.

7. A current limiting system combining a token bucket and a leaky bucket for a payment system, characterized in that: The method is used to implement a current limiting method combining a token bucket and a leaky bucket for a payment system according to any one of claims 1 to 6, comprising: a scoring module, a token adjustment module, a token allocation module, a leaky bucket module, and a step adjustment factor module. The scoring module is used to obtain indicators of the payment system and calculate the health score; The token adjustment module is used to determine the load status of the payment system according to the health score and adjust the token generation rate and token bucket capacity; The token allocation module is used to receive a service request and determine whether to approve or reject the service request based on the number of available tokens; The leaky bucket module is used to read and process the passed service requests according to the leaky bucket algorithm; The step adjustment factor module is used to adjust the adjustment step of the token generation rate and the token bucket capacity according to the response of the payment system after multiple adjustments.

Images