Token bucket and leaky bucket combined flow limiting method and system for payment system

Abstract

The invention relates to the technical field of payment system flow control, and particularly discloses a token bucket and leaky bucket combined flow limiting method and system for a payment system, and the method comprises the steps: S1, obtaining an index of the payment system, and calculating a health score; s2, judging the load state of the payment system according to the health score, and adjusting the token generation rate and the token bucket capacity; the token generation rate and the adjusting step length of the token bucket capacity are adjusted according to the response condition of the payment system after multiple times of adjustment; s3, receiving a service request, and judging whether the service request passes or is rejected according to the number of the available tokens; and S4, reading and processing the passed service request according to a leaky bucket algorithm. By dynamically adjusting the token generation speed, the hierarchical token bucket design and the leaky bucket smooth output mechanism, the stability of the system and the user experience are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of payment system traffic control, and particularly to a flow limiting method and system combining a token bucket and a leaky bucket for a payment system. Background Art

[0002] At present, the payment system lacks an effective flow limiting mechanism, which makes it show obvious vulnerability when facing high-concurrency requests. Different from general Web applications, the payment system has extremely high requirements for security and real-time performance. Each payment operation needs to go through a strict verification process, and the consistency and integrity of data must be ensured. Therefore, any flow limiting scheme must take these special requirements into account. An ideal flow limiting mechanism should not only ensure the transaction success rate, that is, complete as many effective transactions as possible during peak hours, but also have the ability to identify abnormal behaviors, such as detecting and preventing suspicious repeated payment attempts. In addition, the flow limiting mechanism also needs to have high scalability and flexibility to support different levels of service division.

[0003] In the current technical environment, although there are various flow limiting products and solutions, there are still many limitations and deficiencies when dealing with complex application scenarios, especially payment systems used in multiple countries.

[0004] (1) Fixed-rate flow limiting is a simple and direct flow limiting method that restricts traffic by setting a fixed upper limit on the number of requests. During peak periods, it may occur that even when the set maximum request volume is reached, it is still insufficient to meet the actual demand. This rigid flow limiting method not only wastes resources but also may cause users to encounter frequent request rejections or timeouts in high-concurrency scenarios, seriously affecting the user experience.

[0005] (2) The token bucket and leaky bucket algorithms are two common flow limiting mechanisms, which are respectively used to control the inflow and outflow traffic speeds. To a certain extent, these two algorithms relieve the system pressure, but when used alone, it is difficult for them to comprehensively cover various complex scenarios in the payment system.

[0006] Chinese Patent Application Publication No. CN119520187A discloses a unified access management method for computing power based on API technology, which uses an adaptive flow limiting method to dynamically adjust the token generation rate and the leak rate of the leaky bucket according to the real-time traffic changes of the interface, and performs fine-grained control and smooth processing on the traffic. Although this invention combines the token bucket and leaky bucket algorithms, it does not consider the differences in requests at different levels in the payment system.

[0007] Different from general flow-limiting scenarios, in a payment system, certain core transaction APIs, such as payment confirmation and refund processing, have extremely high requirements for the system's response time, while ordinary query APIs, such as order query and account balance query, have relatively loose requirements. If all requests are treated equally, key services may not be processed in a timely manner because resources are occupied by low-priority requests, thereby affecting the stability of the entire system and the user experience. Summary of the Invention

[0008] The present invention aims to solve the above problems. To this end, the present invention provides a flow-limiting method and system combining a token bucket and a leaky bucket for a payment system, which improves the stability of the system and the user experience by dynamically adjusting the token generation speed, hierarchical token bucket design, and leaky bucket smooth output mechanism.

[0009] The present invention provides a flow-limiting method combining a token bucket and a leaky bucket for a payment system, and the technical solutions adopted are as follows: including: S1: Obtain the metrics of the payment system and calculate the health score; S2: Determine the load status of the payment system according to the health score, and adjust the token generation rate and the token bucket capacity; the adjustment step sizes of the token generation rate and the token bucket capacity are adjusted according to the response situation of the payment system after multiple adjustments; When the health score is less than the low load threshold, reduce the token generation rate and the token bucket capacity; When the health score is greater than the high load threshold, increase the token generation rate and the token bucket capacity; When the health score is less than or equal to the high load threshold and greater than or equal to the low load threshold, keep the token generation rate and the token bucket capacity unchanged; S3: Receive a service request, and determine whether to pass or reject the service request according to the number of available tokens; S4: According to the leaky bucket algorithm, read and process the passed service requests.

[0010] Furthermore, the metrics of the payment system include: CPU utilization rate, memory occupancy rate, network bandwidth, real-time traffic, response time, and error rate; the health score is calculated by weighted summation of each metric.

[0011] Furthermore, the function of the health score is: Health score = w1 (1 - normalized CPU utilization rate) + w2 (1 - normalized memory occupancy rate) + w3 (normalized network bandwidth) + w4 (1 - normalized real-time traffic) + w5 (1 - normalized response time) + w6 (1-Error rate after normalization) Among them, w1 is the first weight coefficient, w2 is the second weight coefficient, w3 is the third weight coefficient, w4 is the fourth weight coefficient, w5 is the fifth weight coefficient, and w6 is the sixth weight coefficient.

[0012] Furthermore, different priority service requests are set with different token buckets. When the health score is less than the low load threshold, increase the token generation rate and token bucket capacity of high-priority service requests, and decrease the token generation rate and token bucket capacity of low-priority service requests; When the health score is greater than the high load threshold, decrease the token generation rate and token bucket capacity of high-priority service requests, and increase the token generation rate and token bucket capacity of low-priority service requests.

[0013] Furthermore, the specific process of S3 is as follows: S31: Receive a service request, and look up the corresponding token bucket in the hash table according to the user ID or IP address; S32: Update the status of the token bucket; S33: Check whether the service request complies with the traffic limiting rule. If it does not comply with the traffic limiting rule, reject this service request; if it complies with the traffic limiting rule, execute S34; S34: Determine whether there are available tokens in the token bucket. If there are enough tokens, allow the request to pass and deduct the corresponding tokens; if there are not enough tokens, reject the service request.

[0014] Furthermore, the traffic limiting rule includes that the number of service requests per unit time exceeds the limit, and it is judged whether the service requests of this user ID or IP address comply with the traffic limiting rule by the time stamps of the nearest N service requests stored in the queue; In S34, if the request is allowed to pass, add the service request time stamp to the queue.

[0015] Furthermore, in S3, add the passed service requests to the MQ message queue; in S4, according to the leaky bucket algorithm, read and process the passed service requests from the MQ message queue at a fixed rate.

[0016] Furthermore, if the response situation of the payment system after multiple adjustments simultaneously meets conditions (1) to (4), increase the token generation rate adjustment step and the token bucket capacity adjustment step; Condition (1): The health score improves, Condition (2): The request failure rate decreases, Condition (3): The response time is shortened, Condition (4): The throughput increases; If any of the situations from (5) to (8) occurs in the response of the payment system after multiple adjustments, the adjustment step size of the token generation rate and the adjustment step size of the token bucket capacity are reduced. Situation (5): The health score is unstable. Situation (6): The request failure rate has no obvious change or deteriorates. Situation (7): The response time is extended. Situation (8): The throughput decreases.

[0017] The present invention also provides a flow limiting system combining a token bucket and a leaky bucket for a payment system, and the technical solution adopted is as follows: including a scoring module, a token adjustment module, a token allocation module, a leaky bucket module, and a step size adjustment factor module. The scoring module is used to obtain the metrics of the payment system and calculate the health score. The token adjustment module is used to determine the load status of the payment system according to the health score and adjust the token generation rate and the token bucket capacity. The token allocation module is used to receive service requests and determine whether to pass or reject the service requests according to the number of available tokens. The leaky bucket module is used to read and process the passed service requests according to the leaky bucket algorithm. The step size adjustment factor module is used to adjust the adjustment step sizes of the token generation rate and the token bucket capacity according to the response situation of the payment system after multiple adjustments.

[0018] One or more of the above technical solutions in the embodiments of the present invention have at least one of the following technical effects: 1. The present invention improves the stability of the payment system.

[0019] By introducing a dynamic adjustment of the token generation speed, a hierarchical token bucket design, and a leaky bucket smooth output mechanism, the present invention significantly improves the stability of the payment system. First, the monitoring tool Grafana is used to collect system status metrics (such as CPU utilization, memory occupancy, and network bandwidth) in real time, and the current load situation of the system is comprehensively evaluated through a health scoring function. This intelligent dynamic adjustment mechanism enables the system to flexibly adjust the flow limiting strategy according to real-time monitoring data, avoiding resource waste or system overload problems caused by static flow limiting rules. For example, during a promotional event, when a large number of users initiate service requests simultaneously, the system can reduce the token generation rate in a timely manner according to the health score to prevent too many requests from flooding into the system, thus effectively avoiding system crashes.

[0020] In addition, the hierarchical token bucket design further enhances the stability of the system. By setting up independent token bucket instances for service requests with different priorities and allocating different initial token quantities and generation rates, it ensures that critical services can still obtain sufficient resource guarantees under high-load conditions. This hierarchical design not only improves the availability of critical services but also achieves optimal resource allocation.

[0021] 2. The present invention helps to improve the user experience.

[0022] The present invention significantly improves the user experience through refined flow limiting control and traffic smoothing processing. First, a data structure combining a hash table and a queue is adopted to implement a personalized flow limiting strategy at the user level. The hash table is used to quickly find the token bucket instance corresponding to the user ID or IP address, while the queue stores the time stamps of the most recent N requests of each user to facilitate determining whether the flow limiting rules are met. This design not only improves the search efficiency but also enables personalized flow limiting strategy configuration for specific users or IPs, ensuring that the normal requests of legitimate users are processed in a timely manner and reducing the probability of users encountering request rejections or timeouts.

[0023] After initially limiting the flow using a token bucket at the access layer, a leaky bucket algorithm is introduced to smooth the traffic and prevent downstream services from being overloaded due to sudden traffic surges. The optimized leaky bucket mechanism combines with an MQ message queue to implement the function of asynchronous request processing, reducing the synchronous waiting time. This not only improves the system's response speed but also enhances the system's fault tolerance. Even if a certain consumer fails, other consumers can still continue to process requests, ensuring the continuity and reliability of the system, thus significantly improving the user experience.

[0024] 3. The present invention improves the scalability and flexibility of the payment system.

[0025] The design of the present invention fully considers the scalability and flexibility of the system, enabling the payment system to easily cope with changing business requirements and technical challenges. First, the hierarchical token bucket design allows for dynamic adjustment of flow limiting parameters according to different service interface types and user groups, ensuring that the system can operate efficiently in high-concurrency scenarios. Second, the introduction of a feedback closed-loop mechanism endows the flow limiting strategy with self-adaptive adjustment capabilities. The system regularly evaluates the adjusted flow limiting effect and further optimizes the flow limiting parameters according to the actual situation. This continuous optimization ability not only improves the system's flexibility but also ensures that the flow limiting strategy is always in the best state. In addition, the leaky bucket mechanism combined with an MQ message queue realizes asynchronous request processing, further enhancing the system's scalability and fault tolerance. Even in the face of a large number of concurrent requests, the system can still operate efficiently and stably, providing strong technical support for enterprises.

[0026] 4. The present invention reduces the operation and maintenance costs and complexity of the payment system.

[0027] Through the automated and intelligent design, the present invention significantly reduces the operation and maintenance costs and complexity of the payment system. First, the introduction of the dynamic adjustment mechanism and the feedback closed-loop mechanism reduces the need for manual intervention. The system can automatically adjust the current limiting strategy according to real-time monitoring data without the need for operation and maintenance personnel to manually modify the configuration file or restart the service, greatly reducing the operation and maintenance burden. Second, the hierarchical token bucket design and the data structure based on hash tables and queues simplify the implementation process of current limiting control. Operation and maintenance personnel only need to focus on how to reasonably set the priorities and initial parameters of each service interface, and the system will automatically complete the subsequent current limiting operations. In addition, the leaky bucket mechanism combined with the MQ message queue not only improves the processing capacity of the system but also enhances the fault tolerance of the system, reducing the system downtime caused by single-point failures. These improvement measures work together to significantly reduce the operation and maintenance costs and complexity, bringing considerable economic benefits.

[0028] Additional aspects and advantages of the present invention will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0030] Figure 1 is the flowchart of the method provided by the present invention.

[0031] Figure 2 is the block diagram of the system structure provided by the present invention.

[0032] Reference Signs: 1, scoring module; 2, token adjustment module; 3, token distribution module; 4, leaky bucket module; 5, step size adjustment factor module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0033] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention with reference to the drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention. The following embodiments are used to illustrate the present invention but cannot be used to limit the scope of the present invention.

[0034] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the embodiments of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0035] The following combines Figure 1 and Figure 2 to further elaborate on the present invention, and describes a rate limiting method and system combining a token bucket and a leaky bucket for a payment system of the present invention: In this embodiment, as Figure 1 shown, a rate limiting method combining a token bucket and a leaky bucket for a payment system is provided, including the following steps: S1: Obtain the metrics of the payment system and calculate the health score.

[0036] The metrics of the payment system include: CPU utilization rate, memory occupancy rate, network bandwidth, real-time traffic, response time, and error rate. The health score is calculated by weighted summation of each metric.

[0037] The metrics (CPU utilization rate, memory occupancy rate, network bandwidth, real-time traffic, response time, error rate) selected when defining the health score function are all key factors directly affecting the system performance and stability. They jointly determine the current load condition and service quality of the system. The CPU utilization rate and memory occupancy rate directly affect the server's ability to process requests. Excessive utilization rates may cause the system response to slow down or even crash. The network bandwidth reflects the bottleneck situation of data transmission, especially important in high-concurrency scenarios. The real-time traffic provides information on the number of requests currently processed by the system, helping to evaluate the busyness of the system. The response time is a direct indicator of user experience. A shorter response time means better user satisfaction. The error rate refers to the proportion of errors occurring in the system during the process of processing requests. A higher error rate usually indicates that there are problems in the system.

[0038] In this embodiment, the monitoring tool Grafana is used to collect metrics such as the CPU utilization rate, memory occupancy rate, and network bandwidth of the payment system.

[0039] The goal of the health score is to comprehensively reflect the current state of the system (such as CPU utilization, memory occupancy, network latency, etc.), and determine whether the system is in a healthy state based on these metrics. Different metrics may have different degrees of impact on system health, so it is necessary to assign weights to each metric. For example: if the system is sensitive to the CPU, a higher weight can be given to CPU utilization; if network latency has a greater impact on the user experience, a higher weight can be given to network latency. The weight assignment can be obtained based on experience or historical data analysis.

[0040] Since the value ranges of different metrics are different, it is necessary to normalize all metrics to unify them into the same range, such as 0 to 1. In this embodiment, the maximum-minimum normalization method is used for processing. For example: the CPU utilization is normalized from 0% to 100% to 0 to 1, and the network latency is normalized from 0 ms to 500 ms to 0 to 1.

[0041] Combining the normalized metric values with the weights, the final health score function is designed as follows: Health score = w1 (1 - normalized CPU utilization) + w2 (1 - normalized memory occupancy rate) + w3 (normalized network bandwidth) + w4 (1 - normalized real-time traffic) + w5 (1 - normalized response time) + w6 (1 - normalized error rate) where w1 is the first weight coefficient, w2 is the second weight coefficient, w3 is the third weight coefficient, w4 is the fourth weight coefficient, w5 is the fifth weight coefficient, w6 is the sixth weight coefficient, and they satisfy .

[0042] S2: Determine the load status of the payment system based on the health score, and adjust the token generation rate and token bucket capacity; the adjustment step sizes of the token generation rate and token bucket capacity are adjusted according to the response of the payment system after multiple adjustments.

[0043] For the convenience of subsequent flow limiting decisions, two thresholds need to be set, namely the low load threshold and the high load threshold. By comparing the health score with the two thresholds, the load status of the payment system is determined, and then how to adjust the token generation rate and token bucket capacity is determined. Specifically: When the health score is less than the low load threshold, it indicates that the system is overloaded, and the token generation rate and token bucket capacity need to be reduced. That is, when, , .

[0044] When the health score is greater than the high load threshold, it indicates that the system is idle, and the token generation rate and token bucket capacity can be appropriately increased. That is When , .

[0045] When the health score is less than or equal to the high load threshold and greater than or equal to the low load threshold, keep the token generation rate and token bucket capacity unchanged.

[0046] Among them, is the health score, is the low load threshold, is the high load threshold, is the token generation rate at the current moment, is the token generation rate at the next moment, is the token generation rate adjustment step size, is the token bucket capacity at the current moment, is the token bucket capacity at the next moment, is the token bucket capacity adjustment step size. And The selection of is crucial and should not be too large to avoid causing system oscillations. Appropriate step size values should be selected according to the actual situation.

[0047] Based on experience and historical data analysis, in this embodiment, the low load threshold is set to 0.3 and the high load threshold is set to 0.7.

[0048] In addition, the upper and lower limits of the token generation rate and token bucket capacity also need to be set, and the adjusted token generation rate and token bucket capacity should not exceed the corresponding upper and lower limits.

[0049] According to the characteristics of the payment system, this embodiment also sets different independent token buckets for service interfaces or user groups with different priorities to ensure that critical services can still obtain sufficient resources under high load. By creating independent token bucket instances for each service level or user group and dynamically adjusting their parameters (token generation rate and token bucket capacity) according to the load status of the payment system, the stability of the system and the user experience can be effectively guaranteed.

[0050] Specifically, for requests corresponding to service interfaces, different priorities are assigned according to their types. For example: the priorities of payment confirmation and refund processing are set to high, and the priorities of order query and account balance query are set to low. For user groups, different priorities are assigned according to their importance. For example: the priority of VIP customers is set to high, and the priority of general customers is set to low. Of course, according to the actual situation, the number of priority levels is not limited to 2.

[0051] Token bucket status: Among them, is the number of tokens at the current moment, is the capacity of the token bucket, is the token generation rate, that is, the number of tokens generated per second, , is the time interval between two moments, is the number of tokens at the previous moment, is to take the minimum value.

[0052] The calculation formula for the capacity of the token bucket: Among them, is the maximum number of requests allowed per second, is the time interval for replenishing tokens, usually 1 second. Tokens are generated at the token generation rate, and the total number of tokens does not exceed the capacity of the token bucket.

[0053] When adjusting the token generation rate and the capacity of the token bucket, it is necessary to dynamically adjust the capacity of the token bucket and the token generation rate at each level according to the priority to ensure that key services obtain resources first. Adjustment rules based on different request priorities: When the health score is less than the low load threshold and the system is overloaded, increase the token generation rate and the capacity of the token bucket for high-priority service requests, and decrease the token generation rate and the capacity of the token bucket for low-priority service requests. That is, significantly reduce the token generation rate and the capacity of the token bucket for low-priority service requests, and slightly increase the token generation rate and the capacity of the token bucket for high-priority service requests to ensure that high-priority service requests are processed smoothly.

[0054] When the health score is greater than the high load threshold and the system is idle, reduce the token generation rate and the capacity of the token bucket for high-priority service requests, and increase the token generation rate and the capacity of the token bucket for low-priority service requests. It should be noted that by reasonably setting the initial values of the token generation rate and the capacity of the token bucket for high-priority service requests, it is possible to prevent the system from being idle and the processing capacity of priority service requests from being insufficient. These initial values are set and adjusted according to the actual usage situation.

[0055] When the health score is less than or equal to the high load threshold and greater than or equal to the low load threshold, keep the token generation rate and the capacity of the token bucket unchanged, that is, do not change the token generation rate and the capacity of the token bucket for high-priority service requests and low-priority service requests.

[0056] Taking 2 priorities as an example, the calculation formula for the token generation rate is: When the system is overloaded: Among them, is the token generation rate at the next moment for high-priority service requests, is the token generation rate at the current moment for high-priority service requests, is the adjustment step size of the token generation rate for high-priority service requests, is the token generation rate at the next moment for low-priority service requests, is the token generation rate at the current moment for low-priority service requests, is for low-priority service requests, .

[0057] When the system is idle: .

[0058] The adjustment method of the token bucket capacity is similar to that of the token generation rate.

[0059] This embodiment introduces a feedback mechanism to evaluate the throttling effect after adjustment. According to the response of the payment system after multiple adjustments, the adjustment step sizes of the token generation rate and the token bucket capacity are adjusted.

[0060] This embodiment is to ensure that the system's throttling policy can be dynamically adjusted according to the actual operating conditions, so as to better cope with system load changes. If the system is still overloaded after adjustment, the throttling rules are further tightened; otherwise, they are appropriately relaxed. Specifically, the feedback closed-loop mechanism is a key component to ensure that the system's throttling policy can be dynamically adjusted according to the actual operating conditions. First, the system regularly collects the latest health scores and other key metric data from the monitoring layer (such as Grafana), and evaluates the effect of the current throttling policy based on these data. If the evaluation result shows that the system is still overloaded (the health score is lower than the low-load threshold), the throttling rules are further tightened. The choice of the adjustment step size is crucial and should not be too large to avoid causing system oscillations. An appropriate step size value should be selected according to the actual situation. Subsequently, the adjusted throttling policy is applied to the system in real time, and a new round of monitoring and evaluation is started to verify the effectiveness of the adjustment measures. This process balances the response speed and system overhead by setting a reasonable evaluation time interval to ensure that the throttling policy can be continuously optimized, forming a complete feedback closed-loop. Thus, the payment system can effectively protect itself from overload when facing sudden traffic, and at the same time make full use of these resources when resources are abundant to provide a better user experience. This adaptive adjustment ability not only improves the system's stability but also enhances its flexibility and fault tolerance.

[0061] In the face of high-concurrency scenarios or abnormal situations, the system needs to dynamically adjust the token bucket capacity and token generation rate at all levels to ensure that critical services can obtain resources preferentially. To improve the system's adaptability, this embodiment also introduces a Step Adjustment Factor (SAF), enabling the step size of each adjustment (the step size of token generation rate adjustment and the step size of token bucket capacity adjustment) to vary dynamically according to the system's response.

[0062] Design and implementation of SAF: Initial setting: First, set an initial value for the adjustment step size. This value can be determined based on historical data or empirical rules.

[0063] Step Adjustment Factor: Introduce the Step Adjustment Factor (SAF), which is used to adjust the amplitude of each adjustment. SAF is dynamically adjusted according to the historical response of the payment system. In this embodiment, if it is found that the current adjustment has a significant effect on improving the system load, SAF can be appropriately increased. For example, the step size of token generation rate adjustment and the step size of token bucket capacity adjustment can be increased to 1.1 times the current value; conversely, if the system performance fluctuates greatly or shows no obvious improvement after adjustment, SAF is decreased, such as reduced to 0.9 times the current value. Otherwise, keep the current step size of token generation rate adjustment and the step size of token bucket capacity adjustment unchanged.

[0064] Situations where the system load improvement is significant are as follows: Improved health score: If the health score of the adjusted payment system has increased significantly, it indicates that the current resource usage of the system is more balanced and the load has been effectively alleviated. For example, key indicators such as CPU utilization rate and memory occupancy rate have dropped to a more reasonable level.

[0065] Reduced request failure rate: As the system load decreases, the ability to process requests increases, resulting in a significant drop in the request failure rate. For example, before adjustment, due to overload, 30% of requests may fail, while after adjustment, this proportion may drop to within 5%.

[0066] Shortened response time: As a direct indicator of user experience, the shortening of the system response time means that the service quality felt by users has been improved. For example, the average response time has been reduced from more than 1 second before to within 250 milliseconds.

[0067] Increased throughput: While maintaining or improving service quality, the system can handle more transactions or query requests (TPS / QPS). For example, the core transaction API can reach and maintain the target of 1500 TPS under stable conditions.

[0068] If the response of the payment system after multiple adjustments meets the above conditions simultaneously, increase the adjustment step size of the token generation rate and the adjustment step size of the token bucket capacity.

[0069] The large fluctuations or lack of obvious improvement in the system performance after adjustment refer to the following situations: Unstable health score: Even if the throttling parameters are adjusted, if the system health score still fluctuates within a large range and does not tend to an ideal value, it indicates that the adjustment measures have not achieved the expected effect. For example, if the health score drops suddenly from 0.8 to 0.3 and then rises back to 0.7 in a short period, it shows that the system state is extremely unstable.

[0070] No obvious change or deterioration in the request failure rate: If the request failure rate remains high or even higher after adjustment, it indicates that the current throttling strategy is not suitable for the current traffic pattern or system state. For example, the original 10% request failure rate rises to 15% after adjustment.

[0071] Prolonged response time: After the system is adjusted, if the average response time of users becomes longer instead of shorter, this is also one of the manifestations that the performance has not been improved. For example, the original average response time is 300 milliseconds, and it becomes more than 500 milliseconds after adjustment.

[0072] Decrease in throughput: Despite the implementation of throttling measures, the overall throughput of the system has declined and cannot meet the business requirements. For example, the core transaction API that could originally handle 1500 TPS stably can only maintain around 1200 TPS after adjustment.

[0073] If any of the above situations occur in the response of the payment system after multiple adjustments, reduce the adjustment step size of the token generation rate and the adjustment step size of the token bucket capacity.

[0074] S3: Receive a service request and determine whether to pass or reject the service request based on the number of available tokens.

[0075] This embodiment uses the data structures of a hash table and a queue. The hash table is used to quickly find the token bucket instance corresponding to the user ID or IP address, while the queue stores the time stamps of the most recent N service requests of each user to facilitate determining whether the throttling rules are met. Specifically, the hash table: used to quickly find the token bucket instance corresponding to the user ID or IP address. Key: user ID or IP address. Value: corresponding token bucket object. Queue: stores the time stamps of the most recent N service requests of the user ID or IP address.

[0076] The specific process is as follows: S31: Receive a service request. When a service request arrives, find the corresponding token bucket from the hash table according to the user ID or IP address.

[0077] S32: Update the status of the token bucket. Calculate the number of newly added tokens based on the last update time.

[0078] S33: Check whether the service request conforms to the rate limiting rule. If it does not conform to the rate limiting rule, reject this service request; if it conforms to the rate limiting rule, execute S34.

[0079] In this embodiment, the rate limiting rule includes that the number of service requests within a unit time exceeds the limit. The request frequency of the user ID or IP address is judged whether it conforms to the rate limiting rule by the time stamps of the last N service requests stored in the queue. For example, a maximum of 5 service requests are allowed within 1 second. For example, according to the request time stamps in the queue, if it is determined that this service request is the 6th service request within 1 second, it is considered that this service request does not conform to the rate limiting rule. According to the actual situation, other rate limiting rules can also be set.

[0080] S34: Judge whether there are available tokens in the token bucket. If there are enough tokens, allow the service request to pass, deduct the corresponding tokens, and add the service request time stamp to the queue; if there are not enough tokens, reject the service request.

[0081] It should be noted that in this step, service requests with different priorities correspond to different token buckets.

[0082] S4: According to the leaky bucket algorithm, read and process the passed service requests.

[0083] The basic principle of the leaky bucket algorithm: The leaky bucket "leaks water" (i.e., processes requests) at a fixed rate. Regardless of how the input traffic fluctuates, the output traffic always remains constant. If the bucket is full (i.e., the request queue overflows), the excess requests are discarded.

[0084] In this embodiment, the performance of the leaky bucket is optimized through the MQ message queue. An MQ message queue is introduced between the token bucket and the leaky bucket as a buffer layer to enhance the asynchronous processing ability and system fault tolerance. The requests first enter the MQ message queue, and the consumers read and process the requests from the queue at a fixed rate. Even if a certain consumer fails, other consumers can still continue to process the requests. That is, in S3, the passed service requests are added to the MQ message queue; in S4, according to the leaky bucket algorithm, the passed service requests are read and processed from the MQ message queue at a fixed rate.

[0085] In addition, the backlog status of the MQ message queue can be fed back to the token bucket to make it better adjust the token issuance speed and form a closed loop. Service requests with different priorities are consumed by different consumer groups to ensure the priority of core transaction requests. The MQ message queue not only enhances the fault tolerance of the system, but also provides a data basis for the state perception of the token bucket and the leaky bucket.

[0086] The token bucket algorithm is combined with the leaky bucket algorithm. The token bucket algorithm is mainly used to limit the request rate entering the system and allows a certain degree of burst traffic. It is suitable for filtering excessive requests at the network edge (such as API gateways) to prevent the system resources from being exhausted instantly. The leaky bucket algorithm is mainly used to smooth the output traffic and ensure that downstream services will not be overloaded due to burst traffic from upstream. It "leaks water" at a fixed rate, and regardless of how the input traffic fluctuates, the output traffic always remains constant.

[0087] The rate at which the leaky bucket processes service requests can also be synchronously adjusted with the parameters of the token bucket according to the health score. For example: when the system is idle, while increasing the parameters of the token bucket, increase the processing rate of service requests. This mechanism can ensure that both can respond collaboratively when the load of the payment system changes.

[0088] This method is particularly suitable for payment systems in high-concurrency scenarios. This method designs a dynamic adjustment of the token generation speed, a hierarchical token bucket design, and a leaky bucket smooth output mechanism, aiming to improve the stability and user experience of the payment system. This method uses the monitoring tool Grafana to collect real-time state metrics of the payment system and calculates a scoring function reflecting the system health status based on the metric data. This health scoring function comprehensively considers the influence weights of multiple factors and obtains a unified health score after normalization processing, which is used to determine whether the system is currently overloaded or idle.

[0089] According to the health score, this method can dynamically adjust the token generation rate and token bucket capacity in the token bucket. When the health score is lower than the preset low-load threshold, the system will gradually reduce the token generation rate to reduce the traffic entering the payment system; on the contrary, when the health score is higher than the high-load threshold, increase the token generation rate to allow more service requests to enter the payment system. In addition, to meet the needs of service requests with different priorities, this method proposes a design scheme for hierarchical token buckets. For high-priority service requests, allocate a higher initial token quantity and generation rate to ensure that they can still obtain sufficient resources under high-load conditions. For low-priority service requests, set lower priorities and corresponding parameters.

[0090] To achieve refined rate limiting control, this method also adopts the data structures of a hash table and a queue. The hash table is used to quickly find the token bucket instance corresponding to the user ID or IP address, while the queue stores the most recent N request timestamps of each user to facilitate judging whether the rate limiting rules are met. This design not only improves the search efficiency but also enables personalized rate limiting strategy configuration for specific users or IPs.

[0091] After the initial traffic limiting using the token bucket, this method introduces the leaky bucket algorithm to smooth the traffic and prevent downstream services from being overloaded due to sudden traffic surges. The optimized leaky bucket mechanism combines with the MQ message queue to achieve the function of asynchronous request processing, enhancing the fault tolerance and processing efficiency of the payment system.

[0092] This method introduces a feedback closed-loop mechanism, enabling the traffic limiting strategy to have an adaptive adjustment ability. The payment system regularly evaluates the adjusted traffic limiting effect and further optimizes the traffic limiting parameters (the adjustment step size of the token generation rate and the adjustment step size of the token bucket capacity) according to the actual situation. This continuous optimization ability not only improves the flexibility of the system but also ensures that the traffic limiting strategy is always in the best state.

[0093] To verify the effectiveness of this method, a series of experiments were designed and implemented, including basic scenarios, high-concurrency scenarios, and abnormal traffic (such as simulating DDoS attacks) scenarios.

[0094] 1. Test environment configuration Server configuration: High-configured servers or server clusters are used to support high-concurrency requests, specifically including key hardware metrics such as CPU, memory, and network bandwidth.

[0095] Simulation tool: The JMeter performance testing tool is used to generate the load to ensure that it can simulate a request volume of up to 1500 TPS.

[0096] 2. Test request volume Core transaction API: A target load of 1500 transactions per second (TPS).

[0097] Ordinary query API: Adjust the QPS according to the actual situation. Here, it is assumed to be 2000 QPS as a reference.

[0098] 3. Analysis of experimental results 3.1 Basic scenario (for the 1500 TPS target) In the basic scenario, the system mainly processes requests for the core transaction API and the ordinary query API.

[0099] Performance without using the multi-level traffic limiting method: TPS: When attempting to reach 1500 TPS, the system experiences severe delays and an increase in the failure rate. The actual TPS that can be stably processed is approximately 1000 times per second. Beyond this value, the system response time increases sharply, and the request failure rate reaches 30%.

[0100] Resource utilization: The CPU is close to full load, and the memory occupancy rate exceeds 80%, affecting other services.

[0101] Performance after adopting this method TPS: The target of 1500 TPS was successfully achieved and maintained, and the system response time remained within a reasonable range, with an average response time of less than 250 milliseconds. The request failure rate was below 5%.

[0102] Resource utilization optimization: By dynamically adjusting the token bucket and leaky bucket parameters, the CPU utilization was maintained at around 75%, and the memory occupancy was controlled at 60%, effectively avoiding resource bottlenecks.

[0103] General query API: Under the core transaction requests of handling 1500 TPS simultaneously, the general query API could still maintain approximately 1900 QPS, and the request failure rate was below 5%.

[0104] 3.2 High-concurrency and abnormal traffic scenarios Suddenly receive requests far exceeding the normal load within a short period of time, such as 3000 TPS per second, and test whether the system can handle the impact through the burst processing ability of the token bucket.

[0105] Experimental results: TPS: The system successfully processed some burst requests, but as the load continued to increase, it gradually entered the flow-limiting state and finally stabilized at around 1800 TPS.

[0106] Response time: The average response time was within 400 milliseconds, and the P99 response time did not exceed 800 milliseconds.

[0107] Request failure rate: Approximately 8%, mainly because some requests were intercepted by the flow-limiting mechanism.

[0108] 3.3 Impact scenario of long-tail requests Simulate that some requests have significantly extended response times due to network latency or database locks, etc., and observe the overall throughput change of the system and whether the flow-limiting strategy can effectively isolate abnormal requests.

[0109] Experimental results: Proportion of long-tail requests: 10% of the total request volume.

[0110] System performance: The overall TPS decreased slightly but still remained above 1400 times per second, the P99 response time did not exceed 1 second, and abnormal requests were effectively isolated.

[0111] In summary, in the face of higher-concurrency requests or abnormal traffic (such as simulating DDoS attacks), this method also performs excellently. It can ensure that the core transaction API reaches 1500 TPS while protecting the system from malicious traffic, significantly reducing the request failure rate and optimizing resource allocation.

[0112] Through the above detailed experimental design and result analysis, it can be seen that the application of this method not only enables the payment system to successfully cope with the load requirement of up to 1500 TPS, but also significantly improves the system stability, resource utilization efficiency, and user experience. This proves the effectiveness of this method in meeting the requirements of a high-performance payment system.

[0113] This embodiment also provides a flow limiting system combining a token bucket and a leaky bucket for a payment system, as Figure 2 shown. The technical solution adopted is as follows: It includes: a scoring module 1, a token adjustment module 2, a token distribution module 3, a leaky bucket module 4, and a step size adjustment factor module 5.

[0114] The scoring module is used to obtain the metrics of the payment system and calculate the health score; The token adjustment module is used to determine the load status of the payment system according to the health score and adjust the token generation rate and the token bucket capacity; The token distribution module is used to receive service requests and determine whether to pass or reject the service requests according to the number of available tokens; The leaky bucket module is used to read and process the passed service requests according to the leaky bucket algorithm; The step size adjustment factor module is used to adjust the adjustment step sizes of the token generation rate and the token bucket capacity according to the response situation of the payment system after multiple adjustments.

[0115] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A flow limiting method combining a token bucket and a leaky bucket for a payment system, characterized in that, including: S1: Obtain the metrics of the payment system and calculate the health score; S2: Determine the load status of the payment system according to the health score, and adjust the token generation rate and token bucket capacity; the adjustment step sizes of the token generation rate and token bucket capacity are adjusted according to the response of the payment system after multiple adjustments; When the health score is less than the low load threshold, reduce the token generation rate and token bucket capacity; When the health score is greater than the high load threshold, increase the token generation rate and token bucket capacity; When the health score is less than or equal to the high load threshold and greater than or equal to the low load threshold, keep the token generation rate and token bucket capacity unchanged; S3: Receive service requests, and determine whether to pass or reject the service requests according to the number of available tokens; S4: According to the leaky bucket algorithm, read and process the passed service requests.

2. The flow rate limiting method combining a token bucket and a leaky bucket for a payment system according to claim 1, characterized in that, The metrics of the payment system include: CPU utilization rate, memory occupancy rate, network bandwidth, real-time traffic, response time, and error rate; the health score is calculated by weighted summation of each metric.

3. The flow limiting method combining a token bucket and a leaky bucket for a payment system according to claim 1, characterized in that, The function of the health score is: Health score = w1 (1 - Normalized CPU utilization) + w2 (1 - Normalized memory occupancy) + w3 (Normalized network bandwidth) + w4 (1 - Normalized real-time traffic) + w5 (1 - Normalized response time) + w6 (1 - Normalized error rate) where, w1 is the first weight coefficient, w2 is the second weight coefficient, w3 is the third weight coefficient, w4 is the fourth weight coefficient, w5 is the fifth weight coefficient, and w6 is the sixth weight coefficient.

4. The flow rate limiting method combining a token bucket and a leaky bucket for a payment system according to claim 1, characterized in that, Set different token buckets for service requests with different priorities, When the health score is less than the low load threshold, increase the token generation rate and token bucket capacity of high-priority service requests, and reduce the token generation rate and token bucket capacity of low-priority service requests; When the health score is greater than the high load threshold, reduce the token generation rate and token bucket capacity of high-priority service requests, and increase the token generation rate and token bucket capacity of low-priority service requests.

5. The flow rate limiting method combining a token bucket and a leaky bucket for a payment system according to claim 1, characterized in that, The specific process of S3 is: S31: Receive service requests, and find the corresponding token bucket from the hash table according to the user ID or IP address; S32: Update the status of the token bucket; S33: Check whether the service request conforms to the traffic limiting rule. If it does not conform to the traffic limiting rule, reject this service request; if it conforms to the traffic limiting rule, execute S34; S34: Determine whether there are available tokens in the token bucket. If there are enough tokens, allow the request to pass and deduct the corresponding tokens; If there are not enough tokens, reject the service request.

6. The flow limiting method combining a token bucket and a leaky bucket for a payment system according to claim 5, wherein, The traffic limiting rules include that the number of service requests per unit time exceeds the limit, and it is judged whether the service requests of this user ID or IP address conform to the traffic limiting rule by the time stamps of the nearest N service requests stored in the queue; In S34, if the request is allowed to pass, add the service request time stamp to the queue.

7. The flow limiting method combining a token bucket and a leaky bucket for a payment system according to claim 1, characterized in that, In S3, add the passed service requests to the MQ message queue; in S4, according to the leaky bucket algorithm, read and process the passed service requests from the MQ message queue at a fixed rate.

8. The flow rate limiting method combining a token bucket and a leaky bucket for a payment system according to claim 1, characterized in that, If the response of the payment system after multiple adjustments simultaneously meets conditions (1) to (4), increase the adjustment step size of the token generation rate and the adjustment step size of the token bucket capacity; Condition (1): The health score improves, Condition (2): The request failure rate decreases, Condition (3): The response time is shortened, Condition (4): The throughput increases; If any one of the situations from situation (5) to situation (8) occurs in the response of the payment system after multiple adjustments, then reduce the adjustment step size of the token generation rate and the adjustment step size of the token bucket capacity; Situation (5): The health score is unstable, Situation (6): The request failure rate has no obvious change or deteriorates, Situation (7): The response time is extended, Situation (8): The throughput decreases.

9. A flow-limiting system combining a token bucket and a leaky bucket for a payment system, characterized in that, A rate limiting method combining a token bucket and a leaky bucket for a payment system as described in any one of claims 1 to 8, comprising: a scoring module, a token adjustment module, a token allocation module, a leaky bucket module, and a step size adjustment factor module, The scoring module is configured to obtain the metrics of the payment system and calculate the health score; The token adjustment module is configured to determine the load status of the payment system according to the health score and adjust the token generation rate and the token bucket capacity; The token allocation module is configured to receive a service request and determine whether to pass or reject the service request according to the number of available tokens; The leaky bucket module is configured to read and process the passed service requests according to the leaky bucket algorithm; The step size adjustment factor module is configured to adjust the adjustment step sizes of the token generation rate and the token bucket capacity according to the response situation of the payment system after multiple adjustments.

Images