An interface flow limiting method based on token bucket algorithm

Abstract

The application discloses an interface flow limiting method based on a token bucket algorithm. The method collects system load indicators and historical request flow data, dynamically generates a rate adjustment signal through a multi-dimensional weighted scoring mechanism, updates the token generation rate of a local token bucket, and broadcasts the rate parameter to other service nodes under the same interface via a distributed coordination node. In the token generation link, the inventory tokens are sequentially executed for time-limited removal and water level truncation operation. When the request arrives, the token acquisition is atomically executed, and a release signal or a flow limiting signal is generated. The differentiated degradation disposal is performed on the flow limiting request according to the priority of the calling party.

Description

Technical Field

[0001] This invention relates to the field of network communication technology, specifically to an interface rate limiting method based on the token bucket algorithm. Background Technology

[0002] The token bucket algorithm is a common technique in API rate limiting, achieving flow control by generating tokens into a bucket at a fixed rate and consuming tokens when requests arrive. However, existing token bucket algorithms have the following problems: First, they typically use a fixed token generation rate parameter, which cannot be dynamically adjusted according to the real-time system load, resulting in insufficient rate limiting accuracy when the system experiences changes in load. Second, existing solutions are designed for single-node scenarios and lack a synchronous broadcast mechanism for token generation rates among distributed multi-service nodes, leading to inconsistent rate limiting behavior across nodes in multi-node deployments. Third, existing solutions lack time constraints on the amount of tokens in the bucket, causing the accumulated tokens during low-load periods to be consumed in a concentrated manner when sudden traffic surges, creating a "pseudo-burst" effect that renders the rate limiting mechanism ineffective.

[0003] In the prior art, the patent "A Smart Bandwidth Allocation Method and Allocation System Based on Token Bucket" (CN113489655A) discloses a method that combines a monitoring center and a scheduling center to dynamically adjust the token bucket rate limiting parameters according to the current bandwidth usage of the device, providing a certain degree of dynamic rate adjustment approach to the first problem mentioned above. However, this patent has the following shortcomings: its rate adjustment relies on a single bandwidth usage indicator and does not introduce a comprehensive scoring mechanism based on multiple load indicators such as CPU utilization, memory usage, and interface response latency, which is insufficient to fully depict the actual stress state of the system in complex business scenarios; furthermore, this solution is designed for single-node bandwidth control scenarios and does not provide a synchronous broadcast mechanism for token generation rates among distributed multi-service nodes, which cannot guarantee the consistency of rate limiting behavior among nodes in multi-node distributed deployment scenarios; finally, this solution does not set a time-limit constraint mechanism for the existing tokens in the token bucket, and the existing tokens accumulated by the system during low-load periods are still at risk of being consumed in a concentrated manner when sudden traffic arrives, and the "pseudo-surge" problem is not solved.

[0004] To address the aforementioned issues, it is necessary to propose an interface rate limiting method based on the token bucket algorithm. Summary of the Invention

[0005] The purpose of this invention is to solve the problems existing in the background technology and to propose an interface rate limiting method based on the token bucket algorithm.

[0006] The objective of this invention can be achieved through the following technical solutions: This invention provides an interface rate limiting method based on the token bucket algorithm, which includes the following steps: S1. Collect the current system load index data and historical request traffic statistics, calculate the comprehensive load score according to the preset weight formula, compare the comprehensive load score with the preset grading threshold, and generate a rate adjustment signal.

[0007] Specifically, at the arrival of each preset collection period, the server reads the current CPU utilization, memory usage, and average interface response latency; the request counter performs a sliding window statistical analysis on the total number of requests arriving at the interface over several past statistical periods to calculate the historical average request rate and real-time request rate; a preset weighting formula is called to calculate the comprehensive load score based on CPU utilization, memory usage, average interface response latency, historical average request rate, and real-time request rate; the comprehensive load score is obtained by weighted summation of the following four items: the first item is CPU utilization multiplied by the corresponding weight coefficient; the second item is memory usage... The load factor is multiplied by the corresponding weighting coefficient; the third item is the ratio of the average interface response latency to the upper limit threshold of the interface response latency service quality, multiplied by the corresponding weighting coefficient; the fourth item is the ratio of the real-time request rate to the historical average request rate, multiplied by the corresponding weighting coefficient; the sum of the above four weighting coefficients is one; after the comprehensive load score is calculated, the comprehensive load score is compared with the preset grading threshold set, which contains three thresholds arranged from low to high. Based on the comparison result, a rate adjustment signal is generated, and the rate adjustment signal takes one of four discrete levels: significant increase, slight increase, slight decrease, and significant decrease.

[0008] S2. Based on the rate adjustment signal generated in step S1, update the token generation rate of the local token bucket, and after the update is completed, synchronize the updated rate parameters to the distributed coordination node, which then broadcasts the rate parameters to other service nodes under the same interface.

[0009] Specifically, each service node maintains a local token bucket instance. The local token bucket is stored in the node's memory as a data structure, containing the current number of tokens, the token generation rate, the bucket capacity limit, and the timestamp of the last token generation. The rate adjustment signal generated in step S1 is passed to the rate update logic of this node. The rate update logic updates the token generation rate according to the value of the rate adjustment signal and according to a preset adjustment step size. The updated token generation rate is subject to boundary constraints with the preset rate upper limit and rate lower limit. After the token generation rate of the local token bucket is updated, the updated rate parameters are encapsulated as a rate synchronization message and written to the rate configuration item of the corresponding interface in the distributed coordination node. After receiving the rate synchronization message, the distributed coordination node pushes the rate change notification to other service nodes that have subscribed to the rate configuration change channel of the interface through a message push mechanism. After receiving the rate change notification, other service nodes read the latest token generation rate in the distributed coordination node and update the token generation rate of their respective local token buckets accordingly.

[0010] S3. Perform token generation operation in the local token bucket. When generating a token, check the token's lifespan and the water level in the bucket. Based on the detection results, perform time-out clearing or water level truncation on the existing tokens, and output the number of valid tokens after clearing to step S4.

[0011] Specifically, the token generation operation is triggered at the arrival of each token generation cycle. The difference between the current timestamp and the last token generation timestamp is multiplied by the current token generation rate, and the result is rounded down to obtain the number of tokens to be added this time. The last token generation timestamp is then updated to the current timestamp. Each existing token in the token bucket maintains a generation timestamp. Before adding the newly generated tokens to the token bucket, two operations are performed in sequence: expiration clearing and water level truncation. Expiration clearing involves iterating through the generation timestamps of all existing tokens in the current bucket, marking tokens whose generation time is earlier than the current timestamp minus the preset token survival time limit as expired and removing them from the token bucket. After the expired tokens are cleared, the newly generated tokens are added to the token bucket, and the total number of tokens in the bucket after clearing is counted as the number of tokens after expiration clearing. Water level truncation involves comparing the number of tokens after expiration clearing with the bucket capacity limit, and taking the smaller value between the two as the number of valid tokens.

[0012] S4. Receive the interface request, attempt to obtain a token from the local token bucket, generate a pass signal or a rate limiting signal based on the acquisition result, and pass the signal type and request context information to step S5.

[0013] Specifically, when an interface request arrives, the interface identifier, caller identifier, and request timestamp are parsed from the request message and encapsulated into a request context structure. Then, a mutex lock is applied to the number of valid tokens in the local token bucket, and the token acquisition operation is performed atomically: if the current number of valid tokens is not less than one, the number of valid tokens is decremented by one, and a pass signal is generated; if the current number of valid tokens is zero, a rate limiting signal is generated. The pass signal or rate limiting signal is combined with the request context and passed to step S5. After the token acquisition is completed, the request counter is updated, and this request is recorded in the sliding window statistics structure in step S1.

[0014] S5. Based on the signal type and request context information passed in step S4, perform normal business processing on the release request and perform differentiated degradation operation on the rate limiting request according to the preset hierarchical handling strategy.

[0015] Specifically, when the signal type is a pass signal, the request is forwarded to the normal business processing flow; when the signal type is a rate limiting signal, the pre-defined caller priority configuration table is queried based on the caller identifier in the request context to determine the priority level of the caller, and the corresponding level of degradation handling strategy is executed: for high-priority callers, their requests are placed in a local bounded waiting queue and a maximum waiting time is set. If a valid token appears in the token bucket within the maximum waiting time, the request is retried to obtain the token and pass; if the wait times out, a service busy response is returned to the caller; for medium-priority callers, pre-defined degradation response data is returned to them; for low-priority callers, a standard rate limiting error response is returned to them, and the rejection event is recorded in the rate limiting log.

[0016] Compared with the prior art, the beneficial effects of the present invention are: This invention dynamically adjusts the token generation rate through a weighted scoring system based on multi-dimensional load metrics, solving the problem of insufficient rate limiting accuracy caused by static parameters; it achieves eventual consistency of token generation rates across nodes in a distributed scenario through a two-tier architecture consisting of a local token bucket and distributed coordination nodes; it eliminates the "pseudo-burst" effect caused by the backlog of existing tokens through a dual control mechanism of token expiration clearing and bucket capacity truncation; and it achieves differentiated degradation processing for different types of requests through a caller priority-based hierarchical handling strategy. Attached Figure Description

[0017] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating an interface rate limiting method based on the token bucket algorithm proposed in an embodiment of the present invention. Figure 2This is a comparison chart showing the actual amount of data allowed during burst traffic under the token validity management mechanism proposed in the embodiments of the present invention, with and without TTL schemes. Detailed Implementation

[0018] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] This invention provides an interface rate limiting method based on the token bucket algorithm. By collecting system load metrics and historical request traffic data in real time when interface requests arrive, a token generation rate adjustment signal is dynamically generated using a comprehensive scoring mechanism. Combined with a two-level token management architecture consisting of a local token bucket and distributed coordination nodes, and a dual constraint mechanism of token validity and bucket capacity level, this method solves the problems of parameter fixation, inconsistent token states among distributed nodes, and rate limiting failure caused by the unlimited accumulation of existing tokens in dynamic traffic scenarios. These problems are described in detail below.

[0020] First embodiment: Please see Figure 1 , Figure 1 This is a flowchart illustrating an interface rate limiting method based on the token bucket algorithm disclosed in an embodiment of the present invention. Figure 1 As shown, the method may include the following steps: S1. Collect the current system load index data and historical request traffic statistics, calculate the comprehensive load score according to the preset weight formula, compare the comprehensive load score with the preset grading threshold, and generate a rate adjustment signal.

[0021] Specifically, at the arrival of each preset data collection period, the system monitoring process reads the current CPU utilization, memory usage, and average interface response latency through the operating system kernel interface. All three metrics are temporarily stored numerically in a memory buffer. The default preset data collection period is 500 milliseconds.

[0022] Meanwhile, the request counter performs a sliding window statistical analysis on the total number of requests arriving at the interface over several past statistical periods (e.g., 10 past statistical periods with 1 second as one statistical period), calculating the historical average request rate and the real-time request rate in the most recent statistical period, both in times per second. All five data items are encapsulated in the form of a load snapshot data packet in the form of a structure for subsequent steps to read.

[0023] The system uses a preset weighting formula to calculate a comprehensive load score based on CPU utilization, memory usage, average interface response latency, historical average request rate, and real-time request rate. Specifically, the comprehensive load score is obtained by a weighted sum of the following four factors: The first item is the CPU utilization rate multiplied by the corresponding weighting coefficient; The second item is the memory usage rate multiplied by the corresponding weighting coefficient; The third item is the ratio of the average interface response latency to the upper limit threshold of the interface response latency service quality, multiplied by the corresponding weighting coefficient; The fourth item is the ratio of the real-time request rate to the historical average request rate, multiplied by the corresponding weighting coefficient. The sum of the four weighting coefficients is one, which is pre-configured by operations personnel according to the business scenario. When the average response latency of the interface exceeds the service quality upper limit threshold, the ratio of the third item is greater than one, and its contribution to the comprehensive load score will be significantly amplified, enabling the system to trigger rate limiting tightening actions more sensitively when response latency deteriorates. When the real-time request rate is significantly higher than the historical average request rate, the ratio of the fourth item is also greater than one, and its contribution to the comprehensive load score is correspondingly increased, enabling burst traffic to be detected in a timely manner and reflected in the scoring results.

[0024] After calculating the overall load score, the overall load score is compared with a preset set of tiered thresholds, which contains three thresholds arranged from low to high. Based on the comparison result, a corresponding rate adjustment signal is generated, and the value of the rate adjustment signal is one of the following four discrete levels: Tier 1: Significantly increased; Gear 2: Slightly increased; Gear 3: Slightly reduced; Level 4: Significantly Reduced. This corresponds to four different operational intentions: relaxing the token generation rate when the system load is low and tightening the token generation rate when the system load is high. This rate adjustment signal will be used as an input parameter for step S2 and passed to the token generation rate update logic.

[0025] It's important to note that this step introduces a comprehensive load scoring mechanism, rather than simply triggering rate limiting based on a single metric (such as CPU utilization). The significance of this design lies in the fact that a single metric often fails to fully reflect the system's true load condition—for example, normal CPU utilization but degraded response latency, or stable historical traffic but current sudden spikes are typical scenarios where a single metric can easily lead to misjudgment. By weighting and summing multi-dimensional metrics, the comprehensive load score can more stably and accurately depict the system's current actual processing capacity, thus making subsequent token generation rate adjustments more targeted and avoiding over- or under-limiting due to misjudgment.

[0026] S2. Based on the rate adjustment signal generated in step S1, update the token generation rate of the local token bucket, and after the update is completed, synchronize the updated rate parameters to the distributed coordination node, which then broadcasts the rate parameters to other service nodes under the same interface.

[0027] Specifically, the present invention maintains a local token bucket instance on each service node. The local token bucket is stored in the memory of the node in the form of a data structure, which includes the current number of tokens, the token generation rate, the upper limit of the bucket capacity, and the timestamp of the last token generation.

[0028] The rate adjustment signal generated in step S1 is passed to the rate update logic of this node. The rate update logic updates the token generation rate according to the value of the rate adjustment signal and a preset adjustment step size: when the rate adjustment signal is a significant upward adjustment, the token generation rate is increased by a preset large step size based on the current value, and the smaller value is taken from the preset rate upper limit to ensure the rate does not exceed the upper limit; when the rate adjustment signal is a slight upward adjustment, the token generation rate is increased by a preset small step size based on the current value, again taking the smaller value from the rate upper limit; when the rate adjustment signal is a slight downward adjustment, the token generation rate is decreased by a preset small step size based on the current value, and the larger value is taken from the preset rate lower limit to ensure the rate does not fall below the lower limit; when the rate adjustment signal is a significant downward adjustment, the token generation rate is decreased by a preset large step size based on the current value, again taking the larger value from the rate lower limit. Both the rate upper and lower limits are pre-configured by operations personnel according to service capacity planning to prevent rate adjustments from exceeding the limits.

[0029] After the token generation rate in the local token bucket is updated, the node encapsulates the updated rate parameters into a rate synchronization message and writes it to the corresponding rate configuration item in the distributed coordinating node via a network call. This configuration item uses the interface identifier as the key and the serialized token generation rate value and the current timestamp as the value. Upon receiving the rate synchronization message, the distributed coordinating node pushes the rate change notification to other service nodes that have subscribed to the rate configuration change channel for that interface via a message push mechanism. After receiving the rate change notification, other service nodes read the latest token generation rate from the distributed coordinating node and update their own local token bucket's token generation rate accordingly, completing cross-node rate consistency synchronization.

[0030] It's important to note that this step employs a two-tier architecture combining a local token bucket with rate broadcasting from the distributed coordinating node, rather than centralizing all token generation and consumption at the distributed coordinating node. The core significance of this design lies in the fact that real-time token acquisition and consumption are entirely completed in local memory, avoiding the performance overhead of remote network calls for every request, and significantly reducing latency in rate limiting decisions. Rate parameter synchronization is only triggered when the token generation rate changes, a low-frequency control plane operation with extremely limited impact on network bandwidth and the load on the distributed coordinating node. Simultaneously, through a message push mechanism, each node can complete synchronization within a very short time window after a rate parameter change, ensuring eventual consistency of token generation rates across multiple nodes in a distributed scenario, effectively avoiding overall rate limiting accuracy distortion caused by asynchronous rates among nodes.

[0031] S3. Perform token generation operation in the local token bucket. When generating a token, check the token's lifespan and the water level in the bucket. Based on the detection results, perform time-out clearing or water level truncation on the existing tokens, and output the number of valid tokens after clearing to step S4.

[0032] Specifically, the token generation operation is triggered at the arrival of each token generation cycle. The token generation cycle is determined by the current token generation rate, which is equal to the reciprocal of the rate. The token generation logic first calculates the number of tokens to be added by multiplying the difference between the current timestamp and the last token generation timestamp by the current token generation rate, and then rounds the result down to ensure that no non-integer tokens are generated. After the calculation is complete, the last token generation timestamp is updated to the current timestamp.

[0033] Before adding the newly generated tokens to the token bucket, this step introduces a dual existing token management mechanism, performing two operations in sequence: expiration period clearing and water level cutoff. Expiration Time Removal: This invention maintains a generation timestamp for each existing token in the token bucket. At the start of each token generation cycle, the generation timestamps of all existing tokens in the current bucket are iterated. Tokens whose generation time is earlier than the current timestamp minus a preset token lifespan limit are marked as expired and removed from the token bucket. The token lifespan limit can be set to, for example, 1 second. After removing expired tokens, newly generated tokens are added to the token bucket. The total number of tokens in the bucket after removal is recorded as the number of tokens after expiration time removal.

[0034] Each token carries the current timestamp as the generation timestamp.

[0035] Water Level Cutoff: After the time-limited clearing is completed, the number of tokens remaining after the time-limited clearing is compared with the bucket's maximum capacity. If the number of tokens after the time-limited clearing exceeds the bucket's maximum capacity, the total number of tokens in the bucket is cut off to the maximum capacity, and any excess is discarded. If the number of tokens after the time-limited clearing does not exceed the bucket's maximum capacity, the number remains unchanged. The number of tokens in the bucket after the cutoff is the valid number of tokens, which is the smaller of the number of tokens remaining after the time-limited clearing and the bucket's maximum capacity.

[0036] The number of valid tokens will be used as the input parameter for step S4 and passed to the request token acquisition and release judgment logic.

[0037] It should be noted that the dual token control design proposed in this step, combining time-sensitive clearing and water level cutoff, is a direct improvement on the deficiency of existing token bucket algorithms in that there is no expiration mechanism for existing tokens. In existing schemes, when the system is under low load for a long time, tokens will continue to accumulate in the bucket until the capacity limit is reached; once a sudden surge in requests occurs, a large number of accumulated tokens will be consumed all at once in a very short time, causing the actual instantaneous request volume that the system can withstand to far exceed its processing capacity, producing a "pseudo-burst" effect, which makes the rate limiting mechanism essentially ineffective.

[0038] The time-limited clearing mechanism, by assigning time-limited validity to tokens, forcibly clears out old tokens that have accumulated during periods of low load, ensuring that the number of tokens in the bucket always reflects the recent effective traffic processing capacity, rather than the "false surplus" accumulated during historical low periods. The water level cutoff further ensures that even if there is a certain delay in the execution of time-limited clearing, the total number of tokens in the bucket will not exceed the system's designed capacity limit. Together, these two mechanisms constitute a double protection against the token backlog problem.

[0039] S4. Receive the interface request, attempt to obtain a token from the local token bucket, generate a pass signal or a rate limiting signal based on the acquisition result, and pass the signal type and request context information to step S5.

[0040] Specifically, when an interface request arrives, the request processing thread parses the interface identifier, caller identifier, and request timestamp from the request message and encapsulates them into a request context structure.

[0041] The caller is identified as either the client's IP address or the user's ID.

[0042] The request processing thread then attempts to atomically acquire a token from the local token bucket. The atomic acquisition operation ensures concurrency safety by applying a mutex lock to the number of valid tokens in local memory: if the current number of valid tokens is not less than one, the number of valid tokens is decremented by one; if the operation succeeds, a release signal is generated, and the release signal is combined with the request context and passed to step S5; if the current number of valid tokens is zero, the token acquisition fails, a rate-limiting signal is generated, and the rate-limiting signal is combined with the request context and passed to step S5.

[0043] After the token is acquired, the request processing thread updates the request counter and records the current request in the sliding window statistics structure in step S1. This ensures that when the next collection cycle arrives, the statistical value of the historical average request rate can reflect the arrival status of the current request, forming a data flow closed loop between steps S1 and S4.

[0044] It should be noted that the token acquisition operation in this step is completed entirely in local memory and does not involve any remote calls. Therefore, the incremental latency for processing a single request is extremely low, and stable rate limiting performance can be maintained even in high-concurrency scenarios. Passing the request context and signal type to step S5 together ensures that subsequent differentiated handling operations can be aware of the specific identity information of the rate-limited request, providing necessary context support for the fine-grained handling in step S5.

[0045] S5. Based on the signal type and request context information passed in step S4, perform normal business processing on the release request and perform differentiated degradation operation on the rate limiting request according to the preset hierarchical handling strategy.

[0046] Specifically, the signal type and request context output in step S4 are passed to the request dispatch logic of this step: When the signal type is a pass signal, the request is forwarded to the normal business processing flow, where the backend service performs complete business logic processing on the request and returns the response result. No additional intervention is required in this step.

[0047] When the signal type is a rate-limiting signal, the request distribution logic queries a pre-defined caller priority configuration table based on the caller identifier in the request context to determine the priority level of the caller. The priority configuration table is pre-configured by operations personnel, dividing callers into several priority levels according to their type, with each level corresponding to a different degradation handling strategy. Caller types include, but are not limited to: internal core services, VIP merchants, ordinary users, and unauthenticated callers.

[0048] For high-priority callers, such as internal core services, when their requests are rejected by the token bucket, the dispatch logic does not discard the request directly. Instead, it places the request in a local bounded waiting queue and sets a maximum waiting time for it. The request waits in the waiting queue for subsequent token replenishment. If an available token becomes available in the token bucket within the maximum waiting time, the request is retried to acquire a token and allowed to proceed. If the wait times out, a "service busy" message is returned to the caller, instead of directly returning a rate-limiting error code.

[0049] For medium-priority callers, such as VIP merchants, when their requests are rejected by the token bucket, the distribution logic returns pre-defined degraded response data to the caller, such as returning the cached previous normal response result or simplified business data, to ensure that the caller can obtain meaningful response content rather than an empty error.

[0050] For low-priority callers, such as ordinary users or unauthenticated callers, when their requests are rejected by the token bucket, the distribution logic directly returns a standard rate-limiting error response to the caller and records the rejection event to the rate-limiting log for subsequent traffic auditing.

[0051] All three types of processing results are completed within this step and will not flow back to step S4. The entire flow restriction processing chain ends here.

[0052] It's important to note that this step introduces a priority-based handling strategy for callers, rather than simply rejecting all rate-limited requests. The significance of this is that in real-world business scenarios, different callers have significantly different levels of dependence on API availability. A simplistic, uniform rejection strategy would cause unnecessary service interruptions for high-value, highly dependent core callers, while allowing too much low-priority traffic. By combining token bucket rate-limiting decisions with caller identity awareness, this step enables the rate-limiting mechanism to protect overall system stability while providing differentiated service quality guarantees for different business scenarios, improving the business adaptability and user-friendliness of the rate-limiting solution in actual deployments.

[0053] It should be further explained that steps S1 to S5 constitute a complete closed-loop linkage system: step S1 is responsible for sensing the system status and generating a rate adjustment signal; step S2 is responsible for converting the adjustment signal into a token rate update and synchronizing it across nodes; step S3 is responsible for performing dual control over timeliness and water level during the token generation stage; step S4 is responsible for performing atomic acquisition and release judgment of the token when the request arrives; and step S5 is responsible for performing differentiated business processing for different types of signals. The steps are sequentially transmitted through parameters such as the rate adjustment signal, token generation rate, number of valid tokens, signal type, and request context, forming a complete data flow path from load perception, dynamic parameter adjustment, token quality control, request release judgment to differentiated degradation processing. Each step is both independent and collaborative, jointly achieving refined and adaptive control of interface flow limiting behavior.

[0054] Second embodiment: The second embodiment is based on the first embodiment, and further elaborates on the calculation process of the comprehensive load score in step S1, including the method of determining the weight coefficients and the specific implementation of the sliding window statistics. The remaining steps S2, S3, S4 and S5 are consistent with the first embodiment, and will not be described again here.

[0055] In step S1 of the first embodiment, the comprehensive load score is obtained by weighted summation of four factors: CPU utilization, memory usage, the ratio of average interface response latency to the service quality upper limit threshold, and the ratio of real-time request rate to historical average request rate. The weight coefficients for each factor are pre-configured by operations personnel. However, in actual deployment scenarios, the sensitivity of interfaces of different business types to the above four indicators varies significantly—computation-intensive interfaces are more sensitive to CPU utilization, while memory-intensive interfaces or database query interfaces react more drastically to changes in response latency. If all four weight coefficients are statically configured, it is difficult to reuse the same weight configuration among interfaces with significantly different business characteristics, and it is impossible to automatically correct weight deviations based on historical data during system operation. To solve the above problems, this embodiment further refines the method for determining the weight coefficients and the statistical method for calculating the historical average request rate.

[0056] Regarding the determination of weight coefficients, this embodiment introduces a weight adaptive mechanism during the system initialization phase. Specifically, before the system goes live, a pre-heating observation period is executed. The duration of the pre-heating observation period can be set to, for example, 24 hours, covering the complete peak and off-peak business cycles. During the pre-heating observation period, the system simultaneously records the values ​​of four raw indicators and the actual response status of the interface at the corresponding time in each collection cycle. The actual response status of the interface is measured by the error rate and timeout rate of the interface in that collection cycle. After the pre-heating observation period ends, the system performs correlation analysis on the collected historical data, calculating the Pearson correlation coefficient between the values ​​of the four raw indicators and the deterioration events of the actual response status of the interface. Indicators with higher correlation coefficients indicate that their value changes have a more significant impact on the interface response quality, and the corresponding weight coefficient should be larger; indicators with lower correlation coefficients correspond to smaller weight coefficients. The system normalizes each weight coefficient according to the above correlation coefficients, so that the sum of the four weight coefficients is one, and writes the normalized weight coefficients into the configuration items for step S1 to call during the formal operation phase. During the formal operation phase, the system re-executes the above correlation analysis process every preset weight update cycle (e.g., every seven days) to continuously correct the weight coefficients in a rolling update manner, ensuring that the weight configuration of the comprehensive load score can remain adaptive as the business traffic characteristics evolve over a long period of time.

[0057] It should be noted that this embodiment introduces a weight adaptive mechanism based on correlation analysis, replacing the static weight configuration method in the first embodiment that relies entirely on manual experience. This improvement allows the weight allocation of the comprehensive load score to be automatically derived from real historical data, rather than relying on the subjective judgment of operations personnel on business characteristics. This reduces the risk of misjudgment of rate limiting due to improper weight configuration, and enables the same weight update mechanism to adapt to interfaces of different business types, improving the portability of the solution.

[0058] Regarding the statistical method for the historical average request rate, this embodiment further refines the specific data structure and update logic of the sliding window statistics in the first embodiment. Specifically, the system maintains a circular buffer for each protected interface. This circular buffer contains several time slots, each corresponding to a statistical period (e.g., each time slot corresponds to one second). The total number of time slots is equal to the number of statistical periods covered by the sliding window (e.g., ten time slots are set to cover the past ten seconds). Each time slot stores two pieces of data: the request count arriving at the interface within the statistical period, and the start timestamp corresponding to the time slot. The circular buffer has a pointer that always points to the current time slot.

[0059] At the start of each data collection period, the system first determines whether the current time slot has expired based on the current timestamp. Specifically, it checks if the difference between the current timestamp and the start timestamp of the current time slot exceeds the duration of one statistical period. If expired, the pointer moves forward one position to the next time slot, the request count for that time slot is reset to zero, and the start timestamp is updated to the start time of the current statistical period. Subsequent requests are then accumulated in this new time slot. If not expired, incoming requests continue to be accumulated in the current time slot's count. The historical average request rate is calculated by summing the request counts of all time slots in the circular buffer and dividing by the total duration covered by the sliding window (i.e., the product of the statistical period duration and the number of time slots). This yields the average number of requests per unit time, i.e., the historical average request rate. The real-time request rate is directly estimated by dividing the request count of the current time slot by the duration of the current statistical period.

[0060] The above-mentioned sliding window statistical implementation based on a circular buffer only requires a constant number of pointer movement, clearing, and summation operations in each collection cycle. There is no full scan of historical data, and the time cost of statistical calculation is fixed and extremely low. It is suitable as the underlying data source for step S1 in high-concurrency rate limiting scenarios.

[0061] It should be noted that this embodiment uses a circular buffer instead of a linked list or dynamic array to implement sliding window statistics due to considerations of memory allocation overhead. The circular buffer allocates a fixed-size contiguous memory region at initialization. Subsequent update operations only modify the allocated memory in-place, without triggering any dynamic memory allocation or reclamation operations. This fundamentally eliminates the latency jitter that may be introduced by frequent dynamic memory operations in high-concurrency scenarios, ensuring that the data collection and calculation process in step S1 has stable and predictable time characteristics for the data supply to subsequent steps.

[0062] Third embodiment: This embodiment verifies the improvement effect of the token time-limited control mechanism proposed in the first embodiment through load simulation experiments. It focuses on comparing the difference in the actual number of requests allowed during burst traffic in two scenarios: enabling the token time-limited control mechanism (i.e., the first embodiment scheme, hereinafter referred to as the "TTL-enabled scheme") and not enabling the token time-limited control mechanism (i.e., the existing standard token bucket scheme, hereinafter referred to as the "TTL-free scheme").

[0063] The experiment was conducted on a server with a 4-core CPU and 8GB of memory. The target rate for interface rate limiting was set to 1000 times / second, the maximum token bucket capacity was set to 1000 tokens, the maximum token lifespan (TTL) was set to 1 second, and the collection period was set to 500 milliseconds.

[0064] The experimental traffic pattern was designed as follows: For the first 60 seconds, the system was under low load, with an incoming request rate of 700 requests / second, far below the rate limiting target. During this period, the token bucket continuously accumulated tokens. From the 61st second, the traffic switched to burst mode, with the incoming request rate instantly increasing to 5000 requests / second, lasting for 10 seconds. From the 71st second, the traffic dropped back to 1000 requests / second and remained stable until the experiment ended at the 80th second. Experimental comparison data is as follows: Figure 2 As shown.

[0065] Please see Figure 2 As shown, Figure 2 This chart compares the actual throughput during burst traffic events under the token validity management mechanism with and without TTL.

[0066] The experimental results are as follows: In the no-TTL scheme, during the low-load phase (from the 1st to the 60th second), the token bucket continuously consumes tokens at a rate of 700 times / second, while continuously generating tokens at a rate of 1000 tokens / second. The existing tokens accumulate to the bucket capacity limit of 1000 within about 1 second, and thereafter the bucket always maintains a full supply. At the 61st second after the burst traffic, the actual number of requests allowed was 2000, of which 1000 came from the full supply of tokens accumulated during the low-load period, and the other 1000 came from the newly generated tokens in the 61st second. The actual allowance exceeded the system's designed single-second processing capacity limit by 100%. From the 62nd second onwards, the existing tokens were exhausted, and the actual allowance dropped back to 1000 times / second. In the TTL-enabled scheme, tokens generated during low-load periods undergo continuous expiration and removal over time. Tokens generated more than one second ago are marked as expired and removed at the end of each token generation cycle. The number of valid tokens in the bucket always only includes those generated within the most recent second that have not been consumed. At the end of the 60th second, the number of valid tokens in the bucket is approximately 300, corresponding to the net remaining tokens after being generated at a rate of 1000 tokens / second and consumed at a rate of 700 times / second during the low-load period in the most recent second. In the 61st second after the arrival of burst traffic, the actual number of allowed requests is approximately 1300 (300 existing tokens plus 1000 newly generated tokens in the 61st second), which is about 35% less than the non-TTL scheme, effectively suppressing the interference of existing token backlog on the accuracy of burst traffic control. From the 62nd second onwards, the actual allowed requests also converge to 1000 times / second.

[0067] The experimental results above show that, in scenarios where the system suddenly encounters a traffic surge after being in a low-load state for a long time, the non-TTL solution, due to the lack of time constraints on existing tokens, causes the actual allowance of the sudden traffic in the first second to reach twice the system's designed processing capacity, forming a significant "pseudo-burst" effect. In contrast, the TTL solution of this invention, through the token time-limited clearing mechanism, controls the excess allowance of the sudden traffic in the first second to within 30% of the designed capacity, making the actual behavior of the system under traffic surge scenarios more consistent with the rate limiting design target.

[0068] It should be understood that the terms “comprising” and “including” used in this disclosure and claims indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0069] It should also be understood that the terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used in this disclosure and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this disclosure and claims means any combination and all possible combinations of one or more of the associated listed items, and includes such combinations; The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to any specific implementation. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. An interface rate limiting method based on the token bucket algorithm, characterized in that, Includes the following steps: S1. Collect the current system load index data and historical request traffic statistics, perform weighted calculation on the collected data according to the preset weight formula to obtain the comprehensive load score, compare the comprehensive load score with the preset grade threshold set, and generate a rate adjustment signal. S2. Based on the rate adjustment signal generated in step S1, update the token generation rate of the local token bucket according to the preset adjustment step size, and write the updated rate parameter to the distributed coordination node after the update is completed. The distributed coordination node then broadcasts the rate parameter to other service nodes under the same interface. Other service nodes update the token generation rate of their respective local token buckets according to the received rate parameter. S3. Perform a token generation operation in the local token bucket. Before adding the newly generated token to the token bucket, perform an expiration clearing operation and a water level cutoff operation on the existing tokens in sequence to obtain the number of valid tokens. S4. Receive the interface request, atomically perform the token acquisition operation on the number of valid tokens in the local token bucket, generate a pass signal and decrement the number of valid tokens by one if the number of valid tokens is not less than one, generate a rate limiting signal if the number of valid tokens is zero, and pass the signal type and request context information to step S5. S5. When the signal type is a pass signal, forward the request to the normal business processing flow; when the signal type is a rate limiting signal, determine the priority level of the caller according to the caller identifier in the request context information, and perform differentiated degradation operation on the request according to the corresponding degradation handling strategy.

2. The interface rate limiting method based on the token bucket algorithm according to claim 1, characterized in that, In step S1, the load metric data includes CPU utilization, memory usage, and average interface response latency; the historical request traffic statistics include historical average request rate and real-time request rate; the comprehensive load score is obtained by weighted summation of the following four items: CPU utilization multiplied by the corresponding weight coefficient, memory usage multiplied by the corresponding weight coefficient, the ratio of average interface response latency to the interface response latency service quality upper limit threshold multiplied by the corresponding weight coefficient, and the ratio of real-time request rate to historical average request rate multiplied by the corresponding weight coefficient, and the sum of the above four weight coefficients is one.

3. The interface rate limiting method based on the token bucket algorithm according to claim 2, characterized in that, In step S1, the graded threshold set includes three thresholds arranged from low to high; the value of the rate adjustment signal is one of four discrete levels: large increase, slight increase, slight decrease, and large decrease.

4. The interface rate limiting method based on the token bucket algorithm according to claim 1, characterized in that, In step S2, the local token bucket is stored in the memory of the service node in the form of a data structure, which includes the current number of tokens, the token generation rate, the upper limit of the bucket capacity, and the timestamp of the last token generation; the update result of the rate parameter is subject to boundary constraints with the preset upper limit and lower limit of the rate.

5. The interface rate limiting method based on the token bucket algorithm according to claim 1, characterized in that, In step S2, the distributed coordination node broadcasts the rate parameters to other service nodes in the following way: the distributed coordination node pushes rate change notifications to other service nodes that have subscribed to the interface rate configuration change channel through a message push mechanism. After receiving the rate change notification, other service nodes read the latest token generation rate from the distributed coordinating node and update their local token buckets accordingly.

6. The interface rate limiting method based on the token bucket algorithm according to claim 1, characterized in that, In step S3, the specific steps of the expiration clearing operation are as follows: each existing token in the token bucket maintains a generation timestamp, iterates through the generation timestamps of all existing tokens in the current bucket, marks tokens whose generation time is earlier than the current timestamp minus the preset token survival time limit as expired and removes them from the token bucket, after the expired tokens are cleared, the newly generated tokens are added to the token bucket, and the total number of tokens in the bucket after clearing is counted and recorded as the number of tokens after expiration clearing; the specific steps of the water level cutoff operation are as follows: take the smaller value between the number of tokens after expiration clearing and the upper limit of the bucket capacity as the number of valid tokens.

7. The interface rate limiting method based on the token bucket algorithm according to claim 1, characterized in that, In step S3, the specific steps of the token generation operation are as follows: multiply the difference between the current timestamp and the previous token generation timestamp by the current token generation rate, round down the calculation result to obtain the number of tokens to be added this time, and update the previous token generation timestamp to the current timestamp.

8. The interface rate limiting method based on the token bucket algorithm according to claim 1, characterized in that, In step S4, the request context information is obtained by parsing the interface identifier, caller identifier, and request timestamp from the request message; the caller identifier is the client IP or user ID; after the token acquisition operation is completed, this request is recorded in the sliding window statistics structure in step S1 to update the request counter.

9. The interface rate limiting method based on the token bucket algorithm according to claim 1, characterized in that, In step S5, the differentiated degradation operation includes: for high-priority callers, placing their requests into a local bounded waiting queue and setting a maximum waiting time; if a valid token appears in the token bucket within the maximum waiting time, re-executing the token acquisition operation and allowing the request to proceed; if the wait times out, returning a service busy response to the caller; for medium-priority callers, returning preset degradation response data to them; and for low-priority callers, returning a standard rate limiting error response to them and recording the rejection event in the rate limiting log.

10. The interface rate limiting method based on the token bucket algorithm according to claim 2, characterized in that, In step S1, the weight coefficients are determined as follows: a warm-up observation period is performed before the system is officially launched. During the warm-up observation period, the values ​​of the four original indicators and the actual response status of the interface at the corresponding time are recorded in each collection cycle. After the warm-up observation period ends, the Pearson correlation coefficient between the values ​​of the four original indicators and the deterioration event of the actual response status of the interface is calculated. The weight coefficients are normalized according to the Pearson correlation coefficients so that the sum of the four weight coefficients is one. During the formal operation phase, the above calculation process is re-executed every preset weight update cycle to update the weight coefficients on a rolling basis.

Images