Distributed current limiting method, device, system and storage medium

Abstract

The application discloses a kind of distributed current limiting method, device, system and storage medium, including periodically generating and broadcasting concurrent suggestion interval based on local running state;Receive concurrent suggestion interval generated by itself and from other service instances and carry out intersection operation, obtain global feasible interval, upper limit is the minimum value of the upper limit of all concurrent suggestion intervals, lower limit is the maximum value of the lower limit of all concurrent suggestion intervals;Determine local current limiting threshold based on global feasible interval and local dynamic weight.The intersection operation is carried out after each service instance of the application receives concurrent suggestion interval to determine global feasible interval, ensure that upper limit is monotonically not increased and upper limit cannot be exceeded, obtain safe invariant constraint chain, without central coordinator or global synchronous clock, in any network condition, guarantee that global concurrency is not more than safe upper limit, determine local current limiting threshold based on global feasible interval and local dynamic weight, realize fine, adaptive current limiting management.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a distributed rate limiting method, apparatus, system, and storage medium. Background Technology

[0002] In large-scale distributed architectures, rate limiting is crucial for ensuring system stability. Mainstream rate limiting solutions fall into two categories: 1) Centralized global counters (such as Redis) can precisely control the total amount of data, but suffer from single points of failure and performance bottlenecks; 2) Decentralized solutions allow each instance to count independently and synchronize via the Gossip protocol, offering good fault tolerance and scalability, but may exceed the global limit in the short term, making them unsuitable for scenarios requiring strict resource control. Furthermore, existing solutions struggle to dynamically adjust multi-tenant resource allocation and lack effective mechanisms for passing latency budgets or concurrency credits in complex microservice call chains. This leads to a mismatch between rate limiting decisions for upstream and downstream services and actual processing capacity, impacting the effectiveness of global collaborative degradation and congestion control. Summary of the Invention

[0003] This invention aims to at least solve one of the technical problems existing in the prior art. To this end, this invention proposes a distributed rate limiting method, device, system, and storage medium, which achieves a globally insurmountable upper limit through the intersection of intervals.

[0004] In a first aspect, embodiments of the present invention provide a distributed rate limiting method, applied to a service instance of a distributed system, including:

[0005] The system periodically generates concurrency suggestion intervals based on the local running status and broadcasts these intervals to other service instances in the distributed system.

[0006] The system receives concurrent suggestion intervals generated by itself and from other service instances, and performs an intersection operation on all received concurrent suggestion intervals to obtain a globally feasible interval. The upper limit of the globally feasible interval is the minimum value of the upper limits of all concurrent suggestion intervals, and the lower limit of the globally feasible interval is the maximum value of the lower limits of all concurrent suggestion intervals.

[0007] Based on the global feasible range and the local dynamic weight, the local rate limiting threshold is determined.

[0008] According to some embodiments of the present invention, the step of periodically generating concurrent suggestion intervals based on local running state includes:

[0009] Obtain local running status, which includes CPU utilization, available memory ratio, GC activity metrics, network I / O metrics, number of requests being processed, request queue length, and some or all of the success rate, latency rate, and default rate based on historical statistics.

[0010] Based on preset rules or interval generation models, concurrent suggestion intervals are generated according to the local running status.

[0011] According to some embodiments of the present invention, determining the local rate limiting threshold based on the globally feasible interval and the local dynamic weight includes:

[0012] Based on the weight ratio of the local dynamic weight and the upper limit of the global feasible interval, a first concurrency threshold is determined by rounding down.

[0013] The first tail difference margin is determined based on the first concurrency threshold of all service instances and the upper limit of the global feasible interval;

[0014] The first tail difference surplus is allocated one by one to the multiple service instances with the highest dynamic weight by increasing the concurrency threshold by one unit for each service instance, thus obtaining the local rate limiting threshold.

[0015] According to some embodiments of the present invention, the local dynamic weight is obtained by weighted summation based on the reciprocal of CPU utilization, the proportion of available memory, and the complement of the default rate.

[0016] According to some embodiments of the present invention, the step of determining the local rate limiting threshold based on the globally feasible interval and the local dynamic weight further includes:

[0017] Under the constraint of the local rate limiting threshold, a tenant-level rate limiting quota is allocated to the tenant.

[0018] According to some embodiments of the present invention, the allocation of tenant-level rate limiting quotas to the tenant under the constraint of the local rate limiting threshold includes:

[0019] Based on the tenant dynamic weight of each tenant and the local rate limiting threshold, a second concurrency threshold is determined by rounding down.

[0020] The second tail difference margin is determined based on the second concurrency threshold of all tenants and the local rate limiting threshold;

[0021] The second tail difference surplus is allocated to the tenants with the highest dynamic weights by increasing the concurrency threshold by one unit for each tenant, thus obtaining the tenant-level rate limiting quota.

[0022] According to some embodiments of the present invention, the step of determining the local rate limiting threshold based on the global feasible interval and the local dynamic weight further includes: upon receiving a service call request carrying a budget token, parsing the budget token, and making a response decision based on the local rate limiting threshold, the concurrent credit value obtained from the budget token, and the remaining latency in milliseconds.

[0023] According to some embodiments of the present invention, the distributed rate limiting method further includes:

[0024] Under the premise of meeting the preset borrowing conditions, the flow limit quota of the lending tenant is allocated to the borrowing tenant, and the lending information is recorded, including the lending tenant, the borrowing tenant, the lending quota, the lending validity period, and the reduction factor.

[0025] When the loan validity period expires, or when the default rate of the lending tenant exceeds a preset threshold and continues for a preset period of time, the loan quota will be gradually returned from the borrowing tenant to the lending tenant in multiple cycles according to the reduction factor.

[0026] According to some embodiments of the present invention, the distributed rate limiting method further includes:

[0027] After the network partition failure is recovered, the global feasible interval is updated, and the local rate limiting threshold is updated in multiple cycles according to the updated global feasible interval. The change of the local rate limiting threshold in a single cycle is less than or equal to a preset proportion of the current local rate limiting threshold.

[0028] Secondly, embodiments of the present invention provide a distributed rate limiting device applied to a service instance of a distributed system, including:

[0029] The interval management module is used to periodically generate concurrent suggestion intervals based on the local running status and broadcast the concurrent suggestion intervals to other service instances of the distributed system;

[0030] The CRDT aggregation module is used to receive concurrent suggestion intervals generated by itself and from other service instances, and to perform an intersection operation on all received concurrent suggestion intervals to obtain a globally feasible interval. The upper limit of the globally feasible interval is the minimum value of the upper limits of all concurrent suggestion intervals, and the lower limit of the globally feasible interval is the maximum value of the lower limits of all concurrent suggestion intervals.

[0031] The quota allocation module is used to determine the local rate limiting threshold based on the global feasible range and the local dynamic weight, and to be constrained by the local rate limiting threshold.

[0032] Thirdly, embodiments of the present invention provide a distributed rate limiting system, including a processor and a memory, wherein the memory stores a computer program, and the processor runs the computer program to implement the above-described distributed rate limiting method.

[0033] Fourthly, embodiments of the present invention provide a storage medium storing a computer program, which implements the above-described distributed rate limiting method when the computer program is run.

[0034] The embodiments of the present invention have at least the following beneficial effects:

[0035] This invention generates concurrent suggestion intervals periodically based on local operating status and broadcasts them to other service instances. Each service instance receives the concurrent suggestion intervals and performs an intersection operation to determine the globally feasible interval, ensuring that the upper limit is monotonically non-increasing and cannot be exceeded, thus obtaining a safe and invariant constraint chain. It does not require a central coordinator or a global synchronization clock and ensures that the global concurrency does not exceed the safe upper limit under any network conditions. Based on the globally feasible interval and local dynamic weights, the local rate limiting threshold is determined, achieving fine-grained and adaptive rate limiting management.

[0036] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0037] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0038] Figure 1 This is a flowchart illustrating the steps of the distributed rate limiting method according to an embodiment of the present invention.

[0039] Figure 2 This is a schematic diagram of the distributed rate limiting device according to an embodiment of the present invention;

[0040] Figure 3 This is a timing diagram of the budget transfer process according to an embodiment of the present invention;

[0041] Figure 4 This is a timing diagram of the consistency compensation process in an embodiment of the present invention;

[0042] Figure 5 This is a block diagram illustrating the principle of a distributed rate limiting system according to an embodiment of the present invention. Detailed Implementation

[0043] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0044] In the description of this invention, "several" means one or more, "multiple" means two or more, "greater than," "less than," "exceeding," etc. are understood to exclude the stated number, and "above," "below," "within," etc. are understood to include the stated number. If "first," "second," etc. are used in the description, they are only for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance or implicitly indicating the number of indicated technical features or the order of the indicated technical features.

[0045] Terminology Explanation:

[0046] A node is a physical or virtual computing unit in a distributed system, typically referring to a server, container, virtual machine, or runtime environment, with independent resources such as CPU, memory, and network. A node is the basic carrier for system deployment and execution, supporting one or more service instances.

[0047] A service instance is a specific process or containerized copy of a microservice or application component running on a specific node. The role of a service instance is to provide specific functional interfaces (such as REST API, RPC) to the outside world and is the target unit for service calls.

[0048] Network partitioning refers to a phenomenon in a distributed system where, due to network failures (such as link interruption, switch failure, firewall policy error, DNS failure, etc.), a group of nodes that could originally communicate with each other are divided into two or more isolated subnets. Nodes within each subnet can still communicate, but the subnets cannot communicate with each other.

[0049] Conflict-free Replicated Data Type (CRDT) is a data structure designed specifically for highly available, eventually consistent distributed systems. Its core characteristic is that multiple replicas can be automatically merged into a consistent state without coordination in the event of concurrent updates, network partitions, or out-of-order synchronization, and the result is correct and conflict-free.

[0050] Eventual consistency is an important weak consistency model in distributed systems. Its core idea is that, without new updates, after a finite but uncertain period of time, all replicas in the system will automatically converge to a consistent state. Eventual consistency satisfies the following conditions: the system allows concurrent writes to multiple replicas; update operations may not immediately propagate to all nodes; but once writes stop and the network is normal and fault-free, the system will eventually achieve consistency across all replicas through a background synchronization mechanism.

[0051] The concurrency recommendation range [l_i, u_i] refers to the lower and upper limits of the number of concurrent requests that service instance i can safely handle within an evaluation period, based on its own health status and processing capacity. l_i represents the minimum number of concurrent requests required to ensure basic service quality, and u_i represents the maximum safe number of concurrent requests that will not cause overload.

[0052] Global feasible interval [L_min, L_max]: refers to the globally shared safe concurrency interval obtained after the concurrency suggestion intervals reported by all service instances in the cluster are processed by CRDT interval intersection and merging operation. L_min is the maximum value of the lower safety limit of all instances, and L_max is the minimum value of the upper safety limit of all instances, representing the weakest link in the entire cluster.

[0053] Default rate: refers to the proportion of requests that fail to be processed within a statistical time window due to exceeding service capacity (such as timeout, rejection, or rate limiting). It is one of the core indicators for measuring service quality.

[0054] Budget token: A data structure carried in the RPC request header to transmit resource budget tokens.

[0055] RPC (Remote Procedure Call) request: It is one of the core mechanisms for inter-service communication in a distributed system. It refers to a function call request initiated by a client to a remote server over the network. The request includes information such as the method name and parameters. After the server executes the corresponding logic, it returns the result. The whole process appears to the caller as "as a local call".

[0056] KV storage: a type of storage that uses KV (key-value) technology.<Key,Value> For a storage system that organizes data in a formal way, the Key is a unique identifier, usually a string or a byte array; the Value is data of any type (such as a string, JSON, binary, serialized object, etc.).

[0057] Tenant: In a multi-tenancy architecture, a tenant is an independent logical unit that uses the same software instance or service platform. It usually represents an application or business entity, and its data, configuration, and running status are isolated from other tenants.

[0058] In large-scale distributed service architectures, rate limiting is a key fundamental mechanism for ensuring high availability and stability. When a system faces sudden traffic surges or bottlenecks in downstream dependencies, effective rate limiting can prevent cascading failures caused by overload.

[0059] Currently, mainstream distributed rate limiting solutions can be mainly divided into two categories:

[0060] The first type is based on centralized global counters, such as using Redis or a dedicated quota service to uniformly manage global token buckets or request quotas. The advantage of this type of solution is that it can achieve precise global total control, but it has significant disadvantages: the centralized service itself constitutes a single point of failure (SPOF) and a performance bottleneck, and the availability and scalability of the centralized service itself directly affect the stability of the entire system;

[0061] The second type is a scheme where each service instance counts independently and synchronizes periodically. Each node maintains its local rate-limiting threshold and exchanges information with other nodes through protocols such as Gossip, eventually converging towards a global limit. This type of scheme is decentralized and has good fault tolerance and scalability. However, the consistency model of this type of scheme usually adopts an eventual consistency model. In common distributed environments with network partitions, message delays, or out-of-order delivery, before synchronization is complete, the sum of the local decisions of each partition or node may exceed the preset global limit in a short period of time. This is an unacceptable security risk for scenarios that require strict control of resource consumption.

[0062] Furthermore, in multi-tenant or multi-service priority application scenarios, existing solutions often employ static proportional partitioning or fixed weight allocation strategies, making it difficult to dynamically allocate resources fairly and efficiently based on the actual load and importance of each tenant. Simultaneously, in complex microservice call chains, requests traverse multiple services, but there is a lack of an effective mechanism to pass end-to-end latency budgets or concurrency credits along the call chain. This leads to a disconnect between upstream service rate limiting decisions and downstream service actual processing capabilities, making it difficult to achieve globally optimal collaborative degradation and congestion control.

[0063] To this end, this embodiment proposes a distributed rate limiting method, device, system, and storage medium, which achieves distributed rate limiting by using interval intersection to ensure that the global upper limit cannot be exceeded, tenant resource allocation, and end-to-end collaborative control.

[0064] Please refer to Figure 1 This embodiment discloses a distributed rate limiting method, including steps S100 to S300. It should be noted that the steps in this embodiment are numbered only for ease of review and understanding, and not to limit the execution order of the steps.

[0065] This embodiment of the distributed rate limiting method is applied to service instances in a distributed system. The distributed system has multiple nodes, each node deploys a corresponding service instance, and each service instance deploys a distributed rate limiting device. Please refer to [link / reference needed]. Figure 2The distributed rate limiting device includes an interval management module 110, a CRDT aggregation module 120, a quota allocation module 130, and a budget transmission module 140. In some application examples, the distributed rate limiting device also includes a loan management module 150 and a consistency compensation module 160. Different modules can interact through a lightweight message bus (such as the Gossip protocol or a shared KV store).

[0066] In a distributed system, the distributed rate limiting devices on each service instance perform the same function. The following example, using one of the service instances as the execution subject, details the content of each step:

[0067] S100: Periodically generate concurrent suggestion intervals based on local running status and broadcast concurrent suggestion intervals to other service instances in the distributed system;

[0068] For example, in a distributed system, the performance of different nodes may vary significantly. This difference is typically reflected in multiple operational status metrics, such as hardware configuration, resource scheduling strategies, load fluctuations, or external interference. These metrics collectively determine the actual concurrent request processing capacity of a service instance deployed on a particular node at a specific time. Due to differences in local operational status, the number of concurrent requests that each service instance can efficiently handle within the same time period often varies. If a static or uniform concurrency control strategy is adopted, some high-performance nodes may experience resource idleness, while low-performance nodes may suffer increased response latency or even service degradation due to overload.

[0069] The interval management module 110 of the service instance continuously monitors the key operating status parameters of the local node and periodically generates a concurrency suggestion interval [l_i, u_i] based on the local operating status. This concurrency suggestion interval is not fixed but is dynamically updated periodically as the local operating status changes, such as node load, resource availability, and system health, ensuring that the concurrency suggestion interval reflects the actual processing capacity of the service instance. To enable the entire distributed system to collaboratively optimize global load distribution, each service instance actively broadcasts its new concurrency suggestion interval to other service instances in the distributed system after generation. This allows all participating service instances to promptly understand each other's current concurrency processing capacity boundaries, providing a basis for subsequent request routing, task distribution, or collaborative decisions. An example of the broadcast message body format is shown below.<instanceId,epoch,[l_i,u_i],timestamp,signature> `instanceId` is the unique identifier of the service instance; `epoch` is the generation number, used to handle state updates after the service instance restarts; `timestamp` is the broadcast reporting time; and `signature` is a tamper-proof digital signature.

[0070] It is worth mentioning that the concurrent suggestion interval in this embodiment is based on a point-to-point broadcast method, rather than relying on a centralized coordination node or centralized service instance to collect, calculate and distribute concurrent control policies. This can be used to implement a decentralized design and avoid single point of failure or performance bottleneck of the central node.

[0071] S200: Receive concurrent suggestion intervals generated by itself and from other service instances, and perform an intersection operation on all received concurrent suggestion intervals to obtain a global feasible interval. The upper limit of the global feasible interval is the minimum value of the upper limits of all concurrent suggestion intervals, and the lower limit of the global feasible interval is the maximum value of the lower limits of all concurrent suggestion intervals.

[0072] For example, the CRDT aggregation module 120 in the service instance is used to receive concurrent suggestion intervals generated by the service instance itself and sent by other service instances, and to perform an intersection operation on the received concurrent suggestion intervals to determine a globally feasible interval. In this process, the core operation is the merge operation, using symbolic representation. Representation. Assuming there are three concurrent suggestion intervals S_a = [l_i_a, u_i_a], S_b = [l_i_b, u_i_b], and S_c = [l_i_c, u_i_c], then the globally feasible interval Sg = [L_min, L_max] obtained through the merging operation can be represented as Sg = S_a S_b S_c. Specifically, the merge operation for any two intervals S_a and S_b is defined as S_a S_b = [max(l_i_a, l_i_b), min(u_i_a, u_i_b)], where max represents the maximum value operation and min is used to find the minimum value. Therefore, the unification law of the unification operation satisfies the following property:

[0073] Idempotency: When any concurrent suggestion interval is merged with itself, the result remains unchanged, i.e., S S = S;

[0074] Commutative law: Regardless of the merging order, the result remains consistent, i.e., S_a S_b = S_b S_a;

[0075] Associative law: Regardless of how the grouping method of the concurrent proposal intervals changes, the final merging result will not be affected, i.e., (S_a S_b) S_c = S_a (S_b S_c);

[0076] The upper bound is monotonically non-increasing: This merge operation takes the maximum value of the upper bound of each concurrent suggestion interval. Therefore, for any merge sequence, the upper bound of the global feasible interval (L_max) obtained by merging must be less than or equal to the upper bound of all concurrent suggestion intervals involved in the merge. In other words, in the case of any network partition, message out-of-order, or repeated sending, the upper bound of the global feasible interval (L_max) will not be incorrectly increased. That is, it has the characteristics of monotonically non-increasing upper bound and upper bound cannot be exceeded. A safety invariant constraint chain can be obtained, which is beneficial to ensure the safety of rate limiting.

[0077] Arbitrary order consistent convergence: Based on the commutative and associative laws, the merging operation forms a mathematical semi-lattice structure. According to CRDT theory, all replicas (service instances) will eventually converge to the same state, regardless of the message passing order.

[0078] To further enhance the robustness and security of the system, the issue of outdated information leading to higher upper limits in the global feasible range due to potential node clock errors or malicious attacks can be addressed by configuring an anti-elevation threshold. For example, for received broadcast messages, processing can be only performed if the message's epoch is greater than or equal to the epoch recorded by the current service instance; alternatively, update rules for the global feasible range can be configured such that the lower limit L_min can only increase, while the upper limit L_max can only decrease. During the merging operation performed by the CRDT aggregation module 120, a timeout threshold can be configured. For instance, if a service instance has not broadcast a new concurrent suggestion range for N consecutive periods, the previously broadcast concurrent suggestion range can be temporarily ignored during merging. This prevents the upper limit of the global feasible range from being unnecessarily tightened due to node disconnection, helping to maintain the system's stability and reliability in complex network environments.

[0079] S300. Based on the globally feasible range and local dynamic weights, determine the local rate limiting threshold;

[0080] For example, each service instance is configured with a corresponding local dynamic weight, which comprehensively reflects the current health status and actual carrying capacity of the service instance. The local dynamic weight is not statically set, but supports real-time updates or dynamic adjustments at fixed periods, thereby enabling sensitive responses to changes in the service instance's operating status and ensuring that the weight value accurately represents the service instance's service capabilities.

[0081] The service instance quota allocation module 130 determines the local rate limiting threshold based on the global feasible range and local dynamic weights. This allows different service instances, even if they are in the same system, to obtain differentiated rate limiting quotas according to their actual resource status and load capacity. This avoids waste or overload caused by uniform rate limiting quotas. For example, service instances with stronger performance and lighter load can be assigned higher local rate limiting thresholds, while service instances with higher resource thresholds or lower health will automatically receive lower local rate limiting thresholds, thereby achieving fine-grained and adaptive rate limiting management.

[0082] Therefore, this embodiment periodically generates a concurrency suggestion interval based on the local running status and broadcasts it to other service instances. Each service instance receives the concurrency suggestion interval and performs an intersection operation to determine the globally feasible interval, ensuring that the upper limit is monotonically non-increasing and cannot be exceeded, thus obtaining a safe invariant constraint chain. No central coordinator or global synchronization clock is required. Under any network conditions, it ensures that the global concurrency does not exceed the safe upper limit. Based on the globally feasible interval and local dynamic weights, the local rate limiting threshold is determined, realizing fine-grained and adaptive rate limiting management.

[0083] In some application examples, step S100, which periodically generates concurrency suggestion intervals based on the local runtime state, includes:

[0084] Obtain local runtime status, which includes CPU utilization, available memory ratio, GC activity metrics, network I / O metrics, number of requests being processed, request queue length, and some or all of the success rate, latency rate, and default rate based on historical statistics.

[0085] Based on preset rules or interval generation models, concurrent suggested intervals are generated according to the local running status.

[0086] For example, in a distributed system, the performance of different nodes can vary significantly. This difference stems not only from the heterogeneity of hardware configurations (such as CPU model, memory capacity, network bandwidth, etc.) but also from multi-dimensional dynamic state metrics during runtime. Specifically, local runtime state includes, but is not limited to: CPU utilization, available memory ratio, garbage collection (GC) activity metrics (such as frequency and duration), network I / O metrics (such as throughput and latency), the number of concurrent requests currently being processed, request queue length, and service quality metrics derived from historical observations, such as request success rate, latency rate, and default rate. These metrics collectively determine the actual concurrent request processing capacity that a service instance can safely and efficiently handle in its current state.

[0087] The service instance's interval management module 110 continuously collects and aggregates local operational status data, and derives a reasonable concurrency recommendation interval based on preset rules or interval generation models (such as the classic queuing theory formula Little's Law, i.e., L=λW, where L represents the average number of requests in the system, λ is the arrival rate, and W is the average response time). The upper limit u_i of the concurrency recommendation interval usually corresponds to the maximum sustainable concurrent load of the service instance without causing performance degradation or violating SLO; while the lower limit l_i represents the minimum concurrency guarantee level required to maintain core business functions, ensuring that even when resources are scarce, the critical path can still obtain basic processing capacity.

[0088] For example, when CPU utilization consistently exceeds the empirical threshold of 80%, the interval management module 110 can use the current number of concurrent requests being processed as the upper limit u_i of the concurrency suggestion interval to prevent further overload; simultaneously, it sets the lower limit l_i to a predefined reserved value (such as the minimum number of threads or connections required to support health checks and basic API calls). To avoid frequent and drastic fluctuations in the concurrency suggestion interval due to momentary jitter (such as brief GC pauses or sudden traffic spikes), which could lead to oscillations in upstream call strategies, the interval management module 110 typically smooths the raw monitoring data of the local running status. Common smoothing methods include sliding window average or exponentially weighted moving average (EWMA), which can effectively filter out noise and make the generated concurrency suggestion interval more stable and forward-looking.

[0089] In this way, the interval management module 110 not only achieves accurate modeling of local service capabilities, but also provides a reliable data foundation for subsequent decentralized collaborative rate limiting, load balancing and elastic scheduling, thereby supporting an adaptive and highly available service governance system in a heterogeneous and dynamic distributed environment.

[0090] In some application examples, step S300, based on the globally feasible range and local dynamic weights, determines the local rate limiting threshold, including:

[0091] The first concurrency threshold is determined by rounding down based on the weight ratio of the local dynamic weight and the upper limit of the global feasible interval.

[0092] The first tail difference margin is determined based on the first concurrency threshold of all service instances and the upper limit of the global feasible interval;

[0093] The first tail difference is allocated to the multiple service instances with the highest dynamic weights by increasing the concurrency threshold by one unit for each service instance, thus obtaining the local rate limiting threshold.

[0094] For example, the core objective of the service instance quota allocation module 130 is to allocate a local rate limiting threshold L_i to each service instance i under the hard constraint of the global feasible interval upper limit value L_max, which reflects both resource fairness and processing efficiency. The allocation mechanism must simultaneously satisfy two key properties: First, the sum of the local rate limiting thresholds of all service instances (∑L_i) must not exceed the global upper limit, that is, satisfy the constraint condition ∑L_i≤L_max; Second, when the global feasible interval upper limit L_max changes (especially decreases), the local rate limiting threshold of each service instance has monotonically non-increasing property, that is, there will be no abnormal increase in local thresholds due to global quota shrinkage, thereby ensuring the security and predictability of the rate limiting strategy.

[0095] To achieve the above objectives, the quota allocation algorithm employs a two-stage allocation strategy combining weighted flooding and tail difference recovery. This strategy maximizes the utilization of available resources and reflects the priority distinctions resulting from weight differences, while ensuring that the total quota does not exceed the limit. The specific allocation process is as follows:

[0096] Phase 1: Preliminary Allocation

[0097] The quota for each service instance is calculated based on the weight ratio of the local dynamic weights (w_i) of all service instances. Let the sum of the weights of all service instances be ∑w_j (where j represents the j-th service instance), then the relative proportion of the local dynamic weight of service instance i in the local dynamic weights of all service instances is w_i / ∑w_j. Based on this proportion, the theoretical concurrent quota (i.e., the first concurrent threshold) of the service instance is initially calculated as L_i'=floor((w_i / ∑w_j)*L_max), where floor is used to represent the floor operation, ensuring that the initial allocation sum will not exceed the upper limit of the globally feasible interval L_max.

[0098] Phase Two: Calculation and Recovery of the First Tail Difference Balance

[0099] The initial allocation process involves rounding down, which results in a first tail difference margin. This first tail difference margin is equal to the difference between the upper limit of the global feasible interval (L_max) and the sum of the first concurrency thresholds of all service instances (∑L_i'), i.e., the first tail difference margin R = L_max - ∑L_i'.

[0100] The first tail difference margin R represents the number of integer unit quotas that have not been allocated but can be legally used (0 ≤ R < N, where N is the total number of service instances). To improve resource utilization and strengthen the priority guarantee for high-weight instances, the quota allocation module 130 of the service instance performs tail difference recovery: sort all service instances in descending order according to their local dynamic weight w_i, and sequentially add 1 unit of concurrent threshold to each of the top R service instances with the highest weight. The finally obtained local flow-limiting threshold is L_i = L_i’ + δ_i, where δ_i = 1 if and only if service instance i is among the top R in the weight sorting, otherwise δ_i = 0. In this way, it can be ensured that high-weight service instances can still obtain better quota tilting when resources are紧张, and at the same time, fully utilize all L_max quotas.

[0101] It should be noted that when the upper limit value L_max of the global feasible interval decreases due to load changes or the update of the CRDT merge result, since the calculation of the first concurrent threshold L_i’ directly depends on L_max as a multiplier factor, the preliminary allocation value L_i’ will necessarily show a non-increasing trend. Correspondingly, the first tail difference margin R also non-increases (or even may decrease), resulting in that the number of additional units that can be allocated in the tail difference recovery stage will not increase. Therefore, whether it is preliminary allocation or secondary allocation, the finally output local flow-limiting threshold L_i will not increase with the decrease of L_max, strictly satisfying non-increasing monotonicity. This characteristic is crucial for preventing "reverse incentives" or "quota oscillations" in the flow-limiting strategy when the system pressure increases, and is beneficial to supporting the stable operation and safe load reduction of the distributed system in a dynamic environment.

[0102] Among them, in some application instances, the local dynamic weight is obtained by weighted summation based on the reciprocal of the CPU utilization rate (1 / CPU_util_i), the available memory ratio (mem_free_i), and the complement of the default rate (1-violation_rate_i). For example, w_i = a·(1 / CPU_util_i) + b·mem_free_i + c·(1-violation_rate_i), where a, b, c are configurable non-negative coefficients, CPU_util_i is used to represent the CPU utilization rate of the node where service instance i is located, mem_free_i is used to represent the available memory ratio of the node where service instance i is located, violation_rate_i is used to represent the default rate of service instance i, and the complement of the default rate is used to represent the difference between the value 1 and the default rate. The higher the weight when the CPU is more idle, the more sufficient the memory, and the lower the default rate.

[0103] In some application examples, after step S300, determining the local flow-limiting threshold based on the global feasible interval and the local dynamic weight, it further includes: allocating tenant-level flow-limiting quotas to the affiliated tenant under the constraint of the local flow-limiting threshold.

[0104] For example, after determining the local rate limiting threshold, the quota allocation module 130 also allocates tenant-level rate limiting quotas to the tenant to which the service instance belongs, under the constraint of the local rate limiting threshold, which helps to improve the granularity of rate limiting management.

[0105] In some application examples, tenant-level rate limiting quotas are allocated to the tenant under the constraint of local rate limiting thresholds, including:

[0106] The second concurrency threshold is determined by rounding down based on the tenant dynamic weight of each tenant and the local rate limiting threshold.

[0107] The second tail difference margin is determined based on the second concurrency threshold of all tenants and the local rate limiting threshold;

[0108] The second tail difference surplus is allocated to the tenants with the highest dynamic weights by increasing the concurrency threshold by one unit for each tenant, thus obtaining the tenant-level rate limiting quota.

[0109] For example, after completing the allocation of the local rate limiting threshold L_i at the service instance level, a second-level quota allocation can be performed, namely tenant-level quota management for multi-tenant scenarios. This allows the total processing capacity of each service instance (i.e., the local rate limiting threshold L_i) to be reasonably allocated to its multiple tenants, thereby achieving resource isolation, service quality assurance, and differentiated support for business priorities among different tenants.

[0110] Specifically, for service instance i, a tenant-level rate limiting quota L_{t, i} is maintained for each tenant t, representing the maximum number of concurrent requests that tenant t can use on service instance i. The sum of the quotas of all tenants on service instance i (∑L_{t, i}) satisfies the constraint of the local rate limiting threshold at the service instance level, i.e., ∑L_{t, i} ≤ L_i, which helps ensure that the service availability of other tenants is not affected by a sudden high load of a single tenant on a single service instance. Globally, the resource quota of tenant t in the distributed system is represented by a vector vec_L_t = [L_{t, 1}, L_{t, 2}, ..., L_{t, n}], where n is the total number of service instances in the distributed system. Although the requests of tenant t may be routed to multiple service instances, the total quota of tenant t in the distributed system (∑L_{t, i}) is still indirectly subject to the macro-constraint of the upper limit of the globally feasible interval Lmax, i.e., ∑L_{t, i} ≤ L_max.

[0111] At the implementation level, each service instance can independently perform tenant-level weighted allocation and tail difference recovery. The specific steps include: determining the second concurrency threshold by rounding down based on the tenant dynamic weight and local rate limiting threshold of each tenant; determining the second tail difference margin based on the second concurrency threshold of all tenants and the local rate limiting threshold; and allocating the second tail difference margin to the multiple tenants with the highest dynamic weight one by one, in order to increase the concurrency threshold by one unit for each tenant, to obtain the tenant-level rate limiting quota.

[0112] The dynamic weight of a tenant can be solved by minimizing the weighted default rate function. Specifically, the function is defined as min∑α_t · violation_rate_t, where α_t represents the dynamic weight of the tenant (which can be the priority or business value weight of tenant t), and violation_rate_t represents the default rate of tenant t. By dynamically adjusting the dynamic weight of tenants, the default risk of high-weight tenants is reduced, thereby maximizing overall business benefits or user experience.

[0113] To prevent tenant-level rate limiting quotas from being frequently and drastically adjusted due to instantaneous metric fluctuations (leading to oscillations in upstream call strategies or cache invalidation), rate limits can be imposed on changes to tenant-level rate limiting quotas. For example, it can be stipulated that within a single scheduling cycle, the change in the tenant-level rate limiting quota L_{t, i} of any tenant must not exceed ±10% of its current value. This can improve system stability while giving downstream components sufficient adaptation time.

[0114] In some application examples, step S300, which determines the local rate limiting threshold based on the globally feasible range and local dynamic weights, further includes:

[0115] Upon receiving a service call request carrying a budget token, the budget token is parsed, and a response decision is made based on the local rate limiting threshold, the concurrent credit value obtained from the budget token, and the remaining latency in milliseconds.

[0116] For example, in a Remote Procedure Call (RPC) architecture, the budget delivery module 140 for a service instance includes the injection, parsing, and use of budget tokens. By embedding lightweight resource budget information into the service call chain, the budget delivery module 140 enables upstream and downstream services to make collaborative rate limiting and degradation decisions based on a unified context, thereby improving the stability and response efficiency of the entire distributed system.

[0117] In the specific implementation, please refer to Figure 3When an upstream service instance needs to initiate an RPC request to a downstream service instance, the budget delivery module 140 injects a structured budget token into the request's metadata. The budget token is typically embedded as a field in the request header, for example, written into a custom header (such as X-Resource-Budget) in an HTTP-based RPC, or passed via a metadata mechanism in a gRPC call. The budget token contains two key fields:

[0118] remaining_latency_ms: The number of milliseconds of remaining latency, representing the remaining latency budget (in milliseconds) from the current call point to the maximum end-to-end latency acceptable to the end user that can be used for subsequent processing.

[0119] concurrency_credit: Concurrency credit value, representing the number of remaining concurrency licenses in the current call chain, used to control the number of requests that can be executed simultaneously on the chain.

[0120] In a typical application example, an upstream service instance might configure its budget token as: remaining_latency_ms = 120, concurrency_credit = 1, indicating that the request can tolerate a maximum processing latency of 120 milliseconds and carries only one unit of concurrent credit.

[0121] When a downstream service instance receives an RPC request carrying a budget token, its budget delivery module 140 parses the token, extracts the remaining latency in milliseconds and the specific values ​​of the concurrency credit, and makes a response decision based on the local rate limiting threshold.

[0122] Rate limiting judgment: If the number of concurrent requests currently being processed by the local service instance has reached or exceeded the dynamically calculated local rate limiting threshold, the expected number of concurrent requests will exceed the local rate limiting threshold, or the received concurrent credit value is less than or equal to zero, then the request will be rejected immediately, and a standard rate limiting error response, such as HTTP status code 429 TooMany Requests, will be returned to the upstream to avoid further aggravating the system load.

[0123] Latency-sensitive degradation: If the remaining latency obtained from parsing is lower than the system's preset minimum effective processing threshold (i.e., insufficient time to complete the full business logic), a degradation strategy is triggered. Degradation methods may include: returning an approximate result from the cache (even if slightly outdated, but semantically usable), skipping non-core validation or asynchronous processing steps, or directly returning a simplified response to ensure delivery is completed within a limited latency budget.

[0124] Normal Acceptance and Credit Deduction: If none of the above conditions are triggered, the service instance accepts the request and enters the normal processing flow. Before formally executing the business logic, the concurrent credit value in the budget token is decremented by one (i.e., one concurrent credit is consumed) to reflect that the request has occupied a concurrent slot.

[0125] Furthermore, to achieve a closed-loop feedback mechanism, when a downstream service instance detects a continuous increase in its load (such as a significant increase in request queuing time or CPU utilization approaching its limit), the downstream service instance's budget delivery module 140 can append a throttle suggestion field (throttle_hint) to the RPC response header returned to the upstream. This field serves as a lightweight feedback signal, suggesting that the upstream caller proactively reduce the request distribution weight to that downstream service instance, decrease the call frequency, or adjust the routing strategy, thereby achieving preventative traffic regulation before congestion occurs.

[0126] It's worth noting that in complex distributed call topologies, any service instance may simultaneously play a dual role: downstream for its caller and upstream for the services it depends on. By passing and updating budget tokens layer by layer in the RPC call chain, rate limiting and degradation decisions are no longer limited to the local perspective of a single node, but rather transform rate limiting decisions from isolated actions of a single node into coordinated actions across the entire call chain. The actual processing pressure, latency margin, and concurrency capacity of downstream services can be "transmitted" to upstream in real time with low overhead, allowing traffic regulation to intervene earlier closer to the source. This improves the ability to adapt to sudden loads, effectively suppresses the deterioration of tail latency, and significantly reduces the risk of cascading failures caused by local overload, thereby ensuring high availability and consistent user experience for the overall service.

[0127] In some application examples, distributed rate limiting methods also include:

[0128] If the preset borrowing conditions are met, the current limit quota of the lending tenant is allocated to the debit tenant, and the lending information is recorded, including the lending tenant, the debit tenant, the lending quota, the lending validity period, and the reduction factor.

[0129] When the loan term expires, or when the default rate of the lending tenant exceeds a preset threshold and continues for a preset period of time, the loan quota will be gradually returned from the borrowing tenant to the lending tenant over multiple periods based on the reduction factor.

[0130] For example, in order to cope with sudden traffic surges, the service instance is configured with a lending management module 150, which allows tenants with temporarily idle resources (lending tenants) to lend their traffic-limiting quotas to tenants who urgently need resources (borrowing tenants), thereby achieving flexible reallocation and efficient utilization of resources within the distributed system without exceeding the global feasible range limit L_max.

[0131] To ensure system stability and tenant fairness, borrowing and lending of rate-limited quotas must be restricted by borrowing conditions. In a specific application example, the borrowing conditions are configured as follows:

[0132] The tenant's credit score is within the limit;

[0133] The cooling-off period has passed since the last loan, to prevent frequent borrowing from causing strategy instability or abuse;

[0134] The current resource processing is in a globally constrained state, meaning that the sum of the local rate limiting thresholds of all service instances has reached or is close to the upper limit of the globally feasible range.

[0135] At this point, the rate-limiting quota for lending tenants can be allocated to debit tenants, and the lending information can be recorded. For example, a lending record can be recorded as follows:<lender_tenant_id,borrower_tenant_id,amount,epoch,ttl,decay_factor,signature> The loan management module 150 uses `lender_tenant_id` to represent the unique identifier of the lending tenant, `borrower_tenant_id` to represent the unique identifier of the debit tenant, `amount` to identify the lending quota, `epoch` to represent the generation number, `ttl` to represent the loan validity period, `decay_factor` to represent the decay factor, and `signature` to represent the digital signature. When the loan validity period expires, or when the default rate of the lending tenant exceeds a preset threshold and continues for a preset duration, the lending quota is gradually returned from the debit tenant to the lending tenant over multiple cycles based on the decay factor. The recovery process is not instantaneous, but rather gradually returns the rate-limited quota over several cycles according to the rate of decay factor, effectively avoiding the shock caused by the instantaneous withdrawal of rate-limited quotas. By introducing a controlled, traceable, and gradually recoverable quota transfer mechanism between tenants, the lending management module 150 significantly improves the adaptive capability and resource utilization efficiency of the distributed system in response to sudden traffic surges, while ensuring global resource constraints and tenant isolation. This is conducive to building a highly elastic and highly available multi-tenant service architecture.

[0136] In some application examples, distributed rate limiting methods also include:

[0137] After the network partition failure is recovered, the global feasible range is updated, and the local rate limiting threshold is updated in multiple cycles based on the updated global feasible range. In each cycle, the change of the local rate limiting threshold is less than or equal to the preset proportion of the current local rate limiting threshold.

[0138] For example, to address consistency issues caused by network partitioning failures, the service instance is also configured with a consistency compensation module 160. Please refer to... Figure 4 During a network partition, service instances within different subnets may independently generate concurrent suggestion intervals based on local views and merge them via decentralized broadcast. However, due to the lack of cross-partition communication, the final converged global feasible intervals of each subnet may differ. Once the network recovers and the partitions reconnect, the mathematical convergence of CRDT ensures that all replicas (service instances) eventually reach consensus. However, this new, unified global feasible interval may have significant jumps compared to the old value used by any subnet during the partition. For example, the upper limit L_max of the global feasible interval may suddenly increase due to merging the states of more low-load nodes, or it may suddenly tighten due to the inclusion of nodes with high default rates. If each service instance immediately resets its local rate limiting threshold L_i based on the updated upper limit L_max of the global feasible interval, it is highly likely to cause a sudden increase in capacity or backpressure oscillations. To avoid such risks, the consistency compensation module 160 adopts a multi-period gradient update strategy to gradually adjust the local rate limiting threshold. For example, after a network partition failure is recovered, the global feasible range is updated, and the local rate limiting threshold is updated in multiple cycles based on the updated global feasible range. The change of the local rate limiting threshold in a single cycle is less than or equal to a preset proportion of the current local rate limiting threshold (e.g., not exceeding ±10% of the current value). This can effectively suppress the traffic surge in the early stage of network partition recovery, avoid backpressure oscillations such as backend overload or upstream retry storms that may be caused by a sudden increase in capacity, and achieve a smooth transition to a steady state.

[0139] Based on the above description, the distributed rate limiting method in this embodiment has the following beneficial effects:

[0140] 1. Consistency that the global upper limit cannot be surpassed

[0141] Intersection merging operation: By employing an interval intersection merging operation based on CRDT (Conflict-Free Replicated Data Type), this embodiment ensures the monotonically non-increasing upper limit value L_max of the globally feasible interval and consistent convergence in any order. This means that regardless of how network partitioning affects the state update order of each node, all service instances will eventually converge to a commonly accepted safe upper limit value, resulting in a safe invariant constraint chain, thus avoiding service crashes or performance degradation caused by exceeding concurrency limits.

[0142] Decentralized design: It eliminates the need for a central coordinator or global synchronization clock, reducing the risk of single points of failure and improving the scalability and flexibility of the system.

[0143] 2. Excellent partition tolerance

[0144] Independent convergence capability: During network partitioning, each subnet can independently perform CRDT merging and converge to a locally feasible interval, which will not exceed the global safety limit. This ensures that each subnet can maintain reasonable load control even in network partitioning.

[0145] A natural and smooth unification process: When the network recovers, the state of each partition can be naturally and smoothly unified through standard CRDT merging operations, without the need for complex conflict resolution processes, simplifying the system recovery logic and reducing the need for human intervention.

[0146] 3. Dynamic weighted fairness

[0147] Vectorized quota allocation: Under the premise of ensuring global security, this embodiment achieves fair and efficient resource allocation for tenants with different priorities and service loads through a dynamic weight adjustment mechanism. Specifically, the quota allocation for each tenant on each service instance is determined by calculating the tenant's dynamic weight (such as based on CPU utilization, available memory ratio, default rate, etc.).

[0148] Minimize weighted default rate: By optimizing the objective function (e.g., minimizing the weighted default rate), the quality of service (QoS) and resource utilization in a multi-tenant environment are improved, enabling high-priority services to be better guaranteed while taking into account the basic needs of low-priority services.

[0149] 4. End-to-end coordinated control

[0150] Budget token passing: By passing budget tokens (containing remaining latency milliseconds and concurrent credits) along the RPC call chain, this embodiment transforms rate limiting decisions from isolated actions of a single node into coordinated actions of the entire call chain. The actual pressure on downstream services can be fed back to upstream in real time, thereby achieving earlier and more accurate congestion warnings and traffic regulation.

[0151] Reduce tail latency and cascading congestion risk: The end-to-end collaborative control mechanism effectively reduces end-to-end tail latency and reduces the risk of cascading congestion caused by local overload, thereby improving the overall system response speed and stability.

[0152] 5. Balancing elasticity and steady-state performance

[0153] Credit Limit Lending Ledger Mechanism: To cope with short-term surges in traffic, this embodiment introduces a credit limit lending ledger mechanism, allowing tenants with temporarily idle resources to lend their traffic-limiting quotas to tenants in urgent need of resources. This mechanism provides the system with a flexible buffer, helping to alleviate instantaneous high load pressure.

[0154] Automatic recycling mechanism: The accompanying automatic recycling mechanism ensures that the system can return to a weight-based steady-state order during long-term operation, which guarantees both short-term flexibility and long-term stability.

[0155] 6. Clear project implementation details

[0156] Modular design: The various functional modules (such as interval management module 110, CRDT aggregation module 120, quota allocation module 130, budget transmission module 140, loan management module 150, consistency compensation module 160, etc.), data structures (such as intervals and vectors), and interaction protocols (such as budget tokens) proposed in this embodiment are all clearly defined, making them easy to understand and implement.

[0157] Easy to integrate: These modules and protocols are well-suited for integration into existing microservice frameworks or service meshes, are highly implementable, and can be quickly deployed and applied to production environments to improve the overall performance and reliability of the system.

[0158] Please refer to Figure 2 Based on the same technical concept as the above-mentioned distributed rate limiting method, this embodiment provides a distributed rate limiting device applied to a service instance of a distributed system, including:

[0159] The interval management module 110 is used to periodically generate concurrent suggestion intervals based on the local running status and broadcast the concurrent suggestion intervals to other service instances in the distributed system.

[0160] The CRDT aggregation module 120 is used to receive concurrent suggestion intervals generated by itself and from other service instances, and to perform an intersection operation on all received concurrent suggestion intervals to obtain a global feasible interval. The upper limit of the global feasible interval is the minimum value of the upper limit of all concurrent suggestion intervals, and the lower limit of the global feasible interval is the maximum value of the lower limit of all concurrent suggestion intervals.

[0161] The quota allocation module 130 is used to determine the local rate limiting threshold based on the globally feasible range and the local dynamic weight.

[0162] In some application examples, the quota allocation module 130 is also used to allocate tenant-level rate limiting quotas to its tenants under the constraint of local rate limiting thresholds.

[0163] In some application examples, the distributed rate limiting device also includes a budget delivery module 140, which is used to parse the budget token upon receiving a service call request carrying a budget token and make a response decision based on the local rate limiting threshold and the concurrency credit value and remaining latency milliseconds obtained from the budget token.

[0164] In some application examples, the distributed rate limiting device also includes a lending management module 150 and a consistency compensation module 160. Different modules can interact via a lightweight message bus (such as the Gossip protocol or a shared key-value store).

[0165] The inventive concept of this distributed rate limiting device embodiment is the same as that of the distributed rate limiting method embodiment described above. For any content not covered in this distributed rate limiting device embodiment, please refer to the distributed rate limiting method embodiment described above. It will not be repeated here.

[0166] The system periodically generates concurrency suggestion intervals based on local runtime status and broadcasts them to other service instances. Each service instance receives the concurrency suggestion interval and performs an intersection operation to determine the globally feasible interval, ensuring that the upper limit is monotonically non-increasing and cannot be exceeded, thus obtaining a safe and invariant constraint chain. This eliminates the need for a central coordinator or a global synchronization clock, ensuring that the global concurrency does not exceed the safe upper limit under any network conditions. Based on the globally feasible interval and local dynamic weights, the local rate limiting threshold is determined, achieving fine-grained and adaptive rate limiting management.

[0167] Please refer to Figure 5 This embodiment also provides a distributed rate limiting system, including a processor 210 and a memory 220. The memory 220 stores a computer program, and the processor 210 executes the computer program to implement the distributed rate limiting method described above. The distributed rate limiting method is as described above and will not be repeated here. It periodically generates a concurrency suggestion interval based on the local running state and broadcasts it to other service instances. Each service instance receives the concurrency suggestion interval and performs an intersection operation to determine the globally feasible interval, ensuring that the upper limit is monotonically non-increasing and cannot be exceeded, thus obtaining a safe invariant constraint chain. It does not require a central coordinator or a global synchronization clock, ensuring that the global concurrency does not exceed the safe upper limit under any network conditions. Based on the globally feasible interval and local dynamic weights, the local rate limiting threshold is determined, achieving fine-grained and adaptive rate limiting management.

[0168] This embodiment also provides a storage medium storing a computer program. When the computer program is run, it implements the distributed rate limiting method described above. The distributed rate limiting method is as described above and will not be repeated here. It periodically generates concurrency suggestion intervals based on the local runtime state and broadcasts them to other service instances. Each service instance receives the concurrency suggestion interval and performs an intersection operation to determine the globally feasible interval, ensuring that the upper limit is monotonically non-increasing and cannot be exceeded, thus obtaining a safe and invariant constraint chain. This eliminates the need for a central coordinator or a global synchronization clock, ensuring that the global concurrency does not exceed the safe upper limit under any network conditions. Based on the globally feasible interval and local dynamic weights, the local rate limiting threshold is determined, achieving fine-grained and adaptive rate limiting management.

[0169] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A distributed throttling method applied to a service instance of a distributed system, characterized in that, The method comprises the following steps: Periodically generating a concurrent suggestion interval based on a local running state and broadcasting the concurrent suggestion interval to other service instances of the distributed system, wherein the concurrent suggestion interval is used to represent the lower limit and upper limit of the number of concurrent requests that can be safely processed by a service instance within an evaluation period based on its own health status and processing capacity; Receiving the concurrent suggestion interval generated by itself and from other service instances, and performing intersection operation on all received concurrent suggestion intervals to obtain a global feasible interval, wherein the upper limit of the global feasible interval is the minimum value of the upper limits of all concurrent suggestion intervals, and the lower limit of the global feasible interval is the maximum value of the lower limits of all concurrent suggestion intervals; Determining a local throttling threshold based on the global feasible interval and a local dynamic weight; The step of determining the local throttling threshold based on the global feasible interval and the local dynamic weight comprises the following steps: Determining a first concurrent threshold by taking the integer part according to the weight proportion of the local dynamic weight and the upper limit value of the global feasible interval; Determining a first tail difference margin according to the difference between the upper limit value of the global feasible interval and the sum of the first concurrent thresholds of all service instances; Allocating the first tail difference margin to multiple service instances with the highest dynamic weight in a manner of increasing a unit concurrent threshold for each service instance to obtain the local throttling threshold.

2. The distributed current limiting method of claim 1, wherein, The step of periodically generating a concurrent suggestion interval based on a local running state comprises the following steps: Obtaining a local running state, wherein the local running state comprises CPU utilization, available memory proportion, GC activity index, network I / O index, number of requests being processed, request queue length, and part or all of success rate, delay rate and default rate based on historical statistics; Generating a concurrent suggestion interval according to the local running state based on a preset rule or interval generation model.

3. The distributed current limiting method of claim 1, wherein, After the step of determining the local throttling threshold based on the global feasible interval and the local dynamic weight, the method further comprises the following steps: Allocating a tenant-level throttling quota for a tenant under the constraint of the local throttling threshold.

4. The distributed flow limiting method of claim 1 or 3, wherein, After the step of determining the local throttling threshold based on the global feasible interval and the local dynamic weight, the method further comprises the following steps: In the case of receiving a service call request carrying a budget token, parsing the budget token, and making a response decision according to the local throttling threshold, the concurrent credit value and the remaining delay milliseconds obtained by parsing the budget token.

5. The distributed flow limiting method of claim 1, wherein, The method further comprises the following steps: In the case of meeting a preset borrowing condition, allocating a throttling quota of a lender tenant to a borrower tenant, and recording borrowing and lending information, wherein the borrowing and lending information comprises a lender tenant, a borrower tenant, a borrowing and lending quota, a borrowing and lending validity period and a discount factor; In the case that the borrowing and lending validity period expires, or the default rate of the lender tenant exceeds a preset threshold and lasts for a preset duration, gradually returning the borrowing and lending quota from the borrower tenant to the lender tenant in multiple periods according to the discount factor.

6. The distributed flow limiting method of claim 1, wherein, The method further comprises the following steps: After recovery from network partition failure, update the global feasible interval, and perform multi-period gradient update on the local throttling threshold according to the updated global feasible interval, wherein the variation amplitude of the local throttling threshold in a single period is less than or equal to a preset proportion of the current local throttling threshold.

7. A distributed throttling device applied to a service instance of a distributed system, characterized in that, Comprise: An interval management module configured to periodically generate and broadcast concurrent suggestion intervals to other service instances of the distributed system based on local running states, wherein the concurrent suggestion intervals are used to represent the lower and upper limits of the number of concurrent requests that can be safely processed by a service instance in an evaluation period based on its own health status and processing capacity; A CRDT convergence module configured to receive the concurrent suggestion intervals generated by itself and from the other service instances, and perform intersection operation on all the received concurrent suggestion intervals to obtain a global feasible interval, wherein the upper limit of the global feasible interval is the minimum value of the upper limits of all the concurrent suggestion intervals, and the lower limit of the global feasible interval is the maximum value of the lower limits of all the concurrent suggestion intervals; A quota allocation module configured to determine a local throttling threshold based on the global feasible interval and a local dynamic weight; Wherein, the determination of the local throttling threshold based on the global feasible interval and the local dynamic weight comprises: determining a first concurrent threshold by taking the integer part according to the weight proportion of the local dynamic weight and the upper limit value of the global feasible interval; determining a first tail difference margin according to the difference between the upper limit value of the global feasible interval and the sum of the first concurrent thresholds of all the service instances; allocating the first tail difference margin to multiple service instances with the highest dynamic weight one by one in the manner of increasing a unit concurrent threshold for each service instance to obtain the local throttling threshold.

8. A distributed current limiting system comprising a processor and a memory having stored therein a computer program, characterized in that, The processor is configured to implement the distributed throttling method according to any one of claims 1 to 6 when running the computer program.

9. A storage medium having stored therein a computer program, characterized in that The computer program is configured to implement the distributed throttling method according to any one of claims 1 to 6 when the computer program is run.

Images