Throttling method and device of distributed cluster, electronic equipment and medium

Abstract

This specification provides a rate limiting method, apparatus, electronic device, and medium for a distributed cluster. The method includes: if at least one token in the token set is greater than 0, calculating a token deviation value; if the token deviation value is less than a first threshold and the condition that all tokens in the token set are greater than or equal to 0 is not met, reducing the token generation rate corresponding to the token bucket rate limiter with tokens greater than the token number threshold, and increasing the token generation rate corresponding to the token bucket rate limiter with tokens less than the token number threshold; if every token in the token set is less than or equal to 0, calculating a rate deviation value based on the generation rate set; if the rate deviation value is greater than or equal to a second threshold, reducing the token generation rate corresponding to the token bucket rate limiter with a token generation rate greater than the token generation rate threshold, and increasing the token generation rate corresponding to the token bucket rate limiter with a token generation rate less than the token generation rate threshold.

Description

Technical Field

[0001] This specification relates to the field of cloud computing technology, and in particular to rate limiting methods, devices, electronic equipment and media for distributed clusters. Background Technology

[0002] In distributed application scenarios, the diversity and complexity of services hosted on hosts increase significantly, making it crucial to ensure the quality of critical business services and the ability to provide stable system services. To address resource usage issues related to mixed services on hosts, avoid resource contention among specific services, and guarantee the performance of individual or certain services, dynamic flow control for specific services is needed to meet the specific performance goals of certain applications. Therefore, how to implement rate limiting in distributed clusters is a pressing issue that requires a solution. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this specification provides a method, device, electronic device and medium for rate limiting of distributed clusters.

[0004] According to a first aspect of the embodiments of this specification, a rate limiting method for a distributed cluster is provided, the distributed cluster including multiple token bucket rate limiters, the method comprising: obtaining a token count set and a generation rate set, the token count set including the remaining token count of each of the multiple token bucket rate limiters, the generation rate set including the token generation rate of each of the multiple token bucket rate limiters; if at least one token count in the token count set is greater than 0, then calculating a quantity deviation value based on the token count set; and when the quantity deviation value is less than a first threshold and all token counts in the token count set are greater than 0, the method provides a rate limiting method for a distributed cluster including multiple token bucket rate limiters. If the condition of equal to 0 is not met, the token generation rate corresponding to the token bucket rate limiter with a token number greater than the token number threshold is reduced, and the token generation rate corresponding to the token bucket rate limiter with a token number less than the token number threshold is increased; if each token number in the token number set is less than or equal to 0, then a rate deviation value is calculated based on the generation rate set; if the rate deviation value is greater than or equal to a second threshold, the token generation rate corresponding to the token bucket rate limiter with a token generation rate greater than the token generation rate threshold is reduced, and the token generation rate corresponding to the token bucket rate limiter with a token generation rate less than the token generation rate threshold is increased.

[0005] According to a second aspect of the embodiments of this specification, a rate limiting device for a distributed cluster is provided, comprising: an acquisition module, configured to acquire a token count set and a generation rate set, wherein the token count set includes the remaining token count of each of the plurality of token bucket rate limiters, and the generation rate set includes the token generation rate of each of the plurality of token bucket rate limiters; a quantity deviation value calculation module, configured to calculate a quantity deviation value based on the token count set if at least one token count in the token count set is greater than 0; and a first adjustment module, configured to adjust the quantity deviation value when it is less than a first threshold and all token counts in the token count set are greater than or equal to 0. If the conditions are met, the token generation rate corresponding to the token bucket current limiter with a token number greater than the token number threshold is reduced, and the token generation rate corresponding to the token bucket current limiter with a token number less than the token number threshold is increased; the rate deviation value calculation module is used to calculate the rate deviation value based on the generation rate set if each token number in the token number set is less than or equal to 0; the second adjustment module is used to reduce the token generation rate corresponding to the token bucket current limiter with a token generation rate greater than the token generation rate threshold and increase the token generation rate corresponding to the token bucket current limiter with a token generation rate less than the token generation rate threshold when the rate deviation value is greater than or equal to the second threshold.

[0006] According to a third aspect of the embodiments of this specification, an electronic device is provided, comprising:

[0007] processor;

[0008] Memory used to store processor-executable instructions;

[0009] The processor is configured to execute the rate limiting method for a distributed cluster of the first aspect described above or any of its corresponding embodiments.

[0010] According to a fourth aspect of the embodiments of this specification, a rate limiting device for a distributed cluster is provided, comprising: a computer program product including computer instructions, the computer instructions being configured to cause a computer to execute the rate limiting method for a distributed cluster of the first aspect or any corresponding embodiment described above.

[0011] The technical solutions provided in the embodiments of this specification may include the following beneficial effects:

[0012] In the embodiments described in this specification, the rate limiting configuration of each token bucket rate limiter can be dynamically adjusted according to the load of the token bucket rate limiter, thereby effectively controlling the total traffic accessing the workload in the distributed cluster and achieving a balanced load distribution across the entire distributed cluster.

[0013] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Attached Figure Description

[0014] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this specification and, together with the description, serve to explain the principles of this specification.

[0015] Figure 1A This is a schematic diagram of a system architecture illustrated in this specification according to an exemplary embodiment.

[0016] Figure 1B This is a schematic diagram illustrating rate limiting of a distributed cluster according to an exemplary embodiment.

[0017] Figure 2 This is a flowchart illustrating a rate limiting method for a distributed cluster according to an exemplary embodiment.

[0018] Figure 3 This is a flowchart illustrating another rate limiting method for a distributed cluster according to an exemplary embodiment.

[0019] Figure 4 This is a hardware structure diagram of a computer device containing the rate limiting device of a distributed cluster in an embodiment of this specification.

[0020] Figure 5 This is a block diagram illustrating a rate limiting device for a distributed cluster according to an exemplary embodiment. Detailed Implementation

[0021] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.

[0022] The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of this specification. The singular forms “a,” “the,” and “the” as used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0023] It should be understood that although the terms first, second, third, etc., may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0024] The embodiments described in this specification will now be described in detail.

[0025] The following combination Figure 1A This specification describes the system architecture of a distributed cluster to which the rate limiting method and apparatus for distributed clusters can be applied, as exemplified in the embodiments of this specification. It should be noted that Figure 1 is merely an example of a system architecture applicable to the embodiments of this specification, intended to help those skilled in the art understand the technical content of this specification, but it does not imply that the embodiments of this specification cannot be used in other devices, systems, environments, or scenarios.

[0026] Figure 1A This is a schematic diagram of a system architecture illustrated in this specification according to an exemplary embodiment.

[0027] like Figure 1A As shown, this system architecture may include, for example, a server, an agent, and a load balancer. The server can reside on the control node. The server can be used to set the overall rate limit for the distributed cluster and dynamically adjust the agent token bucket configuration (e.g., QoS configuration) based on the agent's current load to achieve balanced load distribution. The agent can reside on the worker node. The agent can be used to set the local token bucket rate limiter on the worker node according to the server configuration, and to listen for and report current load and token bucket-related data.

[0028] The server can detect when worker nodes are offline or when new worker nodes come online, and readjust the rate limiting configurations for each workload to achieve a balanced load distribution.

[0029] After receiving the token bucket configuration, the load balancer can adjust the routing distribution weight of each agent according to the token generation rate of each token bucket, and distribute requests to each agent according to the weight to achieve load balancing that matches the backend processing capacity.

[0030] Optionally, the Server may also include a Limit Dispatcher module responsible for calculating the token bucket configuration for each Agent based on the algorithm and with the goal of optimal load distribution.

[0031] Optionally, the Agent may also include a Limit Controller and a Limiter. The Limit Controller is responsible for receiving the token bucket configuration distributed by the Server and updating the rate limiting settings of the token bucket rate limiter. The Limiter, also known as the local token bucket rate limiter, is responsible for the rate limiting logic and synchronizes the current token bucket information with the cluster.

[0032] Optionally, the load balancer can be a standalone third-party load balancer or a load balancer built into the Gateway. The Gateway can include features for distributing business requests to Agents. In this embodiment, when rate limiting is applied to the distributed cluster, the Gateway load balancer can be adjusted synchronously to optimize request routing at the request end, thereby improving system performance.

[0033] Optionally, Server, Agent, and Gateway can all include a GossipSvc (Popular Protocol Service) for communication between cluster nodes using the Gossip protocol.

[0034] Figure 1B This is a schematic diagram illustrating rate limiting of a distributed cluster according to an exemplary embodiment. For example... Figure 1B As shown, the process of rate limiting for a distributed cluster may include the following steps:

[0035] 1) Users or services set cluster-wide rate limits;

[0036] 2) The server calculates the configuration of each agent's token bucket based on an algorithm, which may include, for example, the token generation rate;

[0037] 3) 3.1: When the Gateway receives the configuration message, it adjusts the load balancing weight of the Load Balancer according to the token bucket rate limiting configuration. The higher the token generation rate, the higher the weight, and correspondingly, the more requests are distributed.

[0038] 3.2: The Agent receives the configuration information and updates the rate limiting settings of the token bucket rate limiter;

[0039] 4) The requesting party initiates a business request;

[0040] 5) Service requests are distributed via the Gateway's load balancer;

[0041] 6) If rate limiting is not triggered, access the backend business load; if rate limiting is triggered, respond with an error indicating that the request has been rate-limited.

[0042] 7) Update token bucket information data, mainly including the current number of tokens and the current token generation rate;

[0043] 8) Synchronize the token bucket information data of the current Agent token bucket rate limiter to the cluster;

[0044] After receiving the token bucket information data synchronized by the Agent, the Server will execute step 2. Steps 2, 3, 6, and 7 will be executed in a loop, continuously and automatically adjusting the token bucket configuration according to the algorithm to continuously optimize the load distribution.

[0045] The rate limiting method for distributed clusters provided in the embodiments of this specification will be described in detail below. For example... Figure 2 As shown, Figure 2 This is a flowchart illustrating a rate limiting method for a distributed cluster according to an exemplary embodiment of this specification. The method is applied to a distributed cluster, and the rate limiting method for a distributed cluster provided in the embodiments of this specification may include the following steps.

[0046] In step 210, obtain the token set and generate the rate set.

[0047] According to embodiments of this specification, the token count set includes the remaining token count of each of the multiple token bucket current limiters, and the generation rate set includes the token generation rate of each of the multiple token bucket current limiters.

[0048] In step 220, if every token in the token set is greater than or equal to 0, proceed to steps 230-240. If every token in the token set is less than 0, proceed to steps 250-260.

[0049] In step 230, calculate the quantity deviation value based on the token set.

[0050] According to embodiments of this specification, for example, the total positive deviation can be obtained by summing the absolute values ​​of each positive value in the token set. Furthermore, the total negative deviation can be obtained by summing the absolute values ​​of each negative value in the token set. Then, the sum of the positive and negative deviations can be calculated to obtain the quantity deviation value.

[0051] In step 240, if the quantity deviation value is greater than or equal to the first threshold, the token generation rate corresponding to the token bucket current limiter with a token number greater than the token number threshold is reduced, and the token generation rate corresponding to the token bucket current limiter with a token number less than the token number threshold is increased.

[0052] According to the embodiments of this specification, the first threshold can be set according to actual needs.

[0053] According to embodiments of this specification, for example, the generation rates in the generation rate set can be summed to obtain an initial total rate.

[0054] Next, the ratio of the total negative deviation to the total positive deviation can be calculated to obtain the first ratio. The product of the first ratio, the initial total rate, and the first preset adjustment ratio is then calculated to obtain the positive adjustment amount. Additionally, the ratio of the total positive deviation to the total negative deviation can be calculated to obtain the second ratio. The product of the second ratio, the initial total rate, and the first preset adjustment ratio is then calculated to obtain the negative adjustment amount. The first preset adjustment ratio is greater than 0 and less than or equal to 1; its specific value can be set according to actual needs.

[0055] Then, for each token bucket rate limiter, if the number of tokens in the token bucket rate limiter is greater than the token number threshold, the token generation rate of the token bucket rate limiter can be adjusted according to a positive adjustment amount. If the number of tokens in the token bucket rate limiter is less than the first threshold, the token generation rate of the token bucket rate limiter can be adjusted according to a negative adjustment amount.

[0056] In step 250, the rate deviation value is calculated based on the generated rate set.

[0057] According to embodiments of this specification, for example, the average value of the token generation rates in the generation rate set can be calculated to obtain the average rate. Then, the absolute value of the difference between each token generation rate in the generation rate set and the average rate is calculated to obtain the individual deviation value of each token generation rate, and the maximum value among the individual deviation values ​​is determined as the rate deviation value.

[0058] In step 260, if the rate deviation value is greater than or equal to the second threshold, the token generation rate corresponding to the token bucket current limiter with a token number greater than the token generation rate threshold is reduced, and the token generation rate corresponding to the token bucket current limiter with a token number less than the token generation rate threshold is increased.

[0059] According to the embodiments of this specification, the second threshold can be set according to actual needs, and the second threshold can be the same as or different from the first threshold.

[0060] According to embodiments of this specification, for example, for each token bucket current limiter, if the rate deviation value corresponding to that token bucket current limiter is greater than or equal to a second threshold, then the product between the token generation rate of that token bucket current limiter and a second preset adjustment ratio is calculated to obtain the maximum adjustment value. If the token generation rate of that token bucket current limiter is greater than the average rate, then the maximum adjustment value is subtracted from the token generation rate of that token bucket current limiter to obtain the initial adjustment rate. The second preset adjustment ratio is greater than 0 and less than or equal to 1, and its specific value can be set according to actual needs. The second preset adjustment ratio can be the same as or different from the first preset adjustment ratio.

[0061] Next, the sum of all preliminary adjustment rates can be calculated to obtain the total preliminary adjustment rate. The ratio between the sum of the token generation rates in the generation rate set and the total preliminary adjustment rate is then calculated to obtain the adjustment factor.

[0062] Then, the token generation rate of multiple token bucket limiters can be adjusted according to the adjustment factor.

[0063] The rate limiting method for distributed clusters according to the embodiments of this specification can dynamically adjust the rate limiting configuration of each token bucket rate limiter based on the load of the token bucket rate limiter, thereby effectively controlling the total traffic accessing the workload in the distributed cluster and achieving a balanced load distribution across the entire distributed cluster.

[0064] Optionally, when the node information of the cluster is refreshed, such as when the cluster is expanded or reduced, i.e. when workloads are removed or new workloads are added, the rate limiting method of the distributed cluster shown in the embodiments of this specification can be automatically triggered to dynamically adjust the traffic limit of the distributed cluster and achieve a balanced load distribution of the entire distributed cluster.

[0065] Optionally, a counter can be set to record the number of consecutive adjustments to the token generation rate. Each time the token generation rate is adjusted (i.e., step 240 or step 260 is executed), the counter is incremented by 1. When the counter reaches a threshold, adjustments to the distributed cluster token generation rate can be paused. Whenever the remaining tokens in all token bucket rate limiters become non-zero, the counter is reset to zero, i.e., the consecutive adjustment count is reset. If adjustments to the distributed cluster token generation rate were previously paused, they resume after the counter resets to zero.

[0066] Optionally, when synchronizing data to the load balancer, the weight of the load balancer can be adjusted according to the processing capacity of the workload. The higher the token generation rate, the higher the weight, thereby achieving load balancing that matches the backend processing capacity.

[0067] Optionally, when adjusting the rate limit, it is possible to adjust it according to a specified adjustment ratio to achieve a gradual and smooth adjustment of the rate limit, avoiding a sudden and sharp increase or decrease in load traffic, so that the backend workload will not cause anomalies due to a sudden increase in requests.

[0068] Alternatively, worker nodes can be rate-limited via their local token bucket rate limiters. This allows normal service to continue even if a front-end local token bucket rate limiter disconnects from the rate-limiting control plane (e.g., the server), and rate limiting can be performed correctly according to the current configuration.

[0069] The following describes a rate limiting method for distributed clusters using another embodiment.

[0070] In this embodiment, when tokens are not exhausted, Algorithm 1 can be applied to adjust the generation rate of the token bucket current limiter. When tokens are exhausted, Algorithm 2 can be applied to adjust the generation rate of the token bucket current limiter.

[0071] For example, in this embodiment, the input to Algorithm 1 may include:

[0072] N represents the number of token bucket current limiters;

[0073] R represents the set of generation rates: [r1, r2, ..., rN], where r1, r2, ..., rN are the current token generation rates of each token bucket rate limiter;

[0074] T represents the set of tokens: [t1, t2, ..., tN], where t1, t2, ..., tN are the number of tokens currently remaining in each token bucket current limiter;

[0075] δ represents the deviation ratio threshold, i.e., the first threshold;

[0076] α represents the adjustment ratio, i.e. the first preset adjustment ratio.

[0077] The output of Algorithm 1 may include:

[0078] R' is the adjusted generation rate array: [r1', r2', ..., rN'].

[0079] Based on this, applying an algorithm to adjust the generation rate of the token bucket current limiter may include, for example, the following steps:

[0080] 1. Calculate the initial total speed using the following formula:

[0081] totalRate = sum(R)

[0082] Where totalRate is the initial total rate, R is the set of generated rates, and sum() is the summation operation.

[0083] 2. Calculate the total positive deviation and total negative deviation using the following formulas:

[0084] totalPositive = sum of positive values ​​in T

[0085] totalNegative = sum of absolute values ​​of negative values ​​in T

[0086] Where totalPositive represents the total positive deviation, totalNegative represents the total negative deviation, T is the set of tokens, sum of positive values ​​in T means calculating the sum of all positive values ​​in T, and sum of absolute values ​​of negative values ​​in T means calculating the sum of the absolute values ​​of all negative values ​​in T.

[0087] 3. Calculate the total deviation value using the following formula:

[0088] totalDeviation = totalPositive + totalNegative

[0089] Where totalDeviation is the total deviation value. If totalDeviation == 0, return R, meaning no adjustment is needed when totalDeviation is 0.

[0090] 4. Calculate the adjustment amount using the following formula:

[0091] positiveRateAdjustment = (totalNegative / totalDeviation) * totalRate* α negativeRateAdjustment = (totalPositive / totalDeviation) * totalRate * α

[0092] Wherein, positiveRateAdjustment is the positive adjustment amount, negativeRateAdjustment is the negative adjustment amount, and α is the first preset adjustment ratio.

[0093] 5. Initialize the adjusted generation rate set R'

[0094] R' = [r1, r2, ..., rN];

[0095] 6. For each ri in R' (i is a positive integer from 1 to N), perform:

[0096] If ti > 0:

[0097] r'i = ri - (positiveRateAdjustment * (ti / totalPositive))

[0098] Otherwise, if ti < 0:

[0099] r'i = ri + (negativeRateAdjustment * (|ti| / totalNegative))

[0100] After execution, we get R' = [r'1, r'2, ..., r'N];

[0101] 7. Calculate the sum of R'.

[0102] sumNewRates = sum(R')

[0103] 8. Normalized R'

[0104] For each r'i in R', execute:

[0105] r'i *= (totalRate / sumNewRates)

[0106] Here, *= indicates the operation of calculating the product before assigning the value.

[0107] 9. Return to R'.

[0108] Subsequently, the corresponding token bucket rate limiter can be configured based on the generation rate of each token recorded in R'.

[0109] For example, in this embodiment, the input to Algorithm 2 may include:

[0110] N represents the number of token bucket current limiters;

[0111] R represents the current set of generation rates: [r1, r2, ..., rN];

[0112] D represents the maximum rate deviation;

[0113] α represents the adjustment ratio, i.e. the second preset adjustment ratio.

[0114] The output of Algorithm 2 may include:

[0115] R' represents the adjusted set of generation rates: [r1', r2', ..., rN'].

[0116] Based on this, applying Algorithm 2 to adjust the generation rate of the token bucket current limiter may include, for example, the following steps:

[0117] 1. Calculate the initial total rate (totalRate) using the following formula.

[0118] totalRate = sum(R)

[0119] 2. Calculate the average rate using the following formula:

[0120] averageRate = totalRate / N;

[0121] 3. Initialize the new generation rate array R':

[0122] R' = [r1, r2, ..., rN];

[0123] 4. Make initial adjustments to each generation rate:

[0124] For i from 1 to N:

[0125] If |R'[i] - averageRate|>D:

[0126] maxAdjustment = α * R' [i]

[0127] If R'[i]>averageRate:

[0128] R'[i] = R'[i]- maxAdjustment

[0129] otherwise:

[0130] R'[i] = R'[i]+ maxAdjustment;

[0131] 5. Calculate the adjusted total rate (totalRate) using the following formula:

[0132] totalRate' = sum(R');

[0133] 6. Calculate the adjustment factor based on the following:

[0134] f = totalRate / totalRate';

[0135] 7. Apply an adjustment factor to adjust the token generation rate:

[0136] For i from 1 to N:

[0137] R'[i] = R'[i]* f;

[0138] 8. Return to R'.

[0139] Subsequently, the corresponding token bucket rate limiter can be configured based on the generation rate of each token recorded in R'.

[0140] like Figure 3 The diagram shown is a flowchart illustrating another rate limiting method for a distributed cluster according to an exemplary embodiment. This embodiment describes a process for rate limiting of a distributed cluster based on the foregoing embodiments, including the following steps:

[0141] Whenever Agent node information is updated in the cluster, the Server is triggered to recalculate the configuration of each Agent token bucket.

[0142] First, determine if there is at least one Agent in the cluster that has remaining tokens.

[0143] If at least one Agent in the cluster has remaining tokens (i.e., tokens are not exhausted), the current token count of all Agents is counted, and the corresponding quantity deviation value is calculated. The calculation method for the quantity deviation value is described above and will not be repeated here. If the condition that all Agents currently have token counts greater than or equal to 0 and the deviation value is less than "Value 1" is not met, the token generation rate of Agents with remaining token counts exceeding the token count threshold is reduced, and the token generation rate of Agents with remaining token counts below the token count threshold is increased. "Value 1" can be set according to actual needs, for example, it can be 5%. If none of the Agents in the cluster have remaining tokens (i.e., tokens are exhausted), it is determined whether N consecutive adjustments have been made (N is a positive integer, which can be set according to actual needs). If N consecutive adjustments have not been made, the current token generation rate of all Agents is counted, and the rate deviation value is calculated. If the rate deviation value is greater than or equal to "Value 2", the token generation rate of Agents with token generation rates exceeding the token generation rate threshold is reduced, and the token generation rate of Agents with token generation rates below the token generation rate threshold is increased. "Value 2" can be set according to actual needs, for example, it can be 5%. Corresponding to the embodiments of the aforementioned methods, this specification also provides embodiments of a rate limiting device for a distributed cluster and the terminals to which it is applied.

[0144] The embodiments of the rate limiting device for distributed clusters described in this specification can be applied to computer devices, such as servers or terminal devices. The device embodiments can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor reading the corresponding computer program instructions from non-volatile memory into memory and executing them. From a hardware perspective, such as... Figure 4 The diagram shown is a hardware structure diagram of a computer device containing the rate limiting device of a distributed cluster according to an embodiment of this specification. Except for... Figure 4In addition to the processor 410, memory 430, network interface 420, and non-volatile memory 440 shown, the server or electronic device where the device 431 is located in the embodiment may also include other hardware depending on the actual function of the computer device, which will not be described in detail here.

[0145] like Figure 5 As shown, Figure 5 This is a block diagram illustrating a rate limiting device for a distributed cluster according to an exemplary embodiment of this specification. The device includes:

[0146] The acquisition module 510 is used to acquire a token count set and a generation rate set. The token count set includes the remaining token count of each token bucket limiter in the multiple token bucket limiters, and the generation rate set includes the token generation rate of each token bucket limiter in the multiple token bucket limiters.

[0147] The quantity deviation value calculation module 520 is used to calculate the quantity deviation value based on the token set if there is at least one token greater than 0 in the token set.

[0148] The first adjustment module 530 is used to reduce the token generation rate corresponding to the token bucket current limiter with a token number greater than the token number threshold and increase the token generation rate corresponding to the token bucket current limiter with a token number less than the token number threshold when the condition that the quantity deviation value is less than the first threshold and all token numbers in the token number set are greater than or equal to 0 is not met.

[0149] The rate deviation value calculation module 540 is used to calculate the rate deviation value based on the generated rate set if each token in the token set is less than or equal to 0.

[0150] The second adjustment module 550 is used to reduce the token generation rate corresponding to the token bucket current limiter whose token generation rate is greater than the token generation rate threshold and increase the token generation rate corresponding to the token bucket current limiter whose token generation rate is less than the token generation rate threshold when the rate deviation value is greater than or equal to the second threshold.

[0151] Optionally, the quantity deviation value calculation module may include:

[0152] The positive deviation total calculation submodule is used to sum each positive value in the token set to obtain the total positive deviation;

[0153] The negative deviation total calculation submodule is used to sum the absolute values ​​of each negative value in the token set to obtain the total negative deviation.

[0154] The first deviation value calculation module is used to calculate the sum of positive and negative deviations to obtain the quantity deviation value.

[0155] Optionally, the first adjustment module may include:

[0156] The initial total rate calculation submodule is used to sum the generation rates in the generation rate set to obtain the initial total rate;

[0157] The first ratio calculation submodule is used to calculate the ratio of the total negative deviation to the total positive deviation, and obtain the first ratio.

[0158] The positive adjustment amount calculation submodule is used to calculate the product between the first ratio, the initial total rate and the first preset adjustment ratio to obtain the positive adjustment amount.

[0159] The second ratio calculation submodule is used to calculate the ratio of the total positive deviation to the total negative deviation, and obtain the second ratio.

[0160] The negative adjustment calculation submodule is used to calculate the product between the second ratio, the initial total rate and the first preset adjustment ratio to obtain the negative adjustment amount.

[0161] The first adjustment submodule is used to adjust the token generation rate of each token bucket rate limiter by a positive adjustment amount if the number of tokens in the token bucket rate limiter is greater than the token number threshold, and by a negative adjustment amount if the number of tokens in the token bucket rate limiter is less than the token number threshold.

[0162] Optionally, the rate deviation calculation module may include:

[0163] The average rate calculation submodule is used to calculate the average value of the token generation rates in the generation rate set to obtain the average rate.

[0164] The second deviation value calculation submodule is used to calculate the absolute value of the difference between the generation rate of each token in the generation rate set and the average rate, to obtain the individual deviation value of each token generation rate, and to determine the maximum value among the individual deviation values ​​as the rate deviation value.

[0165] Optionally, the second adjustment module may include:

[0166] The maximum adjustment value calculation submodule is used to calculate the product between the token generation rate of the token bucket current limiter and the second preset adjustment ratio for each token bucket current limiter if the rate deviation value corresponding to the token bucket current limiter is greater than or equal to the second threshold, so as to obtain the maximum adjustment value.

[0167] The preliminary adjustment rate calculation submodule is used to subtract the maximum adjustment value from the token generation rate of the token bucket current limiter if the token generation rate of the token bucket current limiter is greater than the average rate, so as to obtain the preliminary adjustment rate.

[0168] The preliminary adjustment rate calculation submodule is used to add the maximum adjustment value to the token generation rate of the token bucket current limiter if the token generation rate of the token bucket current limiter is less than or equal to the average rate, so as to obtain the preliminary adjustment rate.

[0169] The preliminary adjustment total rate calculation submodule is used to calculate the sum of the preliminary adjustment rates to obtain the preliminary adjustment total rate;

[0170] The adjustment factor calculation submodule is used to calculate the ratio between the sum of the token generation rates in the generation rate set and the initial adjustment total rate to obtain the adjustment factor.

[0171] The second adjustment submodule is used to adjust the token generation rate of multiple token bucket limiters according to the adjustment factor.

[0172] The rate limiting device for the distributed cluster according to the embodiments of this specification can dynamically adjust the rate limiting configuration of each token bucket rate limiter according to the load of the token bucket rate limiter, thereby effectively controlling the total traffic of accessing the workload in the distributed cluster and enabling a balanced load distribution throughout the distributed cluster.

[0173] Accordingly, this specification also provides an electronic device, which includes a processor; a memory for storing processor-executable instructions; wherein the processor is configured to: an acquisition module for acquiring a token count set and a generation rate set, the token count set including the remaining token count of each token bucket current limiter in a plurality of token bucket current limiters, and the generation rate set including the token generation rate of each token bucket current limiter in a plurality of token bucket current limiters; a quantity deviation value calculation module for calculating a quantity deviation value based on the token count set if at least one token count in the token count set is greater than 0; and a first adjustment module for adjusting the quantity deviation value when it is less than a first threshold and all token counts in the token count set are greater than 0. If the condition of equal to 0 is not met, the token generation rate corresponding to the token bucket current limiter with a token number greater than the token number threshold is reduced, and the token generation rate corresponding to the token bucket current limiter with a token number less than the token number threshold is increased; the rate deviation value calculation module is used to calculate the rate deviation value based on the generation rate set if each token number in the token number set is less than or equal to 0; the second adjustment module is used to reduce the token generation rate corresponding to the token bucket current limiter with a token generation rate greater than the token generation rate threshold and increase the token generation rate corresponding to the token bucket current limiter with a token generation rate less than the token generation rate threshold when the rate deviation value is greater than or equal to the second threshold.

[0174] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0175] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the solution in this specification according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0176] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0177] Other embodiments of this specification will readily occur to those skilled in the art upon consideration of the specification and practice of the invention claimed herein. This specification is intended to cover any variations, uses, or adaptations that follow the general principles of this specification and include common knowledge or customary techniques in the art not claimed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this specification are indicated by the following claims.

[0178] It should be understood that this specification is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this specification is limited only by the appended claims.

[0179] The above description is merely a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

Claims

1. A rate limiting method for a distributed cluster, characterized in that, The distributed cluster includes multiple token bucket rate limiters, and the method includes: Obtain a set of token counts and a set of generation rates. The set of token counts includes the remaining token count of each token bucket limiter among the plurality of token bucket limiters, and the set of generation rates includes the token generation rate of each token bucket limiter among the plurality of token bucket limiters. If there is at least one token in the token set that is greater than 0, then calculate the quantity deviation value based on the token set; If the condition that the quantity deviation value is less than the first threshold and all tokens in the token set are greater than or equal to 0 is not met, the token generation rate corresponding to the token bucket current limiter with a token number greater than the token number threshold is reduced, and the token generation rate corresponding to the token bucket current limiter with a token number less than the token number threshold is increased. If each token in the token set is less than or equal to 0, then the rate deviation value is calculated based on the generation rate set. If the rate deviation value is greater than or equal to the second threshold, the token generation rate corresponding to the token bucket limiter with a token generation rate greater than the token generation rate threshold is reduced, and the token generation rate corresponding to the token bucket limiter with a token generation rate less than the token generation rate threshold is increased.

2. The method according to claim 1, characterized in that, The step of calculating the quantity deviation value based on the set of tokens includes: The total positive deviation is obtained by summing the positive values ​​in the set of tokens. The total negative deviation is obtained by summing the absolute values ​​of each negative value in the set of tokens. The sum of the positive deviation and the negative deviation is calculated to obtain the quantity deviation value.

3. The method according to claim 2, characterized in that, The method of reducing the token generation rate corresponding to the token bucket rate limiter with a token count greater than the token count threshold, and increasing the token generation rate corresponding to the token bucket rate limiter with a token count less than the token count threshold, includes: The initial total rate is obtained by summing the generation rates in the set of generation rates. Calculate the ratio of the total negative deviation to the total positive deviation to obtain the first ratio; Calculate the product of the first ratio, the initial total rate, and the first preset adjustment ratio to obtain a positive adjustment amount; Calculate the ratio of the total positive deviation to the total negative deviation to obtain a second ratio; Calculate the product of the second ratio, the initial total rate, and the first preset adjustment ratio to obtain a negative adjustment amount; For each token bucket rate limiter, if the number of tokens in the token bucket rate limiter is greater than the token number threshold, the token generation rate of the token bucket rate limiter is reduced according to the positive adjustment amount; if the number of tokens in the token bucket rate limiter is less than the token number threshold, the token generation rate of the token bucket rate limiter is increased according to the negative adjustment amount.

4. The method according to claim 1, characterized in that, The step of calculating the rate deviation value based on the generated rate set includes: Calculate the average value of the token generation rates in the generation rate set to obtain the average rate; Calculate the absolute value of the difference between the generation rate of each token in the generation rate set and the average rate to obtain the individual deviation value of each token generation rate, and determine the maximum value among the individual deviation values ​​as the rate deviation value.

5. The method according to claim 4, characterized in that, The method of reducing the token generation rate corresponding to a token bucket rate limiter with a token generation rate greater than a token generation rate threshold, and increasing the token generation rate corresponding to a token bucket rate limiter with a token generation rate less than a token generation rate threshold, includes: For each token bucket current limiter, if the rate deviation value corresponding to the token bucket current limiter is greater than or equal to the second threshold, the product between the token generation rate of the token bucket current limiter and the second preset adjustment ratio is calculated to obtain the maximum adjustment value. If the token generation rate of the token bucket current limiter is greater than the average rate, then the maximum adjustment value is subtracted from the token generation rate of the token bucket current limiter to obtain the preliminary adjustment rate. If the token generation rate of the token bucket current limiter is less than or equal to the average rate, then the token generation rate of the token bucket current limiter is added to the maximum adjustment value to obtain the initial adjustment rate; Calculate the sum of the initial adjustment rates to obtain the total initial adjustment rate; The adjustment factor is obtained by calculating the ratio between the sum of the token generation rates in the generation rate set and the initial adjustment total rate. The token generation rate of the plurality of token bucket limiters is adjusted according to the adjustment factor.

6. A rate limiting device for a distributed cluster, characterized in that, The distributed cluster includes multiple token bucket rate limiters, and the device includes: The acquisition module is used to acquire a token count set and a generation rate set. The token count set includes the remaining token count of each token bucket limiter among the plurality of token bucket limiters, and the generation rate set includes the token generation rate of each token bucket limiter among the plurality of token bucket limiters. The quantity deviation value calculation module is used to calculate the quantity deviation value based on the token number set if there is at least one token number greater than 0 in the token number set. The first adjustment module is used to reduce the token generation rate corresponding to the token bucket current limiter with a token number greater than the token number threshold and increase the token generation rate corresponding to the token bucket current limiter with a token number less than the token number threshold when the condition that the quantity deviation value is less than the first threshold and all token numbers in the token number set are greater than or equal to 0 is not met. The rate deviation value calculation module is used to calculate the rate deviation value based on the generated rate set if each token in the token set is less than or equal to 0. The second adjustment module is used to reduce the token generation rate corresponding to the token bucket current limiter whose token generation rate is greater than the token generation rate threshold, and increase the token generation rate corresponding to the token bucket current limiter whose token generation rate is less than the token generation rate threshold, when the rate deviation value is greater than or equal to the second threshold.

7. The apparatus according to claim 6, characterized in that, The quantity deviation value calculation module includes: The positive deviation total calculation submodule is used to sum each positive value in the token set to obtain the total positive deviation; The negative deviation total calculation submodule is used to sum the absolute values ​​of each negative value in the token set to obtain the total negative deviation. The first deviation value calculation module is used to calculate the sum of the positive deviation and the negative deviation to obtain the quantity deviation value.

8. The apparatus according to claim 7, characterized in that, The first adjustment module includes: The initial total rate calculation submodule is used to sum the generation rates in the generation rate set to obtain the initial total rate; The first ratio calculation submodule is used to calculate the ratio of the total negative deviation to the total positive deviation to obtain the first ratio. The positive adjustment amount calculation submodule is used to calculate the product between the first ratio, the initial total rate and the first preset adjustment ratio to obtain the positive adjustment amount; The second ratio calculation submodule is used to calculate the ratio of the total positive deviation to the total negative deviation to obtain the second ratio. The negative adjustment amount calculation submodule is used to calculate the product between the second ratio, the initial total rate and the first preset adjustment ratio to obtain the negative adjustment amount; The first adjustment submodule is used to, for each token bucket rate limiter, reduce the token generation rate of the token bucket rate limiter according to the positive adjustment amount if the number of tokens of the token bucket rate limiter is greater than the token number threshold; and increase the token generation rate of the token bucket rate limiter according to the negative adjustment amount if the number of tokens of the token bucket rate limiter is less than the token number threshold.

9. The apparatus according to claim 6, characterized in that, The rate deviation calculation module includes: The average rate calculation submodule is used to calculate the average value of the token generation rates in the generation rate set to obtain the average rate. The second deviation value calculation submodule is used to calculate the absolute value of the difference between the generation rate of each token in the generation rate set and the average rate, to obtain the individual deviation value of each token generation rate, and to determine the maximum value among the individual deviation values ​​as the rate deviation value.

10. The apparatus according to claim 9, characterized in that, The second adjustment module includes: The maximum adjustment value calculation submodule is used to calculate the product between the token generation rate of the token bucket current limiter and the second preset adjustment ratio for each token bucket current limiter if the rate deviation value corresponding to the token bucket current limiter is greater than or equal to the second threshold, so as to obtain the maximum adjustment value. The preliminary adjustment rate calculation submodule is used to subtract the maximum adjustment value from the token generation rate of the token bucket current limiter if the token generation rate of the token bucket current limiter is greater than the average rate, so as to obtain the preliminary adjustment rate. The preliminary adjustment rate calculation submodule is used to add the maximum adjustment value to the token generation rate of the token bucket current limiter if the token generation rate of the token bucket current limiter is less than or equal to the average rate, so as to obtain the preliminary adjustment rate. The preliminary adjustment total rate calculation submodule is used to calculate the sum of the preliminary adjustment rates to obtain the preliminary adjustment total rate; The adjustment factor calculation submodule is used to calculate the ratio between the sum of the token generation rates in the generation rate set and the initial adjustment total rate to obtain the adjustment factor. The second adjustment submodule is used to adjust the token generation rate of the plurality of token bucket current limiters according to the adjustment factor.

11. An electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to perform the method of any one of claims 1 to 5.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the method of any one of claims 1 to 5.

Images